2023-10-21T04:09:11.026+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=31fd475f22d8ee0da906c31968c91305 -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:09:11.027+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:09:11.027+05:30 [Info] Setting ipv6=false 2023-10-21T04:09:11.078+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:09:11.132+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:09:11.142+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:09:11.142+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:09:11.142+05:30 [Info] Setting log level to Info 2023-10-21T04:09:11.142+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:09:11.142+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:09:11.182+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:09:11.183+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:09:11.225+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:09:11 audit: created new audit service 2023-10-21T04:09:11.778+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:09:11.778+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:09:11.778+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:09:11.780+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:09:11.780+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:09:11.780+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:09:11.835+05:30 [Info] security context initialized 2023-10-21T04:09:11.835+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9120:19120 9499:19499 9999:9999] 2023-10-21T04:09:11.835+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:09:11.835+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:09:11.835+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:09:11.835+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:09:11.840+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:09:11.840+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:09:11.840+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:09:11.842+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:09:11.857+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:09:11.859+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:09:11.859+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:09:11.865+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:09:11.869+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:09:12.021+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:09:12.022+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:09:12.022+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:09:12.024+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 39] will report aggregate recorder stats every 5m0s 2023-10-21T04:09:12.037+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:09:12.037+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:09:12.048+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:09:12.052+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 258 us 2023-10-21T04:09:12.050+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:09:12.050+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:09:12.050+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:09:12.051+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:09:12.051+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:09:12.054+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:09:12.066+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:09:12.077+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:09:12.183+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:09:12.183+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:09:12.183+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:12.215+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:09:12.215+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:09:12.215+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:09:12.215+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:09:12.215+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:09:12.216+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerId Value 31fd475f22d8ee0da906c31968c91305 2023-10-21T04:09:12.221+05:30 [Info] Indexer Id 31fd475f22d8ee0da906c31968c91305 2023-10-21T04:09:12.223+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:09:12.223+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:09:12.253+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:09:12.253+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:09:12.281+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 181] updating cluster rev epoch 0 -> 1 2023-10-21T04:09:12.281+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 181] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:09:12.310+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 182] bucketNames has been updated (was map[], now map[bucket_%2:7007f0ab442ac00e52d057a30cd22644 bucket_1:db19c9050b4ab1232e6bc6654011618f]) 2023-10-21T04:09:12.310+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 183] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:09:12.311+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 184] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:09:12.350+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:09:12.350+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:09:12.354+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:09:12.379+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:09:12.379+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:09:12.413+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 182] updated cluster rebalancing status to true 2023-10-21T04:09:12.413+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:09:12.414+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:09:12.414+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:09:12.415+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 241] updating rebalancing status to true 2023-10-21T04:09:12.421+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:09:12.421+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:09:12.421+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:09:12.421+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning false RebalanceToken 2023-10-21T04:09:12.421+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:09:12.421+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:09:12.421+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:09:12.422+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:09:12.423+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:09:12.431+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap , elapsed: 8.029429ms 2023-10-21T04:09:12.431+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 31fd475f22d8ee0da906c31968c91305 2023-10-21T04:09:12.431+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:09:12.431+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:09:12.431+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager false 2023-10-21T04:09:12.431+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:09:12.431+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:09:12.432+05:30 [Error] RebalanceServiceManager::getCachedIndexerNodeUUIDs: 127.0.0.1:9120/getCachedIndexerNodeUUIDs returned err: Unable to find given hostport in cbauth database: `127.0.0.1:9120' 2023-10-21T04:09:12.432+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:09:12.432+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 2023-10-21T04:09:12.435+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305. Err projector.topicMissing 2023-10-21T04:09:12.435+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:09:12.435+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:09:12.435+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 2023-10-21T04:09:12.436+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305. Err projector.topicMissing 2023-10-21T04:09:12.436+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:09:12.436+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:09:12.436+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:12.436+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:09:12.446+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:09:12.447+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:09:12.449+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:09:12.453+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:09:12.453+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:09:12.459+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:09:12.460+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:09:12.460+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:09:12.460+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/stats: no such file or directory 2023-10-21T04:09:12.461+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:09:12.461+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:12.462+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:09:12.462+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:09:12.462+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 31fd475f22d8ee0da906c31968c91305 2023-10-21T04:09:12.462+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{31fd475f22d8ee0da906c31968c91305 7060000 } 2023-10-21T04:09:12.470+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:09:12.470+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:09:12.470+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:09:12.470+05:30 [Info] progress stats collection done. 2023-10-21T04:09:12.470+05:30 [Info] send progress stats to clients 2023-10-21T04:09:12.470+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:09:12.476+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:09:12.477+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:09:12.479+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:09:12.479+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:09:12.481+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:09:12.484+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:09:12.486+05:30 [Info] Setting log level to Info 2023-10-21T04:09:12.486+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:09:12.488+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:09:12.495+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:09:12.495+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:09:12.524+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:09:12.524+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:09:12.524+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:09:12.535+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:09:12.696+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {15e5cffff68b4b70e641b53958f9ea44 [] topology-change-rebalance [{{31fd475f22d8ee0da906c31968c91305 7060000 } recovery-full} {{85232c9ba3e4451b95e9e6fc8337e027 7060000 } recovery-full}] [{281b41507b1f937a1b6c37bc4027eef6 7060000 } {5020693e9ae5c52168d414b02f922ad8 7060000 }]} 2023-10-21T04:09:12.711+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:09:12.732+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57026 2023-10-21T04:09:12.740+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57028 2023-10-21T04:09:12.784+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57032 2023-10-21T04:09:12.821+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57036 2023-10-21T04:09:12.834+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57040 2023-10-21T04:09:12.849+05:30 [Info] security context initialized 2023-10-21T04:09:12.849+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:09:12.873+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:09:12.934+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 196.555978ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.945+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 194.839µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.947+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 123.336231ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.947+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 8.534µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.953+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 167.155344ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.954+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 8.838µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.956+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 209.775347ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.963+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 10.541µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.973+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:09:12.973+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:12.973+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:12.973+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:09:12.974+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 136.66605ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.975+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 11.208µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:12.989+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:09:12.989+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:09:12.993+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:09:12.995+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:09:12.995+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:09:12.999+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:13.014+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:09:13.059+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:09:13.059+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:09:13.069+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:09:13.101+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40458" doAuth() ... 2023-10-21T04:09:13.102+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40458" doReceive() ... 2023-10-21T04:09:13.105+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:09:13.105+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:09:13.127+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57118 2023-10-21T04:09:13.129+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:09:13.135+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40464" doAuth() ... 2023-10-21T04:09:13.136+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40464" doReceive() ... 2023-10-21T04:09:13.145+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40472" doAuth() ... 2023-10-21T04:09:13.149+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40474" doAuth() ... 2023-10-21T04:09:13.151+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40474" doReceive() ... 2023-10-21T04:09:13.152+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40472" doReceive() ... 2023-10-21T04:09:13.177+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 45.379202ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:13.177+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:09:13.177+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:09:13.178+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 10.857µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:13.191+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40490" doAuth() ... 2023-10-21T04:09:13.192+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40490" doReceive() ... 2023-10-21T04:09:13.206+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:09:13.234+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:09:13.238+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:09:13.239+05:30 [Info] initialized currmeta 58 force true 2023-10-21T04:09:13.239+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 2023-10-21T04:09:13.239+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.239+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0] 4894899676975150934:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0] 4894899676975150934:[0] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0] 4894899676975150934:[0] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9246878583854294580:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 8899653570021014071:[0] 9246878583854294580:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 8899653570021014071:[0] 9246878583854294580:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 8899653570021014071:[0] 9246878583854294580:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 6796752657857722550:[0] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 15710672812788661570:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 6796752657857722550:[0] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 15710672812788661570:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[3513301448177380059:[0] 4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 6393061446487756912:[2 3 4 5 6 7 8 1] 6796752657857722550:[0] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 15710672812788661570:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[3513301448177380059:[0] 4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 5299824419646640495:[4 5 6 7 8 1 2 3] 6393061446487756912:[2 3 4 5 6 7 8 1] 6796752657857722550:[0] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 15710672812788661570:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[3513301448177380059:[0] 3841645022501780362:[0] 4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 5299824419646640495:[4 5 6 7 8 1 2 3] 6393061446487756912:[2 3 4 5 6 7 8 1] 6796752657857722550:[0] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 15710672812788661570:[0] 16398781184179534607:[0] 18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[3513301448177380059:[0] 3841645022501780362:[0] 4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 5299824419646640495:[4 5 6 7 8 1 2 3] 6393061446487756912:[2 3 4 5 6 7 8 1] 6796752657857722550:[0] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.240+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0] 3052172645459386575:[3 4 5 6 7 8 1 2] 8899653570021014071:[0] 9010726441942672253:[5 6 7 8 1 2 3 4] 9246878583854294580:[0] 11675990026580235111:[0] 13718648355797887482:[0] 15710672812788661570:[0] 16398781184179534607:[0] 18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841377371176481, Old rollbacktime: 0 {Index inst : [partitions]}: map[3513301448177380059:[0] 3841645022501780362:[0] 4362827867398787577:[0] 4503814398859864266:[0] 4894899676975150934:[0] 5299824419646640495:[4 5 6 7 8 1 2 3] 6393061446487756912:[2 3 4 5 6 7 8 1] 6796752657857722550:[0] 8205593268191821683:[6 7 8 1 2 3 4 5] 16983881020783103544:[0]]. New rollbacktime: 1697841254049695699, Old rollbacktime: 0 2023-10-21T04:09:13.249+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:09:13.249+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:09:13.249+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:09:13.250+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:09:13.259+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:09:13.259+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:09:13.259+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:09:13.260+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:09:13.260+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40504" doAuth() ... 2023-10-21T04:09:13.261+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40504" doReceive() ... 2023-10-21T04:09:13.267+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:09:13.267+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:09:13.267+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:09:13.269+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:09:13.291+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57160 2023-10-21T04:09:13.299+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:09:13.299+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:09:13.299+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:09:13.301+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:09:13.302+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:09:13.302+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:09:13.302+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:09:13.302+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:09:13.302+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:09:13.302+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:09:13.302+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:09:13.307+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 3.502645ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:13.308+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.594µs 2023-10-21T04:09:13.308+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:09:13.308+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:09:13.308+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:09:13.308+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:09:13.311+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:09:13.311+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:13.311+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 3.436121ms 2023-10-21T04:09:13.311+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:09:13.311+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.991241ms len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:13.311+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 615.606801ms in RebalanceServiceManager::prepareRebalance: 2023-10-21T04:09:13.311+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:09:13.311+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:13.312+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:13.313+05:30 [Info] Indexer::ReadMemstats Time Taken 1.07339ms 2023-10-21T04:09:13.313+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 7.464µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:13.314+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.276µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:13.318+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:09:13.318+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:09:13.360+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40518" doAuth() ... 2023-10-21T04:09:13.362+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40518" doReceive() ... 2023-10-21T04:09:13.387+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {15e5cffff68b4b70e641b53958f9ea44 [] topology-change-rebalance [{{31fd475f22d8ee0da906c31968c91305 7060000 } recovery-full} {{85232c9ba3e4451b95e9e6fc8337e027 7060000 } recovery-full}] [{281b41507b1f937a1b6c37bc4027eef6 7060000 } {5020693e9ae5c52168d414b02f922ad8 7060000 }]} 2023-10-21T04:09:13.391+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:09:13.419+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"31fd475f22d8ee0da906c31968c91305","RebalId":"15e5cffff68b4b70e641b53958f9ea44","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:09:13.421+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{31fd475f22d8ee0da906c31968c91305 15e5cffff68b4b70e641b53958f9ea44 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:09:13.424+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{31fd475f22d8ee0da906c31968c91305 15e5cffff68b4b70e641b53958f9ea44 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:09:13.438+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 2 NodeUUIDs: [5020693e9ae5c52168d414b02f922ad8 281b41507b1f937a1b6c37bc4027eef6] 2023-10-21T04:09:13.438+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 3 NodeUUIDs: [5020693e9ae5c52168d414b02f922ad8 281b41507b1f937a1b6c37bc4027eef6 31fd475f22d8ee0da906c31968c91305] 2023-10-21T04:09:13.439+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 3 NodeUUIDs: [5020693e9ae5c52168d414b02f922ad8 281b41507b1f937a1b6c37bc4027eef6 31fd475f22d8ee0da906c31968c91305] 2023-10-21T04:09:13.439+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:13.439+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:13.439+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:09:13.446+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:09:13.445+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 237] aggregate storage info: map[bucket_%2:map[kv:5617856 index:4497804] bucket_1:map[kv:6367424 index:8276538]] 2023-10-21T04:09:13.461+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:09:13.461+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:09:13.462+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:09:13.470+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:09:13.475+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[9664682240473767939:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[9664682240473767939:[4 5 6 7 8 1 2 3] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[9664682240473767939:[4 5 6 7 8 1 2 3] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 3810060733504350542:[0] 13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[6 7 8 1 2 3 4 5] 2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 {Index inst : [partitions]}: map[1532172649864318584:[6 7 8 1 2 3 4 5] 2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[6 7 8 1 2 3 4 5] 2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[6 7 8 1 2 3 4 5] 2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 2483169159760864439:[0] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 2483169159760864439:[0] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 14607307599259773249:[0] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 {Index inst : [partitions]}: map[1532172649864318584:[6 7 8 1 2 3 4 5] 2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 2483169159760864439:[0] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 14607307599259773249:[0] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 {Index inst : [partitions]}: map[1532172649864318584:[6 7 8 1 2 3 4 5] 2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 4004220452657849790:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 2023-10-21T04:09:13.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[6 7 8 1 2 3 4 5] 2170849387178940761:[3 4 5 6 7 8 1 2] 3292093078969249751:[0] 3810060733504350542:[0] 4004220452657849790:[0] 11160011617777336020:[0] 13258659765240018889:[0] 16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841254065587323, Old rollbacktime: 0 {Index inst : [partitions]}: map[80911002600697866:[3 4 5 6 7 8 1 2] 2483169159760864439:[0] 4030261110898422634:[0] 7410154703733445404:[5 6 7 8 1 2 3 4] 9664682240473767939:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 14607307599259773249:[0] 14780363202971367008:[0] 17128008689111072444:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841377369803110, Old rollbacktime: 0 2023-10-21T04:09:13.678+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:09:13.812+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:09:14.431+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:09:14.438+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:09:14.438+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:09:14.446+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:09:14.446+05:30 [Info] NewShardRebalancer nodeId 31fd475f22d8ee0da906c31968c91305 rebalToken &{31fd475f22d8ee0da906c31968c91305 15e5cffff68b4b70e641b53958f9ea44 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:09:14.447+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:09:14.447+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.059374744s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:09:14.447+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{31fd475f22d8ee0da906c31968c91305 15e5cffff68b4b70e641b53958f9ea44 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:09:14.447+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:14.447+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:14.447+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:09:14.449+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:09:14.449+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:14.449+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:09:14.449+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:14.453+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:09:14.453+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:09:14.557+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{281b41507b1f937a1b6c37bc4027eef6 281b41507b1f937a1b6c37bc4027eef6 plasma map[] [{24 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_secondary 4860549750398470299 [{15710672812788661570 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary_defer 15968368593314043432 [{17161083103924027797 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c1 #primary 5197198800322775808 [{13718648355797887482 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c1 #primary_defer 17878371269610391239 [{7287869093983910291 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned 14340086159258068923 [{3052172645459386575 4 1 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 15649171661394925140 [{1955535609704140162 1 0 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 0 1 false plasma 0}]}]} {24 bucket_%2 _default c2% [{bucket_%2 _default c2% idx_secondary 14741949230702299196 [{8899653570021014071 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c2% idx_secondary_defer 8669649359784409226 [{6213797968262183765 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c2% #primary 7874378322029374196 [{11675990026580235111 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c2% #primary_defer 1844709904168202817 [{1938940178723796232 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned 15991242037348127852 [{9010726441942672253 4 1 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned_defer 2374548411389959576 [{16044079815412989935 1 0 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 0 0 false plasma 0}]}]} {31 bucket_1 _default c1 [{bucket_1 _default c1 idx_secondary 14889401051926000176 [{3841645022501780362 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary_defer 18323460706027642575 [{14991206072146508370 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c1 #primary 1571318351652255745 [{6796752657857722550 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c1 #primary_defer 17489537288579938718 [{9134754361302452675 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned 16013578430151540990 [{5299824419646640495 4 1 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 2872726210871813325 [{18089525033132576654 1 0 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 0 0 false plasma 0}]}]} {14 bucket_1 _system _query [{bucket_1 _system _query #primary 12833670934079048728 [{4894899676975150934 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630] []}] 1 0 0 0 false plasma 0}]}]} {23 bucket_1 _default _default [{bucket_1 _default _default idx_secondary 14292583894719865678 [{4362827867398787577 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default _default idx_secondary_defer 15945117906390072636 [{10455069697192970620 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default _default #primary 12338036531800130039 [{4503814398859864266 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default _default #primary_defer 11619165464560280276 [{11158580445292211864 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default _default idx_partitioned 573521267164877540 [{8205593268191821683 4 1 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default _default idx_partitioned_defer 17656144409930763141 [{7291286939424384589 1 0 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 0 1 false plasma 0}]}]} {24 bucket_1 _default c2% [{bucket_1 _default c2% idx_secondary 12140458060295645238 [{3513301448177380059 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c2% idx_secondary_defer 6985049432225870291 [{4138705102427425280 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c2% #primary 5103684059183126577 [{16983881020783103544 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c2% #primary_defer 9619308266130016313 [{11484519016652644658 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c2% idx_partitioned 14026476026078401783 [{6393061446487756912 4 1 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c2% idx_partitioned_defer 10585832761158485697 [{9358717944830195392 1 0 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 0 1 false plasma 0}]}]} {23 bucket_%2 _default _default [{bucket_%2 _default _default idx_secondary 12027054384338714804 [{9246878583854294580 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default _default idx_secondary_defer 18046371353728341033 [{18184041269291378080 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default _default #primary 8857269605176042330 [{16398781184179534607 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default _default #primary_defer 13856704970496601958 [{15283305021611177781 1 0 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default _default idx_partitioned 16162012140118277065 [{18051244886420988269 4 1 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default _default idx_partitioned_defer 15431181238515168303 [{10574086758347311295 1 0 [{1 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3674078063353481907 [{360999437388707481 4 1 [{0 0 {[{0 0 281b41507b1f937a1b6c37bc4027eef6}]} {[] []} [10890719122371003479] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {5020693e9ae5c52168d414b02f922ad8 5020693e9ae5c52168d414b02f922ad8 plasma map[] [{24 bucket_1 _default c2% [{bucket_1 _default c2% idx_secondary 12140458060295645238 [{4004220452657849790 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c2% idx_secondary_defer 6985049432225870291 [{11756349460704871322 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c2% #primary 5103684059183126577 [{11160011617777336020 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c2% #primary_defer 9619308266130016313 [{4435591218553988516 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c2% idx_partitioned 14026476026078401783 [{2170849387178940761 4 1 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c2% idx_partitioned_defer 10585832761158485697 [{7755999278497385093 1 0 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 0 0 false plasma 0}]}]} {23 bucket_%2 _default _default [{bucket_%2 _default _default idx_secondary 12027054384338714804 [{11400459767051699632 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default _default idx_secondary_defer 18046371353728341033 [{8446412942681787061 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default _default #primary 8857269605176042330 [{17729593977321451597 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default _default #primary_defer 13856704970496601958 [{8880461853743568138 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default _default idx_partitioned 16162012140118277065 [{80911002600697866 4 1 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default _default idx_partitioned_defer 15431181238515168303 [{2921784364458449578 1 0 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3674078063353481907 [{14780363202971367008 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837] []}] 1 0 0 1 false plasma 0}]}]} {24 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_secondary 4860549750398470299 [{4030261110898422634 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary_defer 15968368593314043432 [{325705934323625324 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c1 #primary 5197198800322775808 [{14607307599259773249 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c1 #primary_defer 17878371269610391239 [{15131823346681830059 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned 14340086159258068923 [{7410154703733445404 4 1 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 15649171661394925140 [{12043401624902403496 1 0 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 0 0 false plasma 0}]}]} {24 bucket_%2 _default c2% [{bucket_%2 _default c2% idx_secondary 14741949230702299196 [{2483169159760864439 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c2% idx_secondary_defer 8669649359784409226 [{14910157020177557388 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c2% #primary 7874378322029374196 [{17128008689111072444 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c2% #primary_defer 1844709904168202817 [{17037906325662037075 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned 15991242037348127852 [{9664682240473767939 4 1 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned_defer 2374548411389959576 [{249283441109236971 1 0 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 0 1 false plasma 0}]}]} {31 bucket_1 _default c1 [{bucket_1 _default c1 idx_secondary 14889401051926000176 [{18201836893981255880 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary_defer 18323460706027642575 [{2288923528993975684 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c1 #primary 1571318351652255745 [{17148287981272169168 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c1 #primary_defer 17489537288579938718 [{3985672602331234807 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned 16013578430151540990 [{16726320877690809560 4 1 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 2872726210871813325 [{2307463109245789631 1 0 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 0 1 false plasma 0}]}]} {14 bucket_1 _system _query [{bucket_1 _system _query #primary 12833670934079048728 [{13258659765240018889 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314] []}] 1 0 0 1 false plasma 0}]}]} {23 bucket_1 _default _default [{bucket_1 _default _default idx_secondary 14292583894719865678 [{3292093078969249751 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default _default idx_secondary_defer 15945117906390072636 [{11739645294255241349 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default _default #primary 12338036531800130039 [{3810060733504350542 4 1 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default _default #primary_defer 11619165464560280276 [{7597064108789640455 1 0 [{0 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default _default idx_partitioned 573521267164877540 [{1532172649864318584 4 1 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default _default idx_partitioned_defer 17656144409930763141 [{13953156762317267443 1 0 [{1 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 0 {[{0 0 5020693e9ae5c52168d414b02f922ad8}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 0 0 false plasma 0}]}]}] [DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {31fd475f22d8ee0da906c31968c91305 31fd475f22d8ee0da906c31968c91305 plasma map[] [] [] 0 0 0 false} {85232c9ba3e4451b95e9e6fc8337e027 85232c9ba3e4451b95e9e6fc8337e027 plasma map[] [] [] 0 0 0 false}] map[]} 2023-10-21T04:09:14.661+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41922 2023-10-21T04:09:14.663+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:09:14.663+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:09:14.663+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:09:14.663+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:09:14.664+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41922 2023-10-21T04:09:14.673+05:30 [Info] Planner::processDeleteToken: There is no delete token to process for node: 127.0.0.1:9002 2023-10-21T04:09:14.673+05:30 [Info] Planner::processDeleteToken: There is no delete token to process for node: 127.0.0.1:9004 2023-10-21T04:09:14.673+05:30 [Info] Planner::processDeleteToken: There is no delete token to process for node: 127.0.0.1:9001 2023-10-21T04:09:14.673+05:30 [Info] Planner::processDeleteToken: There is no delete token to process for node: 127.0.0.1:9003 2023-10-21T04:09:14.674+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41922 2023-10-21T04:09:14.680+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:09:14.680+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:09:14.681+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:09:14.681+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:09:14.844+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:09:14.844+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:09:14.845+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:09:14.845+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:09:14.845+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:09:14.845+05:30 [Info] Moving index 12338036531800130039:3810060733504350542:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 5197198800322775808:14607307599259773249:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 14889401051926000176:18201836893981255880:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 17489537288579938718:3985672602331234807:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 12140458060295645238:4004220452657849790:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15649171661394925140:12043401624902403496:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 8669649359784409226:14910157020177557388:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 12027054384338714804:11400459767051699632:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 16162012140118277065:80911002600697866:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 1844709904168202817:17037906325662037075:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 14292583894719865678:3292093078969249751:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 9619308266130016313:4435591218553988516:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 3674078063353481907:14780363202971367008:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 4860549750398470299:4030261110898422634:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 15968368593314043432:325705934323625324:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 17878371269610391239:15131823346681830059:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 13856704970496601958:8880461853743568138:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 14741949230702299196:2483169159760864439:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.845+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 2872726210871813325:2307463109245789631:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 6985049432225870291:11756349460704871322:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 5103684059183126577:11160011617777336020:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14026476026078401783:2170849387178940761:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 10585832761158485697:7755999278497385093:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 18046371353728341033:8446412942681787061:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 17656144409930763141:13953156762317267443:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 18323460706027642575:2288923528993975684:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 1571318351652255745:17148287981272169168:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 16013578430151540990:16726320877690809560:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.846+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 14340086159258068923:7410154703733445404:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 2374548411389959576:249283441109236971:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 12833670934079048728:13258659765240018889:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 15945117906390072636:11739645294255241349:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.847+05:30 [Info] Moving index 11619165464560280276:7597064108789640455:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 573521267164877540:1532172649864318584:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 8857269605176042330:17729593977321451597:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15431181238515168303:2921784364458449578:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 7874378322029374196:17128008689111072444:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15991242037348127852:9664682240473767939:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.849+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 8669649359784409226:6213797968262183765:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 1571318351652255745:6796752657857722550:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 17489537288579938718:9134754361302452675:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15968368593314043432:17161083103924027797:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 5197198800322775808:13718648355797887482:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 17878371269610391239:7287869093983910291:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 14292583894719865678:4362827867398787577:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 12833670934079048728:4894899676975150934:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 12140458060295645238:3513301448177380059:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 13856704970496601958:15283305021611177781:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 14741949230702299196:8899653570021014071:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 7874378322029374196:11675990026580235111:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.850+05:30 [Info] Moving index 12338036531800130039:4503814398859864266:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 573521267164877540:8205593268191821683:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 18323460706027642575:14991206072146508370:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 18046371353728341033:18184041269291378080:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 4860549750398470299:15710672812788661570:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 3674078063353481907:360999437388707481:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 11619165464560280276:11158580445292211864:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14889401051926000176:3841645022501780362:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 9619308266130016313:11484519016652644658:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 1844709904168202817:1938940178723796232:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 6985049432225870291:4138705102427425280:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 5103684059183126577:16983881020783103544:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.851+05:30 [Info] Moving index 15945117906390072636:10455069697192970620:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 12027054384338714804:9246878583854294580:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 8857269605176042330:16398781184179534607:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.852+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.853+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.853+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.853+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.853+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.853+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:09:14.853+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 16457727 Units 0 2023-10-21T04:09:14.853+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 16454253 Units 0 2023-10-21T04:09:14.853+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 2245049 Units 0 Tenants 0 2023-10-21T04:09:14.853+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 2241277 Units 0 Tenants 0 2023-10-21T04:09:14.853+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:09:14.853+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:09:14.853+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:09:14.853+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:09:14.853+05:30 [Info] ---------------------------------------- 2023-10-21T04:09:14.853+05:30 [Info] 2023-10-21T04:09:14.853+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:85232c9ba3e4451b95e9e6fc8337e027, useLiveData:false 2023-10-21T04:09:14.853+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:122 shardCompatVersion: 1 2023-10-21T04:09:14.853+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:09:14.853+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.853+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.853+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.853+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.853+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.853+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.853+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.853+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.853+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.853+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.853+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.853+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.853+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.853+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.853+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.853+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.853+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.853+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.853+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.853+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:8669649359784409226, instId:6213797968262183765, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:1571318351652255745, instId:6796752657857722550, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:157356627 (150.067M), mem:27000 (26.3672K), overhead:157329627 (150.041M), min:157356627 (150.067M), data:27000 (26.3672K) cpu:0.0674 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:17489537288579938718, instId:9134754361302452675, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.854+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.854+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.854+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.854+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15968368593314043432, instId:17161083103924027797, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:5197198800322775808, instId:13718648355797887482, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157391622 (150.1M), mem:26000 (25.3906K), overhead:157365622 (150.076M), min:157391622 (150.1M), data:26000 (25.3906K) cpu:0.0837 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17878371269610391239, instId:7287869093983910291, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157328811 (150.04M), mem:36720 (35.8594K), overhead:157292091 (150.005M), min:157328811 (150.04M), data:36720 (35.8594K) cpu:0.0088 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157328565 (150.04M), mem:36507 (35.6514K), overhead:157292058 (150.005M), min:157328565 (150.04M), data:36507 (35.6514K) cpu:0.0094 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157328936 (150.041M), mem:36828 (35.9648K), overhead:157292108 (150.005M), min:157328936 (150.041M), data:36828 (35.9648K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157328892 (150.041M), mem:36790 (35.9277K), overhead:157292102 (150.005M), min:157328892 (150.041M), data:36790 (35.9277K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157328534 (150.04M), mem:36480 (35.625K), overhead:157292054 (150.005M), min:157328534 (150.04M), data:36480 (35.625K) cpu:0.0094 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157326801 (150.039M), mem:34980 (34.1602K), overhead:157291821 (150.005M), min:157326801 (150.039M), data:34980 (34.1602K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.855+05:30 [Info] Index total memory:157328631 (150.04M), mem:36564 (35.707K), overhead:157292067 (150.005M), min:157328631 (150.04M), data:36564 (35.707K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.855+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.855+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.855+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157329161 (150.041M), mem:37023 (36.1553K), overhead:157292138 (150.005M), min:157329161 (150.041M), data:37023 (36.1553K) cpu:0.0090 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:14292583894719865678, instId:4362827867398787577, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157505994 (150.209M), mem:110000 (107.422K), overhead:157395994 (150.105M), min:157505994 (150.209M), data:110000 (107.422K) cpu:0.0782 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:12833670934079048728, instId:4894899676975150934, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157297751 (150.011M), mem:9828 (9.59766K), overhead:157287923 (150.001M), min:157297751 (150.011M), data:9828 (9.59766K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:12140458060295645238, instId:3513301448177380059, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157508595 (150.212M), mem:111000 (108.398K), overhead:157397595 (150.106M), min:157508595 (150.212M), data:111000 (108.398K) cpu:0.0732 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:13856704970496601958, instId:15283305021611177781, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:14741949230702299196, instId:8899653570021014071, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157668210 (150.364M), mem:112000 (109.375K), overhead:157556210 (150.257M), min:157668210 (150.364M), data:112000 (109.375K) cpu:0.0002 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:7874378322029374196, instId:11675990026580235111, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157422480 (150.13M), mem:27000 (26.3672K), overhead:157395480 (150.104M), min:157422480 (150.13M), data:27000 (26.3672K) cpu:0.0781 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_default, collection:_default, defnId:12338036531800130039, instId:4503814398859864266, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0792 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157328033 (150.04M), mem:36046 (35.2012K), overhead:157291987 (150.005M), min:157328033 (150.04M), data:36046 (35.2012K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.856+05:30 [Info] Index total memory:157326990 (150.039M), mem:35143 (34.3193K), overhead:157291847 (150.005M), min:157326990 (150.039M), data:35143 (34.3193K) cpu:0.0070 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.856+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.856+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.856+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157327869 (150.04M), mem:35904 (35.0625K), overhead:157291965 (150.005M), min:157327869 (150.04M), data:35904 (35.0625K) cpu:0.0082 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157328793 (150.04M), mem:36704 (35.8438K), overhead:157292089 (150.005M), min:157328793 (150.04M), data:36704 (35.8438K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157326784 (150.039M), mem:34965 (34.1455K), overhead:157291819 (150.005M), min:157326784 (150.039M), data:34965 (34.1455K) cpu:0.0070 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157328603 (150.04M), mem:36540 (35.6836K), overhead:157292063 (150.005M), min:157328603 (150.04M), data:36540 (35.6836K) cpu:0.0088 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157327691 (150.039M), mem:35750 (34.9121K), overhead:157291941 (150.005M), min:157327691 (150.039M), data:35750 (34.9121K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157328326 (150.04M), mem:36300 (35.4492K), overhead:157292026 (150.005M), min:157328326 (150.04M), data:36300 (35.4492K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:18323460706027642575, instId:14991206072146508370, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:18046371353728341033, instId:18184041269291378080, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4860549750398470299, instId:15710672812788661570, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157656330 (150.353M), mem:112000 (109.375K), overhead:157544330 (150.246M), min:157656330 (150.353M), data:112000 (109.375K) cpu:0.0761 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3674078063353481907, instId:360999437388707481, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:157297533 (150.011M), mem:9639 (9.41309K), overhead:157287894 (150.001M), min:157297533 (150.011M), data:9639 (9.41309K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.857+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:11619165464560280276, instId:11158580445292211864, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:09:14.857+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.857+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.857+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:14889401051926000176, instId:3841645022501780362, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0858 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:9619308266130016313, instId:11484519016652644658, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157329747 (150.041M), mem:37530 (36.6504K), overhead:157292217 (150.006M), min:157329747 (150.041M), data:37530 (36.6504K) cpu:0.0094 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157327677 (150.039M), mem:35738 (34.9004K), overhead:157291939 (150.005M), min:157327677 (150.039M), data:35738 (34.9004K) cpu:0.0074 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157329754 (150.041M), mem:37536 (36.6562K), overhead:157292218 (150.006M), min:157329754 (150.041M), data:37536 (36.6562K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157329756 (150.041M), mem:37538 (36.6582K), overhead:157292218 (150.006M), min:157329756 (150.041M), data:37538 (36.6582K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157329370 (150.041M), mem:37204 (36.332K), overhead:157292166 (150.005M), min:157329370 (150.041M), data:37204 (36.332K) cpu:0.0092 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157328904 (150.041M), mem:36800 (35.9375K), overhead:157292104 (150.005M), min:157328904 (150.041M), data:36800 (35.9375K) cpu:0.0080 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157328323 (150.04M), mem:36297 (35.4463K), overhead:157292026 (150.005M), min:157328323 (150.04M), data:36297 (35.4463K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157329125 (150.041M), mem:36992 (36.125K), overhead:157292133 (150.005M), min:157329125 (150.041M), data:36992 (36.125K) cpu:0.0090 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157327522 (150.039M), mem:35604 (34.7695K), overhead:157291918 (150.005M), min:157327522 (150.039M), data:35604 (34.7695K) cpu:0.0100 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157326547 (150.038M), mem:34760 (33.9453K), overhead:157291787 (150.005M), min:157326547 (150.038M), data:34760 (33.9453K) cpu:0.0086 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.858+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.858+05:30 [Info] Index total memory:157327373 (150.039M), mem:35475 (34.6436K), overhead:157291898 (150.005M), min:157327373 (150.039M), data:35475 (34.6436K) cpu:0.0100 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.858+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.858+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:157328603 (150.04M), mem:36540 (35.6836K), overhead:157292063 (150.005M), min:157328603 (150.04M), data:36540 (35.6836K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:157327129 (150.039M), mem:35264 (34.4375K), overhead:157291865 (150.005M), min:157327129 (150.039M), data:35264 (34.4375K) cpu:0.0090 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:157327178 (150.039M), mem:35306 (34.4785K), overhead:157291872 (150.005M), min:157327178 (150.039M), data:35306 (34.4785K) cpu:0.0098 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:157327258 (150.039M), mem:35375 (34.5459K), overhead:157291883 (150.005M), min:157327258 (150.039M), data:35375 (34.5459K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:157327933 (150.04M), mem:35960 (35.1172K), overhead:157291973 (150.005M), min:157327933 (150.04M), data:35960 (35.1172K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:1844709904168202817, instId:1938940178723796232, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.859+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.859+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.859+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.859+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:6985049432225870291, instId:4138705102427425280, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:5103684059183126577, instId:16983881020783103544, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:157354026 (150.064M), mem:26000 (25.3906K), overhead:157328026 (150.04M), min:157354026 (150.064M), data:26000 (25.3906K) cpu:0.0858 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:157328053 (150.04M), mem:36064 (35.2188K), overhead:157291989 (150.005M), min:157328053 (150.04M), data:36064 (35.2188K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:157456282 (150.162M), mem:37638 (36.7559K), overhead:157418644 (150.126M), min:157456282 (150.162M), data:37638 (36.7559K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:157454397 (150.16M), mem:37269 (36.3955K), overhead:157417128 (150.125M), min:157454397 (150.16M), data:37269 (36.3955K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:157452078 (150.158M), mem:37674 (36.791K), overhead:157414404 (150.122M), min:157452078 (150.158M), data:37674 (36.791K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:157328832 (150.04M), mem:36738 (35.877K), overhead:157292094 (150.005M), min:157328832 (150.04M), data:36738 (35.877K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.860+05:30 [Info] Index total memory:157452362 (150.158M), mem:37675 (36.792K), overhead:157414687 (150.122M), min:157452362 (150.158M), data:37675 (36.792K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.860+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.860+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.860+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:157327280 (150.039M), mem:35394 (34.5645K), overhead:157291886 (150.005M), min:157327280 (150.039M), data:35394 (34.5645K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:157454171 (150.16M), mem:38628 (37.7227K), overhead:157415543 (150.123M), min:157454171 (150.16M), data:38628 (37.7227K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:15945117906390072636, instId:10455069697192970620, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:12027054384338714804, instId:9246878583854294580, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:157650570 (150.347M), mem:110000 (107.422K), overhead:157540570 (150.242M), min:157650570 (150.347M), data:110000 (107.422K) cpu:0.0002 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:157328961 (150.041M), mem:36850 (35.9863K), overhead:157292111 (150.005M), min:157328961 (150.041M), data:36850 (35.9863K) cpu:0.0100 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:157328811 (150.04M), mem:36720 (35.8594K), overhead:157292091 (150.005M), min:157328811 (150.04M), data:36720 (35.8594K) cpu:0.0088 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:157329586 (150.041M), mem:37391 (36.5146K), overhead:157292195 (150.006M), min:157329586 (150.041M), data:37391 (36.5146K) cpu:0.0104 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.861+05:30 [Info] Index total memory:157327924 (150.04M), mem:35952 (35.1094K), overhead:157291972 (150.005M), min:157327924 (150.04M), data:35952 (35.1094K) cpu:0.0082 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.861+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.861+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.861+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:157329104 (150.041M), mem:36974 (36.1074K), overhead:157292130 (150.005M), min:157329104 (150.041M), data:36974 (36.1074K) cpu:0.0098 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:157329025 (150.041M), mem:36905 (36.04K), overhead:157292120 (150.005M), min:157329025 (150.041M), data:36905 (36.04K) cpu:0.0090 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:157328733 (150.04M), mem:36652 (35.793K), overhead:157292081 (150.005M), min:157328733 (150.04M), data:36652 (35.793K) cpu:0.0088 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:157329095 (150.041M), mem:36966 (36.0996K), overhead:157292129 (150.005M), min:157329095 (150.041M), data:36966 (36.0996K) cpu:0.0090 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.862+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.862+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.862+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.862+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:8857269605176042330, instId:16398781184179534607, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:157387575 (150.096M), mem:25000 (24.4141K), overhead:157362575 (150.073M), min:157387575 (150.096M), data:25000 (24.4141K) cpu:0.0001 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.863+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.863+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.863+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.863+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.864+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.864+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.864+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.864+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.864+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.864+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.864+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.864+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.864+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.864+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.864+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.864+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.864+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.864+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.864+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.864+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.864+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.864+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.864+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.864+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.864+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.864+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.864+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.864+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.864+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:09:14.864+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.864+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.864+05:30 [Info] 2023-10-21T04:09:14.864+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:31fd475f22d8ee0da906c31968c91305, useLiveData:false 2023-10-21T04:09:14.864+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:122 shardCompatVersion: 1 2023-10-21T04:09:14.864+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:09:14.864+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:12338036531800130039, instId:3810060733504350542, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0858 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:5197198800322775808, instId:14607307599259773249, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:157391622 (150.1M), mem:26000 (25.3906K), overhead:157365622 (150.076M), min:157391622 (150.1M), data:26000 (25.3906K) cpu:0.0805 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:14889401051926000176, instId:18201836893981255880, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:157415760 (150.123M), mem:112000 (109.375K), overhead:157303760 (150.017M), min:157415760 (150.123M), data:112000 (109.375K) cpu:0.0788 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_1, scope:_default, collection:c1, defnId:17489537288579938718, instId:3985672602331234807, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:12140458060295645238, instId:4004220452657849790, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:157508595 (150.212M), mem:111000 (108.398K), overhead:157397595 (150.106M), min:157508595 (150.212M), data:111000 (108.398K) cpu:0.0932 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.865+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.865+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.865+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.865+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:8669649359784409226, instId:14910157020177557388, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:12027054384338714804, instId:11400459767051699632, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:157650570 (150.347M), mem:110000 (107.422K), overhead:157540570 (150.242M), min:157650570 (150.347M), data:110000 (107.422K) cpu:0.0002 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:157327820 (150.04M), mem:35862 (35.0215K), overhead:157291958 (150.005M), min:157327820 (150.04M), data:35862 (35.0215K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:157326801 (150.039M), mem:34980 (34.1602K), overhead:157291821 (150.005M), min:157326801 (150.039M), data:34980 (34.1602K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.866+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.866+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.866+05:30 [Info] Index total memory:157327671 (150.039M), mem:35733 (34.8955K), overhead:157291938 (150.005M), min:157327671 (150.039M), data:35733 (34.8955K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.866+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157328765 (150.04M), mem:36680 (35.8203K), overhead:157292085 (150.005M), min:157328765 (150.04M), data:36680 (35.8203K) cpu:0.0122 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157327263 (150.039M), mem:35380 (34.5508K), overhead:157291883 (150.005M), min:157327263 (150.039M), data:35380 (34.5508K) cpu:0.0102 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157327618 (150.039M), mem:35687 (34.8506K), overhead:157291931 (150.005M), min:157327618 (150.039M), data:35687 (34.8506K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157327402 (150.039M), mem:35500 (34.668K), overhead:157291902 (150.005M), min:157327402 (150.039M), data:35500 (34.668K) cpu:0.0110 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157327933 (150.04M), mem:35960 (35.1172K), overhead:157291973 (150.005M), min:157327933 (150.04M), data:35960 (35.1172K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:1844709904168202817, instId:17037906325662037075, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:_default, defnId:14292583894719865678, instId:3292093078969249751, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157505994 (150.209M), mem:110000 (107.422K), overhead:157395994 (150.105M), min:157505994 (150.209M), data:110000 (107.422K) cpu:0.0880 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:9619308266130016313, instId:4435591218553988516, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3674078063353481907, instId:14780363202971367008, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157297533 (150.011M), mem:9639 (9.41309K), overhead:157287894 (150.001M), min:157297533 (150.011M), data:9639 (9.41309K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:4860549750398470299, instId:4030261110898422634, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.867+05:30 [Info] Index total memory:157602585 (150.302M), mem:111000 (108.398K), overhead:157491585 (150.196M), min:157602585 (150.302M), data:111000 (108.398K) cpu:0.0893 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.867+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.867+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.867+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:15968368593314043432, instId:325705934323625324, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.868+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.868+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:17878371269610391239, instId:15131823346681830059, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.868+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.868+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:13856704970496601958, instId:8880461853743568138, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.868+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.868+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:14741949230702299196, instId:2483169159760864439, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:157668210 (150.364M), mem:112000 (109.375K), overhead:157556210 (150.257M), min:157668210 (150.364M), data:112000 (109.375K) cpu:0.0002 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.868+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.868+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.868+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.868+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.868+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.868+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.868+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.868+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.868+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.868+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:6985049432225870291, instId:11756349460704871322, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:5103684059183126577, instId:11160011617777336020, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:157356627 (150.067M), mem:27000 (26.3672K), overhead:157329627 (150.041M), min:157356627 (150.067M), data:27000 (26.3672K) cpu:0.0792 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:157329591 (150.041M), mem:37395 (36.5186K), overhead:157292196 (150.006M), min:157329591 (150.041M), data:37395 (36.5186K) cpu:0.0126 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:157327677 (150.039M), mem:35738 (34.9004K), overhead:157291939 (150.005M), min:157327677 (150.039M), data:35738 (34.9004K) cpu:0.0100 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.869+05:30 [Info] Index total memory:157330072 (150.042M), mem:37812 (36.9258K), overhead:157292260 (150.006M), min:157330072 (150.042M), data:37812 (36.9258K) cpu:0.0128 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.869+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.869+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.869+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:157329756 (150.041M), mem:37538 (36.6582K), overhead:157292218 (150.006M), min:157329756 (150.041M), data:37538 (36.6582K) cpu:0.0128 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:157329219 (150.041M), mem:37073 (36.2041K), overhead:157292146 (150.005M), min:157329219 (150.041M), data:37073 (36.2041K) cpu:0.0122 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:157328904 (150.041M), mem:36800 (35.9375K), overhead:157292104 (150.005M), min:157328904 (150.041M), data:36800 (35.9375K) cpu:0.0106 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:157328323 (150.04M), mem:36297 (35.4463K), overhead:157292026 (150.005M), min:157328323 (150.04M), data:36297 (35.4463K) cpu:0.0102 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:157328682 (150.04M), mem:36608 (35.75K), overhead:157292074 (150.005M), min:157328682 (150.04M), data:36608 (35.75K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.870+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.870+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.870+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.870+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:18046371353728341033, instId:8446412942681787061, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.871+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.871+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.871+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.871+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_1, scope:_default, collection:c1, defnId:18323460706027642575, instId:2288923528993975684, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:1571318351652255745, instId:17148287981272169168, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0972 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157328961 (150.041M), mem:36850 (35.9863K), overhead:157292111 (150.005M), min:157328961 (150.041M), data:36850 (35.9863K) cpu:0.0122 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157329227 (150.041M), mem:37080 (36.2109K), overhead:157292147 (150.005M), min:157329227 (150.041M), data:37080 (36.2109K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157329586 (150.041M), mem:37391 (36.5146K), overhead:157292195 (150.006M), min:157329586 (150.041M), data:37391 (36.5146K) cpu:0.0126 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157327924 (150.04M), mem:35952 (35.1094K), overhead:157291972 (150.005M), min:157327924 (150.04M), data:35952 (35.1094K) cpu:0.0102 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157329104 (150.041M), mem:36974 (36.1074K), overhead:157292130 (150.005M), min:157329104 (150.041M), data:36974 (36.1074K) cpu:0.0120 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157328885 (150.041M), mem:36784 (35.9219K), overhead:157292101 (150.005M), min:157328885 (150.041M), data:36784 (35.9219K) cpu:0.0110 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157328595 (150.04M), mem:36533 (35.6768K), overhead:157292062 (150.005M), min:157328595 (150.04M), data:36533 (35.6768K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.872+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.872+05:30 [Info] Index total memory:157328954 (150.041M), mem:36844 (35.9805K), overhead:157292110 (150.005M), min:157328954 (150.041M), data:36844 (35.9805K) cpu:0.0110 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.872+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.872+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157328053 (150.04M), mem:36064 (35.2188K), overhead:157291989 (150.005M), min:157328053 (150.04M), data:36064 (35.2188K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.873+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157455654 (150.161M), mem:37515 (36.6357K), overhead:157418139 (150.126M), min:157455654 (150.161M), data:37515 (36.6357K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.873+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157453141 (150.159M), mem:37023 (36.1553K), overhead:157416118 (150.124M), min:157453141 (150.159M), data:37023 (36.1553K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.873+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157451380 (150.157M), mem:37536 (36.6562K), overhead:157413844 (150.122M), min:157451380 (150.157M), data:37536 (36.6562K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.873+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157328832 (150.04M), mem:36738 (35.877K), overhead:157292094 (150.005M), min:157328832 (150.04M), data:36738 (35.877K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.873+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157450976 (150.157M), mem:37401 (36.5244K), overhead:157413575 (150.121M), min:157450976 (150.157M), data:37401 (36.5244K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.873+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157327162 (150.039M), mem:35292 (34.4648K), overhead:157291870 (150.005M), min:157327162 (150.039M), data:35292 (34.4648K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.873+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.873+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.873+05:30 [Info] Index total memory:157454171 (150.16M), mem:38628 (37.7227K), overhead:157415543 (150.123M), min:157454171 (150.16M), data:38628 (37.7227K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.873+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.874+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.874+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.874+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.874+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:12833670934079048728, instId:13258659765240018889, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:157297848 (150.011M), mem:9912 (9.67969K), overhead:157287936 (150.001M), min:157297848 (150.011M), data:9912 (9.67969K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_1, scope:_default, collection:_default, defnId:15945117906390072636, instId:11739645294255241349, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_1, scope:_default, collection:_default, defnId:11619165464560280276, instId:7597064108789640455, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:157328342 (150.04M), mem:36314 (35.4629K), overhead:157292028 (150.005M), min:157328342 (150.04M), data:36314 (35.4629K) cpu:0.0138 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:157326859 (150.039M), mem:35030 (34.209K), overhead:157291829 (150.005M), min:157326859 (150.039M), data:35030 (34.209K) cpu:0.0116 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:157328021 (150.04M), mem:36036 (35.1914K), overhead:157291985 (150.005M), min:157328021 (150.04M), data:36036 (35.1914K) cpu:0.0136 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:157328649 (150.04M), mem:36580 (35.7227K), overhead:157292069 (150.005M), min:157328649 (150.04M), data:36580 (35.7227K) cpu:0.0128 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.875+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.875+05:30 [Info] Index total memory:157326784 (150.039M), mem:34965 (34.1455K), overhead:157291819 (150.005M), min:157326784 (150.039M), data:34965 (34.1455K) cpu:0.0114 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.875+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.875+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.876+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.876+05:30 [Info] Index total memory:157328603 (150.04M), mem:36540 (35.6836K), overhead:157292063 (150.005M), min:157328603 (150.04M), data:36540 (35.6836K) cpu:0.0144 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.876+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.876+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.876+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.876+05:30 [Info] Index total memory:157327980 (150.04M), mem:36000 (35.1562K), overhead:157291980 (150.005M), min:157327980 (150.04M), data:36000 (35.1562K) cpu:0.0128 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.876+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.876+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.876+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:09:14.876+05:30 [Info] Index total memory:157328606 (150.04M), mem:36542 (35.6855K), overhead:157292064 (150.005M), min:157328606 (150.04M), data:36542 (35.6855K) cpu:0.0124 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.876+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.876+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.876+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:8857269605176042330, instId:17729593977321451597, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:09:14.876+05:30 [Info] Index total memory:157391622 (150.1M), mem:26000 (25.3906K), overhead:157365622 (150.076M), min:157391622 (150.1M), data:26000 (25.3906K) cpu:0.0001 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.876+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.876+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.876+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.876+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.876+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.876+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.876+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.876+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.876+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.876+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.876+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.876+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.876+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:7874378322029374196, instId:17128008689111072444, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:157419000 (150.126M), mem:26000 (25.3906K), overhead:157393000 (150.102M), min:157419000 (150.126M), data:26000 (25.3906K) cpu:0.0881 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:157328673 (150.04M), mem:36600 (35.7422K), overhead:157292073 (150.005M), min:157328673 (150.04M), data:36600 (35.7422K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:157328416 (150.04M), mem:36378 (35.5254K), overhead:157292038 (150.005M), min:157328416 (150.04M), data:36378 (35.5254K) cpu:0.0116 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:157328936 (150.041M), mem:36828 (35.9648K), overhead:157292108 (150.005M), min:157328936 (150.041M), data:36828 (35.9648K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.877+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.877+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.877+05:30 [Info] Index total memory:157328892 (150.041M), mem:36790 (35.9277K), overhead:157292102 (150.005M), min:157328892 (150.041M), data:36790 (35.9277K) cpu:0.0116 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.877+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.878+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.878+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.878+05:30 [Info] Index total memory:157328534 (150.04M), mem:36480 (35.625K), overhead:157292054 (150.005M), min:157328534 (150.04M), data:36480 (35.625K) cpu:0.0114 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.878+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.878+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.878+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.878+05:30 [Info] Index total memory:157326801 (150.039M), mem:34980 (34.1602K), overhead:157291821 (150.005M), min:157326801 (150.039M), data:34980 (34.1602K) cpu:0.0094 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.878+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.878+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.878+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.878+05:30 [Info] Index total memory:157328936 (150.041M), mem:36828 (35.9648K), overhead:157292108 (150.005M), min:157328936 (150.041M), data:36828 (35.9648K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.878+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.878+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:09:14.878+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:09:14.878+05:30 [Info] Index total memory:157328877 (150.041M), mem:36777 (35.915K), overhead:157292100 (150.005M), min:157328877 (150.041M), data:36777 (35.915K) cpu:0.0110 io:0 (0) scan:0 drain:0 2023-10-21T04:09:14.878+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:09:14.878+05:30 [Info] **************************************** 2023-10-21T04:09:14.880+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 2245049 Units 0 Tenants 0 2023-10-21T04:09:14.880+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 2241277 Units 0 Tenants 0 2023-10-21T04:09:14.880+05:30 [Info] Generating Shard Transfer Token (ShardTokenc8:87:93:a:4c:bb:58:5c) for rebalance ( MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: Region: InstId: 18226445516969809850 RealInstId: 1955535609704140162 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 1955535609704140162 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 6213797968262183765 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 6213797968262183765 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17161083103924027797 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17161083103924027797 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3231309696512952224 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 9010726441942672253 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 8899653570021014071 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 18184041269291378080 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 18184041269291378080 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 15710672812788661570 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 15710672812788661570 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3140239290166316345 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 18051244886420988269 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3878185423818313063 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3052172645459386575 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 9246878583854294580 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17238494267014926113 RealInstId: 10574086758347311295 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 10574086758347311295 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 1422056348477710921 RealInstId: 16044079815412989935 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 16044079815412989935 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 13718648355797887482 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 7287869093983910291 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 7287869093983910291 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 15283305021611177781 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 15283305021611177781 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11675990026580235111 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 360999437388707481 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 360999437388707481 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 1938940178723796232 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 1938940178723796232 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 16398781184179534607 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 16398781184179534607 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:09:14.883+05:30 [Info] Generating Shard Transfer Token (ShardToken53:c:96:ac:59:a4:5d:7b) for rebalance ( MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: Region: InstId: 757549517258233573 RealInstId: 9358717944830195392 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 9358717944830195392 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4362827867398787577 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3513301448177380059 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 2866240913596939981 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8205593268191821683 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 14991206072146508370 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14991206072146508370 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3841645022501780362 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3841645022501780362 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10040529926077950124 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6393061446487756912 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12427853620151898915 RealInstId: 7291286939424384589 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 7291286939424384589 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4138705102427425280 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4138705102427425280 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10455069697192970620 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10455069697192970620 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 2330053926472458685 RealInstId: 5299824419646640495 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 5299824419646640495 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10655181962244539875 RealInstId: 18089525033132576654 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 18089525033132576654 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 6796752657857722550 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 6796752657857722550 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 9134754361302452675 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 9134754361302452675 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4894899676975150934 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4894899676975150934 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4503814398859864266 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 11158580445292211864 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11158580445292211864 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11484519016652644658 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11484519016652644658 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 16983881020783103544 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:09:14.887+05:30 [Info] Generating Shard Transfer Token (ShardToken26:d5:d0:e3:1f:91:4a:e4) for rebalance ( MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: Region: InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4004220452657849790 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3292093078969249751 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12845014165282783025 RealInstId: 2307463109245789631 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 2307463109245789631 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11756349460704871322 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11756349460704871322 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 579097042483677314 RealInstId: 2170849387178940761 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4248339762727765604 RealInstId: 7755999278497385093 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 7755999278497385093 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 8934648905917366073 RealInstId: 13953156762317267443 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 13953156762317267443 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2288923528993975684 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2288923528993975684 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10225476268021138633 RealInstId: 16726320877690809560 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11739645294255241349 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11739645294255241349 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 4164706166433630808 RealInstId: 1532172649864318584 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3810060733504350542 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3985672602331234807 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3985672602331234807 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 4435591218553988516 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4435591218553988516 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11160011617777336020 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17148287981272169168 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 13258659765240018889 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 7597064108789640455 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 7597064108789640455 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:09:14.890+05:30 [Info] Generating Shard Transfer Token (ShardToken80:53:e0:a8:fa:c4:c4:85) for rebalance ( MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: Region: InstId: 7543997995673100641 RealInstId: 12043401624902403496 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12043401624902403496 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 14910157020177557388 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14910157020177557388 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11400459767051699632 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 6377858590284099462 RealInstId: 80911002600697866 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4030261110898422634 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 325705934323625324 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 325705934323625324 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 8446412942681787061 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 8446412942681787061 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10774491569505258841 RealInstId: 7410154703733445404 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 1981123602637975601 RealInstId: 249283441109236971 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 249283441109236971 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4611974930531028413 RealInstId: 2921784364458449578 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 2921784364458449578 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10428614901675085295 RealInstId: 9664682240473767939 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 14607307599259773249 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17037906325662037075 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17037906325662037075 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 14780363202971367008 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 15131823346681830059 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 15131823346681830059 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 8880461853743568138 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 8880461853743568138 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17729593977321451597 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17128008689111072444 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:09:14.894+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:09:14.894+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:09:14.901+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenc8:87:93:a:4c:bb:58:5c 2023-10-21T04:09:14.955+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken53:c:96:ac:59:a4:5d:7b 2023-10-21T04:09:14.997+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken26:d5:d0:e3:1f:91:4a:e4 2023-10-21T04:09:15.050+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken80:53:e0:a8:fa:c4:c4:85 2023-10-21T04:09:15.050+05:30 [Info] ShardRebalancer::observeRebalance &{31fd475f22d8ee0da906c31968c91305 15e5cffff68b4b70e641b53958f9ea44 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:09:15.074+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.080+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"31fd475f22d8ee0da906c31968c91305","RebalId":"15e5cffff68b4b70e641b53958f9ea44","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:09:15.080+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:09:15.083+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.090+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.093+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.107+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.108+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:09:15.109+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:09:15.109+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:09:15.109+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 54.891µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:15.129+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.187+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.230+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.231+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:09:15.231+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:09:15.231+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:09:15.231+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 42.444µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:15.305+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.312+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.417+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.417+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:09:15.426+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{31fd475f22d8ee0da906c31968c91305 15e5cffff68b4b70e641b53958f9ea44 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:09:15.427+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 2 2023-10-21T04:09:15.764+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[281b41507b1f937a1b6c37bc4027eef6:ShardToken53:c:96:ac:59:a4:5d:7b 5020693e9ae5c52168d414b02f922ad8:ShardToken26:d5:d0:e3:1f:91:4a:e4], currActiveTransfers: map[281b41507b1f937a1b6c37bc4027eef6:2 5020693e9ae5c52168d414b02f922ad8:2] 2023-10-21T04:09:15.765+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"31fd475f22d8ee0da906c31968c91305","RebalId":"15e5cffff68b4b70e641b53958f9ea44","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:09:15.765+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:09:15.765+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:09:15.765+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:09:15.765+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:09:15.765+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:09:15.767+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.770+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 20.937µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:15.772+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.774+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.776+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.776+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:1 5020693e9ae5c52168d414b02f922ad8:2 85232c9ba3e4451b95e9e6fc8337e027:1], batchSize: 2 2023-10-21T04:09:15.818+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.977+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:15.977+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:1 31fd475f22d8ee0da906c31968c91305:1 5020693e9ae5c52168d414b02f922ad8:1 85232c9ba3e4451b95e9e6fc8337e027:1], batchSize: 2 2023-10-21T04:09:15.977+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:09:15.977+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:09:15.977+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:09:15.977+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.14µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:15.977+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[6611724767191156837:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/6611724767191156837/shards/shard6611724767191156837 15964300471522801617:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/15964300471522801617/shards/shard15964300471522801617] 15e5cffff68b4b70e641b53958f9ea44 ShardToken80:53:e0:a8:fa:c4:c4:85 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0068f4780 0xc0068f47e0 map[] 0xc00754aae0 0xc00755d200 false} 2023-10-21T04:09:15.978+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:09:15.978+05:30 [Info] Directory /opt/build/ns_server/data/n_3/data/@2i/shards does not exist. Skip shard recovery. 2023-10-21T04:09:15.978+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/15964300471522801617/shards/shard15964300471522801617 2023-10-21T04:09:15.980+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/6611724767191156837/shards/shard6611724767191156837 2023-10-21T04:09:15.983+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/15964300471522801617/shards/shard15964300471522801617/data: super block snapshot head offset :0 tail offset:929792 sbgen :198 2023-10-21T04:09:15.983+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/15964300471522801617/shards/shard15964300471522801617/data/recovery: super block snapshot head offset :0 tail offset:819200 sbgen :198 2023-10-21T04:09:16.004+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/15964300471522801617/shards/shard15964300471522801617: metadata saved successfully 2023-10-21T04:09:16.006+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 (15964300471522801617) added 2023-10-21T04:09:16.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 from metadata completed, tenant :bucket_%2_7007f0ab442ac00e52d057a30cd22644 2023-10-21T04:09:16.006+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 63334964 totalBytes: 1773379 numShards: 2 duration: 28.484221ms 2023-10-21T04:09:16.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/15964300471522801617/shards/shard15964300471522801617, newshardId 15964300471522801617, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617, shardsize 1773379, duration 28.194259ms 2023-10-21T04:09:16.007+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 15964300471522801617, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617, err: , elapsed(sec): 0.029246736 2023-10-21T04:09:16.007+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 15964300471522801617 bytesWritten: 1773379, totalBytes: 1773379, transferRate: 6.56807037037037e+07 2023-10-21T04:09:16.017+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/6611724767191156837/shards/shard6611724767191156837/data: super block snapshot head offset :0 tail offset:1200128 sbgen :227 2023-10-21T04:09:16.018+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/6611724767191156837/shards/shard6611724767191156837/data/recovery: super block snapshot head offset :0 tail offset:1089536 sbgen :227 2023-10-21T04:09:16.043+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/6611724767191156837/shards/shard6611724767191156837: metadata saved successfully 2023-10-21T04:09:16.055+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 (6611724767191156837) added 2023-10-21T04:09:16.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 from metadata completed, tenant :bucket_%2_7007f0ab442ac00e52d057a30cd22644 2023-10-21T04:09:16.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken80_53_e0_a8_fa_c4_c4_85/6611724767191156837/shards/shard6611724767191156837, newshardId 6611724767191156837, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837, shardsize 2315088, duration 74.908829ms 2023-10-21T04:09:16.055+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 6611724767191156837 bytesWritten: 2315088, totalBytes: 2315088, transferRate: 3.1284972972972974e+07 2023-10-21T04:09:16.073+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 6611724767191156837, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837, err: , elapsed(sec): 0.095683371 2023-10-21T04:09:16.073+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 15e5cffff68b4b70e641b53958f9ea44, transferId: ShardToken80:53:e0:a8:fa:c4:c4:85 2023-10-21T04:09:16.074+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 15e5cffff68b4b70e641b53958f9ea44, transferId: ShardToken80:53:e0:a8:fa:c4:c4:85, elapsed(sec): 0.000255093 2023-10-21T04:09:16.074+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[6611724767191156837: 15964300471522801617:], shardPaths: map[6611724767191156837:/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 15964300471522801617:/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.096167728 2023-10-21T04:09:16.074+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [6611724767191156837 15964300471522801617], ttid: ShardToken80:53:e0:a8:fa:c4:c4:85, elapsed(sec): 0.096688927 2023-10-21T04:09:16.090+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:16.338+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:1 31fd475f22d8ee0da906c31968c91305:1 5020693e9ae5c52168d414b02f922ad8:1 85232c9ba3e4451b95e9e6fc8337e027:0], batchSize: 2 2023-10-21T04:09:16.339+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:16.339+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:1 5020693e9ae5c52168d414b02f922ad8:1 85232c9ba3e4451b95e9e6fc8337e027:1], batchSize: 2 2023-10-21T04:09:16.348+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:16.348+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:0 5020693e9ae5c52168d414b02f922ad8:1 85232c9ba3e4451b95e9e6fc8337e027:1], batchSize: 2 2023-10-21T04:09:16.348+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken80:53:e0:a8:fa:c4:c4:85 2023-10-21T04:09:16.348+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[6611724767191156837:true 15964300471522801617:true] 2023-10-21T04:09:16.348+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.648µs 2023-10-21T04:09:16.349+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [6611724767191156837 15964300471522801617], isLockedForRecovery: true 2023-10-21T04:09:16.349+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [6611724767191156837 15964300471522801617], errMap: map[6611724767191156837: 15964300471522801617:], elapsed: 20.637µs 2023-10-21T04:09:16.414+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 8669649359784409226, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:16.414+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 8669649359784409226, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:16.435+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:16.435+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:16.518+05:30 [Info] switched currmeta from 58 -> 60 force true 2023-10-21T04:09:16.614+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:16.615+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:1 5020693e9ae5c52168d414b02f922ad8:0 85232c9ba3e4451b95e9e6fc8337e027:1], batchSize: 2 2023-10-21T04:09:16.615+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:09:16.615+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:09:16.615+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:09:16.615+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[3268862397300335529:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/3268862397300335529/shards/shard3268862397300335529 10563635432419104314:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/10563635432419104314/shards/shard10563635432419104314] 15e5cffff68b4b70e641b53958f9ea44 ShardToken26:d5:d0:e3:1f:91:4a:e4 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0068f4780 0xc0068f47e0 map[] 0xc00754b5c0 0xc0083725a0 false} 2023-10-21T04:09:16.615+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/3268862397300335529/shards/shard3268862397300335529 2023-10-21T04:09:16.616+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/10563635432419104314/shards/shard10563635432419104314 2023-10-21T04:09:16.622+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/10563635432419104314/shards/shard10563635432419104314/data: super block snapshot head offset :0 tail offset:1204224 sbgen :227 2023-10-21T04:09:16.623+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/10563635432419104314/shards/shard10563635432419104314/data/recovery: super block snapshot head offset :0 tail offset:1105920 sbgen :227 2023-10-21T04:09:16.625+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/3268862397300335529/shards/shard3268862397300335529/data: super block snapshot head offset :0 tail offset:946176 sbgen :198 2023-10-21T04:09:16.625+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/3268862397300335529/shards/shard3268862397300335529/data/recovery: super block snapshot head offset :0 tail offset:835584 sbgen :198 2023-10-21T04:09:16.629+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/10563635432419104314/shards/shard10563635432419104314: metadata saved successfully 2023-10-21T04:09:16.630+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 (10563635432419104314) added 2023-10-21T04:09:16.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 from metadata completed, tenant :bucket_1_db19c9050b4ab1232e6bc6654011618f 2023-10-21T04:09:16.630+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10563635432419104314 bytesWritten: 2335508, totalBytes: 2335508, transferRate: 1.66822e+08 2023-10-21T04:09:16.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/10563635432419104314/shards/shard10563635432419104314, newshardId 10563635432419104314, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314, shardsize 2335508, duration 14.756673ms 2023-10-21T04:09:16.630+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10563635432419104314, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314, err: , elapsed(sec): 0.015428614 2023-10-21T04:09:16.631+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/3268862397300335529/shards/shard3268862397300335529: metadata saved successfully 2023-10-21T04:09:16.636+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 (3268862397300335529) added 2023-10-21T04:09:16.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 from metadata completed, tenant :bucket_1_db19c9050b4ab1232e6bc6654011618f 2023-10-21T04:09:16.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/15e5cffff68b4b70e641b53958f9ea44_ShardToken26_d5_d0_e3_1f_91_4a_e4/3268862397300335529/shards/shard3268862397300335529, newshardId 3268862397300335529, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529, shardsize 1806105, duration 21.083799ms 2023-10-21T04:09:16.637+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 3268862397300335529, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529, err: , elapsed(sec): 0.021479128 2023-10-21T04:09:16.637+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 3268862397300335529 bytesWritten: 1806105, totalBytes: 1806105, transferRate: 8.6005e+07 2023-10-21T04:09:16.642+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 15e5cffff68b4b70e641b53958f9ea44, transferId: ShardToken26:d5:d0:e3:1f:91:4a:e4 2023-10-21T04:09:16.643+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 15e5cffff68b4b70e641b53958f9ea44, transferId: ShardToken26:d5:d0:e3:1f:91:4a:e4, elapsed(sec): 0.004175144 2023-10-21T04:09:16.646+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[3268862397300335529: 10563635432419104314:], shardPaths: map[3268862397300335529:/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 10563635432419104314:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.031027128 2023-10-21T04:09:16.648+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [10563635432419104314 3268862397300335529], ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4, elapsed(sec): 0.033252322 2023-10-21T04:09:16.672+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:16.683+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14910157020177557388,realInstId: 0, indexDefn DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:16.684+05:30 [Info] Indexer::handleRecoverIndex InstId: 14910157020177557388 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:16.684+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 534.195µs 2023-10-21T04:09:16.684+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 581.1µs 2023-10-21T04:09:16.684+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14910157020177557388 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:16.684+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14910157020177557388, indexDefn DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:16.685+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 270.627226ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:16.685+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.805µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:16.685+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14910157020177557388 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:16.685+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': added 2023-10-21T04:09:16.721+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:09:16.722+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data(shard15964300471522801617) : LSSCleaner initialized 2023-10-21T04:09:16.722+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data(shard15964300471522801617) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:16.755+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:16.758+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:16.770+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery(shard15964300471522801617) : LSSCleaner initialized for recovery 2023-10-21T04:09:16.771+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery(shard15964300471522801617) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery 2023-10-21T04:09:16.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:16.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data], Shared [true] 2023-10-21T04:09:16.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[819200] Data log Head [0] Tail [929792] 2023-10-21T04:09:16.771+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery(shard15964300471522801617) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [819200] 2023-10-21T04:09:16.775+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.778+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex 2023-10-21T04:09:16.778+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is disabled 2023-10-21T04:09:16.782+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.783+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:16.787+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.796+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex 2023-10-21T04:09:16.790+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:1 5020693e9ae5c52168d414b02f922ad8:0 85232c9ba3e4451b95e9e6fc8337e027:0], batchSize: 2 2023-10-21T04:09:16.825+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:16.826+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:0 5020693e9ae5c52168d414b02f922ad8:0 85232c9ba3e4451b95e9e6fc8337e027:0], batchSize: 2 2023-10-21T04:09:16.826+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4 2023-10-21T04:09:16.826+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[3268862397300335529:true 10563635432419104314:true] 2023-10-21T04:09:16.826+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 2.016µs 2023-10-21T04:09:16.828+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [10563635432419104314 3268862397300335529], isLockedForRecovery: true 2023-10-21T04:09:16.829+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [10563635432419104314 3268862397300335529], errMap: map[3268862397300335529: 10563635432419104314:], elapsed: 22.758µs 2023-10-21T04:09:16.834+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.837+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data(shard6611724767191156837) : LSSCleaner initialized 2023-10-21T04:09:16.839+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data(shard6611724767191156837) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.838+05:30 [Info] switched currmeta from 60 -> 61 force true 2023-10-21T04:09:16.839+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex 2023-10-21T04:09:16.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:16.846+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.850+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex 2023-10-21T04:09:16.852+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.852+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex 2023-10-21T04:09:16.852+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.852+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex 2023-10-21T04:09:16.852+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.852+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex 2023-10-21T04:09:16.852+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.852+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex 2023-10-21T04:09:16.852+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.853+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex 2023-10-21T04:09:16.851+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 14889401051926000176, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:16.853+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 14889401051926000176, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:16.853+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.853+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:16.854+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:16.854+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:16.854+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:16.854+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:16.854+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:16.855+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.855+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:16.855+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.857+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:16.857+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.858+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex 2023-10-21T04:09:16.861+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.865+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:16.909+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery(shard6611724767191156837) : LSSCleaner initialized for recovery 2023-10-21T04:09:16.909+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery(shard6611724767191156837) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery 2023-10-21T04:09:16.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:16.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data], Shared [true] 2023-10-21T04:09:16.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1089536] Data log Head [0] Tail [1200128] 2023-10-21T04:09:16.910+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery(shard6611724767191156837) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1089536] 2023-10-21T04:09:16.910+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.910+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex 2023-10-21T04:09:16.911+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.912+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex 2023-10-21T04:09:16.912+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.913+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex 2023-10-21T04:09:16.914+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.914+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex 2023-10-21T04:09:16.913+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.915+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex 2023-10-21T04:09:16.915+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.916+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex 2023-10-21T04:09:16.916+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.917+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex 2023-10-21T04:09:16.918+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.919+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex 2023-10-21T04:09:16.918+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.920+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.920+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:16.920+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.920+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex 2023-10-21T04:09:16.920+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.920+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:16.920+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.920+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex 2023-10-21T04:09:16.922+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex 2023-10-21T04:09:16.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.926+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex 2023-10-21T04:09:16.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.927+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex 2023-10-21T04:09:16.928+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex 2023-10-21T04:09:16.937+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.937+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:16.937+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.938+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:16.938+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.938+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:16.939+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.939+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:16.939+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.939+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:16.940+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.940+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:16.941+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.941+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:16.928+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 18201836893981255880,realInstId: 0, indexDefn DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:16.942+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.942+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex 2023-10-21T04:09:16.942+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.943+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex 2023-10-21T04:09:16.943+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.943+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex 2023-10-21T04:09:16.943+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.943+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex 2023-10-21T04:09:16.944+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.944+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex 2023-10-21T04:09:16.942+05:30 [Info] Indexer::handleRecoverIndex InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:16.942+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.951+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:16.951+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.951+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex 2023-10-21T04:09:16.952+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.952+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:16.952+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.952+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex 2023-10-21T04:09:16.952+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.952+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.952+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:16.953+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.953+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 18201836893981255880, indexDefn DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:16.956+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 104.941904ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:16.953+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 18201836893981255880 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:16.957+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 18201836893981255880 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:16.957+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': added 2023-10-21T04:09:16.953+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex 2023-10-21T04:09:16.958+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is disabled 2023-10-21T04:09:16.958+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.958+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex 2023-10-21T04:09:16.958+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data(shard3268862397300335529) : LSSCleaner initialized 2023-10-21T04:09:16.958+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data(shard3268862397300335529) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:16.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.959+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex 2023-10-21T04:09:16.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.959+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex 2023-10-21T04:09:16.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.959+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex 2023-10-21T04:09:16.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.959+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex 2023-10-21T04:09:16.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.959+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex 2023-10-21T04:09:16.961+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.961+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:16.962+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.962+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery(shard3268862397300335529) : LSSCleaner initialized for recovery 2023-10-21T04:09:16.964+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery(shard3268862397300335529) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery 2023-10-21T04:09:16.966+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data(shard10563635432419104314) : LSSCleaner initialized 2023-10-21T04:09:16.966+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data(shard10563635432419104314) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:16.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:16.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data], Shared [true] 2023-10-21T04:09:16.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[835584] Data log Head [0] Tail [946176] 2023-10-21T04:09:16.967+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery(shard3268862397300335529) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [835584] 2023-10-21T04:09:16.953+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 11.730813ms 2023-10-21T04:09:16.953+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 11.223536ms 2023-10-21T04:09:16.963+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:16.968+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:16.969+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.969+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:16.969+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.969+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery(shard10563635432419104314) : LSSCleaner initialized for recovery 2023-10-21T04:09:16.970+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery(shard10563635432419104314) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:09:16.969+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.970+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:16.971+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:16.971+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.970+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery 2023-10-21T04:09:16.971+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:16.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:16.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data], Shared [true] 2023-10-21T04:09:16.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1105920] Data log Head [0] Tail [1204224] 2023-10-21T04:09:16.971+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery(shard10563635432419104314) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1105920] 2023-10-21T04:09:16.971+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.971+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:16.971+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.971+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:16.972+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.972+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:16.971+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.972+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:16.973+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.973+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:16.973+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.976+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:16.976+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.977+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex 2023-10-21T04:09:16.978+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.976+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.976+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.980+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:16.980+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex 2023-10-21T04:09:16.981+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:16.983+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.983+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:16.983+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.983+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:16.985+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.985+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery(shard15964300471522801617) : replayOffset: 921600 (RP:704512) (CP:921600) 2023-10-21T04:09:16.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [921600] took [213.981494ms] 2023-10-21T04:09:16.985+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9269, memSzIndex 92, numRecords 125 2023-10-21T04:09:16.986+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 89981520 controller mem 13410 2023-10-21T04:09:16.987+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.985+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.985+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex 2023-10-21T04:09:16.987+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 89981520 controller mem 13410 2023-10-21T04:09:16.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 89981520 controller mem 13410 2023-10-21T04:09:16.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.987+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex 2023-10-21T04:09:16.988+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.989+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 89981520 controller mem 13410 2023-10-21T04:09:16.990+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9174, memSzIndex 0, numRecords 117 2023-10-21T04:09:16.990+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 91034436 controller mem 14182 2023-10-21T04:09:16.990+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex 2023-10-21T04:09:16.990+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.991+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:16.990+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.990+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10135, memSzIndex 93, numRecords 132 2023-10-21T04:09:16.992+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10903, memSzIndex 76, numRecords 140 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8462, memSzIndex 0, numRecords 110 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10613, memSzIndex 76, numRecords 137 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 8079, memSzIndex 0, numRecords 106 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1056210, resident 1, memSz 67204, memSzIndex 1014, numRecords 1000 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 92083012 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 11710, memSzIndex 76, numRecords 148 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10081, memSzIndex 91, numRecords 132 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9516, memSzIndex 0, numRecords 120 2023-10-21T04:09:16.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.994+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex 2023-10-21T04:09:16.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1056218, resident 1, memSz 67983, memSzIndex 938, numRecords 1000 2023-10-21T04:09:16.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9633, memSzIndex 77, numRecords 128 2023-10-21T04:09:16.995+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.995+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 7660, memSzIndex 0, numRecords 102 2023-10-21T04:09:16.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.995+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.995+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9803, memSzIndex 77, numRecords 129 2023-10-21T04:09:16.995+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.995+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9791, memSzIndex 0, numRecords 123 2023-10-21T04:09:16.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9800, memSzIndex 0, numRecords 123 2023-10-21T04:09:16.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9824, memSzIndex 93, numRecords 129 2023-10-21T04:09:16.995+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex 2023-10-21T04:09:16.996+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 93131588 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9817, memSzIndex 0, numRecords 123 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.997+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1056158, resident 1, memSz 67809, memSzIndex 890, numRecords 1000 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9771, memSzIndex 76, numRecords 129 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8687, memSzIndex 0, numRecords 112 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8911, memSzIndex 0, numRecords 124 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.997+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9510, memSzIndex 76, numRecords 127 2023-10-21T04:09:16.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 94180164 controller mem 14182 2023-10-21T04:09:16.998+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.998+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 95228740 controller mem 14182 2023-10-21T04:09:16.998+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10725, memSzIndex 92, numRecords 138 2023-10-21T04:09:16.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 95228740 controller mem 14182 2023-10-21T04:09:16.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9849, memSzIndex 76, numRecords 130 2023-10-21T04:09:16.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 95228740 controller mem 14182 2023-10-21T04:09:16.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:16.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 95228740 controller mem 14182 2023-10-21T04:09:17.000+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9091, memSzIndex 0, numRecords 116 2023-10-21T04:09:16.998+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:16.999+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.000+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex 2023-10-21T04:09:17.000+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 95228740 controller mem 14182 2023-10-21T04:09:17.001+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.001+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 95228740 controller mem 14182 2023-10-21T04:09:17.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data(shard15964300471522801617) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [929792] replayOffset [921600] 2023-10-21T04:09:17.000+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex 2023-10-21T04:09:17.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.001+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex 2023-10-21T04:09:17.003+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.004+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex 2023-10-21T04:09:17.002+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.003+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.005+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.005+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex 2023-10-21T04:09:17.006+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex 2023-10-21T04:09:17.007+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.007+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:17.007+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex 2023-10-21T04:09:17.008+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.010+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex 2023-10-21T04:09:17.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [237.136338ms] 2023-10-21T04:09:17.009+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.009+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.011+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.011+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:17.011+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex 2023-10-21T04:09:17.012+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.012+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex 2023-10-21T04:09:17.013+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:17.013+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.014+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:17.014+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.014+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:17.015+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.015+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:17.015+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.016+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.016+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex 2023-10-21T04:09:17.014+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.016+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:17.013+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.019+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.019+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.019+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.020+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.020+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.020+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.021+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.021+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.021+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.021+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.022+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.022+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.023+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.023+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.023+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.024+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.024+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.025+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.025+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.027+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.027+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.027+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.028+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.028+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.029+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.029+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.030+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.030+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.030+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.031+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.031+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.032+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.032+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.033+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.033+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.034+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.034+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.035+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.035+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data], Shared [true]. Built [54] plasmas, took [264.93833ms] 2023-10-21T04:09:17.036+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:17.037+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.037+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:17.038+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.038+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:17.038+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.038+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex 2023-10-21T04:09:17.038+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 11538 recovery UsedSpace 8236 2023-10-21T04:09:17.039+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.039+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex 2023-10-21T04:09:17.040+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.040+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:17.041+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.041+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex 2023-10-21T04:09:17.042+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.042+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex 2023-10-21T04:09:17.043+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.043+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex 2023-10-21T04:09:17.044+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.044+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.044+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:17.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:17.057+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:17.053+05:30 [Info] switched currmeta from 61 -> 62 force true 2023-10-21T04:09:17.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:17.073+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swapper started 2023-10-21T04:09:17.073+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:17.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex started 2023-10-21T04:09:17.073+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.074+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.077+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:17.079+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.080+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.080+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex 2023-10-21T04:09:17.080+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex 2023-10-21T04:09:17.080+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.082+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex 2023-10-21T04:09:17.082+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.082+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:17.083+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.084+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex 2023-10-21T04:09:17.084+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.084+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex 2023-10-21T04:09:17.083+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.084+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.085+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:17.086+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.086+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:17.086+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.087+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:17.087+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.087+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:17.088+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex 2023-10-21T04:09:17.089+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.089+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.089+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.090+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.091+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex 2023-10-21T04:09:17.090+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:17.092+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.092+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex 2023-10-21T04:09:17.092+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.089+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.093+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex 2023-10-21T04:09:17.094+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex 2023-10-21T04:09:17.096+05:30 [Info] switched currmeta from 62 -> 62 force true 2023-10-21T04:09:17.094+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.097+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.097+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.097+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex 2023-10-21T04:09:17.098+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.098+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:17.098+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.098+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex 2023-10-21T04:09:17.099+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.099+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex 2023-10-21T04:09:17.099+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.099+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex 2023-10-21T04:09:17.099+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.100+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex 2023-10-21T04:09:17.100+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.101+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex 2023-10-21T04:09:17.101+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.102+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex 2023-10-21T04:09:17.102+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.103+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex 2023-10-21T04:09:17.103+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.104+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex 2023-10-21T04:09:17.105+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.111+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:17.096+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.111+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:17.112+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.112+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:17.103+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.112+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex 2023-10-21T04:09:17.112+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.113+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex 2023-10-21T04:09:17.113+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.113+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex 2023-10-21T04:09:17.112+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.112+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.118+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:17.118+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:17.122+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.122+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:17.123+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:17.123+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.123+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:17.124+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.124+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:17.125+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.125+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:17.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:17.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.125+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:17.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:17.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:17.128+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:17.129+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.129+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:17.129+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.129+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:17.129+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.129+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:17.130+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:17.130+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:17.130+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:17.130+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:17.130+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:17.131+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.131+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:17.133+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.134+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex 2023-10-21T04:09:17.134+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery(shard6611724767191156837) : replayOffset: 1191936 (RP:1183744) (CP:1191936) 2023-10-21T04:09:17.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1191936] took [224.836418ms] 2023-10-21T04:09:17.134+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 10607, memSzIndex 0, numRecords 106 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12251, memSzIndex 0, numRecords 123 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12489, memSzIndex 0, numRecords 125 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12359, memSzIndex 0, numRecords 124 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11658, memSzIndex 0, numRecords 117 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13131, memSzIndex 0, numRecords 132 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11579, memSzIndex 0, numRecords 116 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 11988, memSzIndex 0, numRecords 120 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12783, memSzIndex 0, numRecords 128 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12277, memSzIndex 0, numRecords 123 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 14710, memSzIndex 0, numRecords 148 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12882, memSzIndex 0, numRecords 129 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13935, memSzIndex 0, numRecords 140 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.136+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 10204, memSzIndex 0, numRecords 102 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12877, memSzIndex 0, numRecords 129 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1055104, resident 1, memSz 63069, memSzIndex 246, numRecords 1000 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1054724, resident 1, memSz 58581, memSzIndex 321, numRecords 1000 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1055958, resident 1, memSz 70503, memSzIndex 369, numRecords 1000 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12921, memSzIndex 0, numRecords 130 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13181, memSzIndex 0, numRecords 132 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12260, memSzIndex 0, numRecords 123 2023-10-21T04:09:17.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.139+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13657, memSzIndex 0, numRecords 137 2023-10-21T04:09:17.139+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.139+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.139+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.139+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1054712, resident 1, memSz 58599, memSzIndex 306, numRecords 1000 2023-10-21T04:09:17.142+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.142+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.142+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.142+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 10974, memSzIndex 0, numRecords 110 2023-10-21T04:09:17.142+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.141+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.142+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.143+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:17.143+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12666, memSzIndex 0, numRecords 127 2023-10-21T04:09:17.143+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.143+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.143+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.143+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12847, memSzIndex 0, numRecords 129 2023-10-21T04:09:17.144+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.144+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11191, memSzIndex 0, numRecords 112 2023-10-21T04:09:17.144+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.144+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.144+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.146+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1055776, resident 1, memSz 65912, memSzIndex 444, numRecords 1000 2023-10-21T04:09:17.146+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.146+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13749, memSzIndex 0, numRecords 138 2023-10-21T04:09:17.146+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.146+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.145+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex 2023-10-21T04:09:17.147+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.148+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex 2023-10-21T04:09:17.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1055958, resident 1, memSz 70487, memSzIndex 370, numRecords 1000 2023-10-21T04:09:17.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1055776, resident 1, memSz 2094, memSzIndex 0, numRecords 21 2023-10-21T04:09:17.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 128885252 controller mem 16890 2023-10-21T04:09:17.150+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data(shard6611724767191156837) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1200128] replayOffset [1191936] 2023-10-21T04:09:17.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [240.165971ms] 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.152+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.151+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.152+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.152+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex 2023-10-21T04:09:17.152+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.152+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.152+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.152+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex 2023-10-21T04:09:17.152+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.152+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex 2023-10-21T04:09:17.153+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.153+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex 2023-10-21T04:09:17.153+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.153+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex 2023-10-21T04:09:17.153+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.154+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.154+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.153+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.154+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex 2023-10-21T04:09:17.154+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.154+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.155+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.155+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.155+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.155+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.155+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.155+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.156+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.156+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.156+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.156+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.156+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.150+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.155+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.157+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.160+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.160+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data], Shared [true]. Built [61] plasmas, took [250.151744ms] 2023-10-21T04:09:17.160+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:17.161+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 12800 recovery UsedSpace 8236 2023-10-21T04:09:17.161+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex 2023-10-21T04:09:17.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.163+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:17.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.163+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:17.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.163+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:17.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.163+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:17.164+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.164+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:17.164+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.164+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:17.164+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.164+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:17.164+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.164+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:17.166+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery(shard3268862397300335529) : replayOffset: 937984 (RP:749568) (CP:937984) 2023-10-21T04:09:17.166+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:17.166+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.167+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex 2023-10-21T04:09:17.167+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.167+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex 2023-10-21T04:09:17.167+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.167+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex 2023-10-21T04:09:17.172+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.172+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex 2023-10-21T04:09:17.172+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.172+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex 2023-10-21T04:09:17.172+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.173+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex 2023-10-21T04:09:17.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.173+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex 2023-10-21T04:09:17.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.173+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex 2023-10-21T04:09:17.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.174+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:17.175+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.175+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:17.175+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.175+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:17.176+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.176+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:17.176+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [937984] took [198.841978ms] 2023-10-21T04:09:17.176+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10319, memSzIndex 76, numRecords 134 2023-10-21T04:09:17.177+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 104788020 controller mem 14310 2023-10-21T04:09:17.177+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.177+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 104788020 controller mem 14310 2023-10-21T04:09:17.177+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8983, memSzIndex 0, numRecords 115 2023-10-21T04:09:17.177+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 104788020 controller mem 14310 2023-10-21T04:09:17.178+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.178+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 104788020 controller mem 14310 2023-10-21T04:09:17.177+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:17.178+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.178+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:17.178+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.179+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.179+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.180+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.180+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9629, memSzIndex 0, numRecords 121 2023-10-21T04:09:17.180+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.180+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.180+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.180+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10626, memSzIndex 77, numRecords 137 2023-10-21T04:09:17.180+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.181+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.181+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.181+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10790, memSzIndex 77, numRecords 139 2023-10-21T04:09:17.181+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.181+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10069, memSzIndex 77, numRecords 132 2023-10-21T04:09:17.181+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.181+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9719, memSzIndex 0, numRecords 122 2023-10-21T04:09:17.182+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.182+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9544, memSzIndex 0, numRecords 120 2023-10-21T04:09:17.182+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.182+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:17.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:17.182+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.183+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1056222, resident 1, memSz 68719, memSzIndex 859, numRecords 1000 2023-10-21T04:09:17.183+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.183+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9237, memSzIndex 76, numRecords 125 2023-10-21T04:09:17.183+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.183+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.183+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.183+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10407, memSzIndex 76, numRecords 135 2023-10-21T04:09:17.184+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.184+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.184+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.184+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1056200, resident 1, memSz 67960, memSzIndex 921, numRecords 1000 2023-10-21T04:09:17.184+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.184+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8367, memSzIndex 0, numRecords 109 2023-10-21T04:09:17.185+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.185+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 8591, memSzIndex 0, numRecords 111 2023-10-21T04:09:17.185+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.185+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10969, memSzIndex 93, numRecords 140 2023-10-21T04:09:17.185+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.185+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9576, memSzIndex 0, numRecords 121 2023-10-21T04:09:17.185+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.186+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 8808, memSzIndex 0, numRecords 113 2023-10-21T04:09:17.186+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.186+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.186+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.186+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1056046, resident 1, memSz 66972, memSzIndex 858, numRecords 1000 2023-10-21T04:09:17.186+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.186+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10205, memSzIndex 77, numRecords 133 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10757, memSzIndex 93, numRecords 138 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.187+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8918, memSzIndex 0, numRecords 124 2023-10-21T04:09:17.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10007, memSzIndex 77, numRecords 131 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.190+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.190+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.190+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.190+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9392, memSzIndex 0, numRecords 119 2023-10-21T04:09:17.191+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.190+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.192+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:17.191+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10298, memSzIndex 93, numRecords 134 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8168, memSzIndex 0, numRecords 107 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9674, memSzIndex 76, numRecords 128 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8675, memSzIndex 0, numRecords 112 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 106885172 controller mem 14310 2023-10-21T04:09:17.193+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data(shard3268862397300335529) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [946176] replayOffset [937984] 2023-10-21T04:09:17.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [225.64042ms] 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex started 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.195+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14910157020177557388 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.195+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14910157020177557388 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:17.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:17.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex] 2023-10-21T04:09:17.195+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex] 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex] 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex stopped 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data], Shared [true]. Built [54] plasmas, took [228.222827ms] 2023-10-21T04:09:17.196+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1056046 memSz 2028 2023-10-21T04:09:17.196+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 1000 metaSz: 2514 rp [1] sn: 4 count: 1000 metaSz: 2579 rp [2] sn: 5 count: 1000 metaSz: 2578 2023-10-21T04:09:17.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:17.196+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.197+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:17.200+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.196+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 37327 recovery UsedSpace 33206 2023-10-21T04:09:17.202+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex 2023-10-21T04:09:17.203+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.203+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:17.203+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.204+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex 2023-10-21T04:09:17.205+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.206+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:17.207+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.207+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex 2023-10-21T04:09:17.207+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.208+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex 2023-10-21T04:09:17.208+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.209+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex 2023-10-21T04:09:17.209+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.210+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex 2023-10-21T04:09:17.209+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:17.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex] 2023-10-21T04:09:17.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex closed 2023-10-21T04:09:17.211+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.211+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex 2023-10-21T04:09:17.211+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:17.212+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:17.212+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:17.212+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:17.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex] 2023-10-21T04:09:17.212+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:17.212+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex] 2023-10-21T04:09:17.212+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.212+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex 2023-10-21T04:09:17.213+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.213+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex 2023-10-21T04:09:17.213+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.214+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex 2023-10-21T04:09:17.216+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.216+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:17.217+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.217+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:17.218+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.218+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:17.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex 2023-10-21T04:09:17.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:17.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:17.219+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swapper started 2023-10-21T04:09:17.220+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.224+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:17.220+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex 2023-10-21T04:09:17.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex started 2023-10-21T04:09:17.225+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.226+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.226+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:17.226+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.227+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:17.227+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.227+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.228+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:17.228+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.228+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:17.233+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery(shard10563635432419104314) : replayOffset: 1196032 (RP:1171456) (CP:1196032) 2023-10-21T04:09:17.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1196032] took [261.68362ms] 2023-10-21T04:09:17.233+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12473, memSzIndex 0, numRecords 125 2023-10-21T04:09:17.233+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.233+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13263, memSzIndex 0, numRecords 133 2023-10-21T04:09:17.234+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.234+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11868, memSzIndex 0, numRecords 119 2023-10-21T04:09:17.234+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.234+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13779, memSzIndex 0, numRecords 138 2023-10-21T04:09:17.234+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.235+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.236+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.236+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 11099, memSzIndex 0, numRecords 111 2023-10-21T04:09:17.236+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.236+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.236+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.236+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.237+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.237+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.237+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.239+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.239+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.239+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.239+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.242+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.242+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.242+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.242+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.243+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.243+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.243+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1055776, resident 1, memSz 66017, memSzIndex 456, numRecords 1000 2023-10-21T04:09:17.243+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.243+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.245+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12044, memSzIndex 0, numRecords 121 2023-10-21T04:09:17.245+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.245+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.245+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.246+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.246+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.246+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1055776, resident 1, memSz 66070, memSzIndex 475, numRecords 1000 2023-10-21T04:09:17.246+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.248+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1055980, resident 1, memSz 70570, memSzIndex 386, numRecords 1000 2023-10-21T04:09:17.248+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.248+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13375, memSzIndex 0, numRecords 134 2023-10-21T04:09:17.248+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.250+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.251+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.251+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.251+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.251+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.251+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.252+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1055776, resident 1, memSz 2093, memSzIndex 0, numRecords 21 2023-10-21T04:09:17.252+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.252+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.252+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.253+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13336, memSzIndex 0, numRecords 134 2023-10-21T04:09:17.253+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.253+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1054716, resident 1, memSz 58647, memSzIndex 306, numRecords 1000 2023-10-21T04:09:17.253+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12183, memSzIndex 0, numRecords 122 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 10692, memSzIndex 0, numRecords 107 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 10883, memSzIndex 0, numRecords 109 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13824, memSzIndex 0, numRecords 139 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11179, memSzIndex 0, numRecords 112 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12097, memSzIndex 0, numRecords 121 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1054714, resident 1, memSz 58631, memSzIndex 306, numRecords 1000 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12754, memSzIndex 0, numRecords 128 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12366, memSzIndex 0, numRecords 124 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 11308, memSzIndex 0, numRecords 113 2023-10-21T04:09:17.254+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13984, memSzIndex 0, numRecords 140 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13131, memSzIndex 0, numRecords 132 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12016, memSzIndex 0, numRecords 120 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11475, memSzIndex 0, numRecords 115 2023-10-21T04:09:17.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1054732, resident 1, memSz 58661, memSzIndex 322, numRecords 1000 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13668, memSzIndex 0, numRecords 137 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13073, memSzIndex 0, numRecords 131 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:09:17.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13459, memSzIndex 0, numRecords 135 2023-10-21T04:09:17.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885012 controller mem 16674 2023-10-21T04:09:17.257+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data(shard10563635432419104314) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1204224] replayOffset [1196032] 2023-10-21T04:09:17.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [286.031532ms] 2023-10-21T04:09:17.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.259+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.260+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.261+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:09:17.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data], Shared [true]. Built [61] plasmas, took [289.441694ms] 2023-10-21T04:09:17.261+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1070 2023-10-21T04:09:17.261+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 1000 metaSz: 2579 rp [1] sn: 5 count: 1000 metaSz: 2578 2023-10-21T04:09:17.261+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 40206 recovery UsedSpace 39565 2023-10-21T04:09:17.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex 2023-10-21T04:09:17.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:17.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:17.269+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex 2023-10-21T04:09:17.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex started 2023-10-21T04:09:17.269+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 18201836893981255880 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:17.270+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.270+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.275+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex] 2023-10-21T04:09:17.275+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:17.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex stopped 2023-10-21T04:09:17.275+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:17.275+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:17.275+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:17.275+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:17.292+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:17.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex] 2023-10-21T04:09:17.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex closed 2023-10-21T04:09:17.293+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:17.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex ... 2023-10-21T04:09:17.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex ... 2023-10-21T04:09:17.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:17.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex sucessfully destroyed 2023-10-21T04:09:17.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex ... 2023-10-21T04:09:17.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex ... 2023-10-21T04:09:17.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:17.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index destroyed 2023-10-21T04:09:17.311+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 18201836893981255880 PartitionId 0 Warmup took 354.345227ms 2023-10-21T04:09:17.312+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 18201836893981255880 partitionId 0 WriterThreads 4 2023-10-21T04:09:17.312+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 18201836893981255880 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index Index: 18201836893981255880 Partition: 0 2023-10-21T04:09:17.312+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 18201836893981255880, shardIdMap: map[0:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:17.312+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 18201836893981255880, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:17.312+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:17.313+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 18201836893981255880 2023-10-21T04:09:17.313+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:17.313+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:17.313+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_1:1697841557312484841] 2023-10-21T04:09:17.313+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:17.313+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:17.314+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 18201836893981255880 Partitions [0] 2023-10-21T04:09:17.314+05:30 [Info] StorageMgr::openSnapshot IndexInst:18201836893981255880 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:17.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:17.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:17.318+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.318+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:17.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 55 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:17.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:17.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:17.327+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:17.327+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex 2023-10-21T04:09:17.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/docIndex started 2023-10-21T04:09:17.327+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.327+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:17.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:17.334+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:17.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:17.336+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.061992ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:17.336+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 87.015µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:17.370+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.371+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.374+05:30 [Info] switched currmeta from 62 -> 64 force true 2023-10-21T04:09:17.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 62 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:17.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:17.388+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.389+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex 2023-10-21T04:09:17.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index/mainIndex started 2023-10-21T04:09:17.390+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14910157020177557388 PartitionId 0 Warmup took 704.860765ms 2023-10-21T04:09:17.390+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14910157020177557388 partitionId 0 WriterThreads 4 2023-10-21T04:09:17.392+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14910157020177557388 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_14910157020177557388_0.index Index: 14910157020177557388 Partition: 0 2023-10-21T04:09:17.392+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14910157020177557388, shardIdMap: map[0:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:17.392+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14910157020177557388, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:17.392+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14910157020177557388 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:17.397+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14910157020177557388 2023-10-21T04:09:17.387+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.397+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.397+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.398+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.400+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14910157020177557388 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:17.401+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:17.402+05:30 [Info] switched currmeta from 64 -> 64 force true 2023-10-21T04:09:17.406+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.905109ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:17.419+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.419+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.414+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:17.421+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14910157020177557388 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:17.430+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14910157020177557388 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:17.431+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14910157020177557388 Partitions [0] 2023-10-21T04:09:17.431+05:30 [Info] StorageMgr::openSnapshot IndexInst:14910157020177557388 Partition:0 No Snapshot Found. 2023-10-21T04:09:17.425+05:30 [Info] switched currmeta from 64 -> 65 force true 2023-10-21T04:09:17.433+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 83.029µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:17.440+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14910157020177557388 Adding Nil Snapshot. 2023-10-21T04:09:17.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9972419390234697, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:09:17.535+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.535+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.542+05:30 [Info] switched currmeta from 65 -> 66 force true 2023-10-21T04:09:17.687+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14910157020177557388:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:17.688+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14910157020177557388 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00755d800 2023-10-21T04:09:17.688+05:30 [Info] handleUpdateIndexRState: Index instance 14910157020177557388 rstate moved to ACTIVE 2023-10-21T04:09:17.692+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:46766:172.31.5.112:12000] 2023-10-21T04:09:17.695+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.359242ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:17.717+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 14741949230702299196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:17.717+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 14741949230702299196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:17.752+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2483169159760864439,realInstId: 0, indexDefn DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:17.762+05:30 [Info] Indexer::handleRecoverIndex InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:17.766+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 3.714754ms 2023-10-21T04:09:17.766+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.962275ms 2023-10-21T04:09:17.766+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2483169159760864439 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:17.766+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2483169159760864439, indexDefn DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:17.767+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 49.450112ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:17.767+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2483169159760864439 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:17.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:17.768+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1056158 memSz 2060 2023-10-21T04:09:17.768+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 1000 metaSz: 2514 rp [1] sn: 4 count: 1000 metaSz: 2580 2023-10-21T04:09:17.769+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 37045 recovery UsedSpace 24620 2023-10-21T04:09:17.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:17.770+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1055958 memSz 882 2023-10-21T04:09:17.770+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 1000 metaSz: 2514 rp [1] sn: 4 count: 1000 metaSz: 2580 2023-10-21T04:09:17.771+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 40531 recovery UsedSpace 22522 2023-10-21T04:09:17.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex 2023-10-21T04:09:17.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:17.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:17.779+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex 2023-10-21T04:09:17.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/mainIndex started 2023-10-21T04:09:17.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex 2023-10-21T04:09:17.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:17.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:17.780+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex 2023-10-21T04:09:17.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index/docIndex started 2023-10-21T04:09:17.781+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2483169159760864439 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:17.781+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:17.781+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:17.795+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.811+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2483169159760864439 PartitionId 0 Warmup took 43.106137ms 2023-10-21T04:09:17.811+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2483169159760864439 partitionId 0 WriterThreads 4 2023-10-21T04:09:17.811+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2483169159760864439 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:17.811+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2483169159760864439 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2483169159760864439_0.index Index: 2483169159760864439 Partition: 0 2023-10-21T04:09:17.812+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2483169159760864439, shardIdMap: map[0:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:17.812+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2483169159760864439, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:17.813+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:17.815+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2483169159760864439 2023-10-21T04:09:17.815+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2483169159760864439 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:17.816+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:17.816+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:17.817+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2483169159760864439 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:17.818+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:17.820+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2483169159760864439 Partitions [0] 2023-10-21T04:09:17.827+05:30 [Info] StorageMgr::openSnapshot IndexInst:2483169159760864439 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:17.837+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.587985ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:17.838+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 80.96µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:17.861+05:30 [Info] switched currmeta from 66 -> 69 force true 2023-10-21T04:09:17.880+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:17.880+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:17.886+05:30 [Info] switched currmeta from 69 -> 69 force true 2023-10-21T04:09:17.912+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46812:172.31.5.112:12000] 2023-10-21T04:09:17.915+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46810:172.31.5.112:12000] 2023-10-21T04:09:17.968+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[18201836893981255880:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:17.968+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 2872726210871813325, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:17.969+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 2872726210871813325, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:17.982+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46814:172.31.5.112:12000] 2023-10-21T04:09:17.993+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46816:172.31.5.112:12000] 2023-10-21T04:09:18.002+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2307463109245789631,realInstId: 0, indexDefn DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:18.002+05:30 [Info] Indexer::handleRecoverIndex InstId: 2307463109245789631 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:18.002+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 388.443µs 2023-10-21T04:09:18.002+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 444.765µs 2023-10-21T04:09:18.003+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.003+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 4 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.003+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.004+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 12047 recovery UsedSpace 8236 2023-10-21T04:09:18.005+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2307463109245789631, indexDefn DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:18.005+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 36.479754ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:18.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.009+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.010+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13003 recovery UsedSpace 8236 2023-10-21T04:09:18.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:18.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.018+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:18.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex started 2023-10-21T04:09:18.018+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.018+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.024+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.024+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:18.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.026+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:18.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex started 2023-10-21T04:09:18.027+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.027+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:18.027+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.028+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.027+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.029+05:30 [Info] switched currmeta from 69 -> 72 force true 2023-10-21T04:09:18.039+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.039+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex] 2023-10-21T04:09:18.041+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:18.043+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex] 2023-10-21T04:09:18.043+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex] 2023-10-21T04:09:18.043+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:18.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex stopped 2023-10-21T04:09:18.044+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.045+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.046+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.047+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.050+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.051+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.073+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46820:172.31.5.112:12000] 2023-10-21T04:09:18.076+05:30 [Info] switched currmeta from 72 -> 72 force true 2023-10-21T04:09:18.095+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex] 2023-10-21T04:09:18.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex closed 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex] 2023-10-21T04:09:18.096+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:18.096+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex] 2023-10-21T04:09:18.096+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex] 2023-10-21T04:09:18.096+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:18.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex stopped 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.096+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.111+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex] 2023-10-21T04:09:18.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex closed 2023-10-21T04:09:18.112+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex ... 2023-10-21T04:09:18.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex ... 2023-10-21T04:09:18.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex ... 2023-10-21T04:09:18.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex ... 2023-10-21T04:09:18.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index destroyed 2023-10-21T04:09:18.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.130+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.130+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.149+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.149+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 55 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:18.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.154+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:18.154+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex 2023-10-21T04:09:18.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/docIndex started 2023-10-21T04:09:18.154+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.154+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 62 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:18.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.173+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:18.173+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex 2023-10-21T04:09:18.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index/mainIndex started 2023-10-21T04:09:18.174+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 4 Warmup took 170.388402ms 2023-10-21T04:09:18.174+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 4 WriterThreads 1 2023-10-21T04:09:18.174+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_4.index Index: 2307463109245789631 Partition: 4 2023-10-21T04:09:18.174+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.174+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 5 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.174+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.174+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.174+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.175+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.175+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 12226 recovery UsedSpace 8236 2023-10-21T04:09:18.175+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13212 recovery UsedSpace 8236 2023-10-21T04:09:18.178+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46824:172.31.5.112:12000] 2023-10-21T04:09:18.192+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46826:172.31.5.112:12000] 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex started 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex started 2023-10-21T04:09:18.194+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:18.194+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex] 2023-10-21T04:09:18.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:18.194+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex] 2023-10-21T04:09:18.194+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex] 2023-10-21T04:09:18.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:18.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex stopped 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.195+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.205+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex] 2023-10-21T04:09:18.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex closed 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex] 2023-10-21T04:09:18.206+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:18.206+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex] 2023-10-21T04:09:18.206+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex] 2023-10-21T04:09:18.206+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex stopped 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.211+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex] 2023-10-21T04:09:18.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex closed 2023-10-21T04:09:18.211+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex ... 2023-10-21T04:09:18.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex ... 2023-10-21T04:09:18.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex ... 2023-10-21T04:09:18.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex ... 2023-10-21T04:09:18.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index destroyed 2023-10-21T04:09:18.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.227+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.227+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.228+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.228+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.231+05:30 [Info] switched currmeta from 72 -> 73 force true 2023-10-21T04:09:18.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.232+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.232+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 56 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:18.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.239+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:18.239+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex 2023-10-21T04:09:18.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/docIndex started 2023-10-21T04:09:18.240+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.240+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 63 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:18.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.240+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:18.240+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex 2023-10-21T04:09:18.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index/mainIndex started 2023-10-21T04:09:18.240+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 5 Warmup took 65.971427ms 2023-10-21T04:09:18.240+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 5 WriterThreads 1 2023-10-21T04:09:18.240+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_5.index Index: 2307463109245789631 Partition: 5 2023-10-21T04:09:18.241+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.241+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 6 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.241+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.241+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.241+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.241+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 12226 recovery UsedSpace 8236 2023-10-21T04:09:18.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.241+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.242+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13429 recovery UsedSpace 8236 2023-10-21T04:09:18.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:18.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.245+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:18.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex started 2023-10-21T04:09:18.245+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.245+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:18.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.246+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:18.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex started 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.247+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:18.247+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex] 2023-10-21T04:09:18.247+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:18.247+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex] 2023-10-21T04:09:18.247+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex] 2023-10-21T04:09:18.247+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:18.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex stopped 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.247+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.251+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex] 2023-10-21T04:09:18.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex closed 2023-10-21T04:09:18.252+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.252+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.252+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.252+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex] 2023-10-21T04:09:18.252+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:18.253+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex] 2023-10-21T04:09:18.253+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex] 2023-10-21T04:09:18.253+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:18.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex stopped 2023-10-21T04:09:18.253+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.253+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.253+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.254+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.261+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex] 2023-10-21T04:09:18.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex closed 2023-10-21T04:09:18.261+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex ... 2023-10-21T04:09:18.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex ... 2023-10-21T04:09:18.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex ... 2023-10-21T04:09:18.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex ... 2023-10-21T04:09:18.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index destroyed 2023-10-21T04:09:18.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.284+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.284+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.287+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.287+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.297+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:18.300+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46828:172.31.5.112:12000] 2023-10-21T04:09:18.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 57 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:18.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.303+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:18.303+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex 2023-10-21T04:09:18.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/docIndex started 2023-10-21T04:09:18.303+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.303+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 64 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:18.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.306+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:18.306+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex 2023-10-21T04:09:18.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index/mainIndex started 2023-10-21T04:09:18.306+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 6 Warmup took 65.082524ms 2023-10-21T04:09:18.306+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 6 WriterThreads 1 2023-10-21T04:09:18.306+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_6.index Index: 2307463109245789631 Partition: 6 2023-10-21T04:09:18.306+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.306+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 7 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.307+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.307+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 12412 recovery UsedSpace 8236 2023-10-21T04:09:18.307+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.307+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.308+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.308+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13429 recovery UsedSpace 8236 2023-10-21T04:09:18.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:18.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.315+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:18.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex started 2023-10-21T04:09:18.315+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.315+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:18.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.316+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:18.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex started 2023-10-21T04:09:18.316+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:18.316+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex] 2023-10-21T04:09:18.316+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:18.316+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex] 2023-10-21T04:09:18.316+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex] 2023-10-21T04:09:18.316+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:18.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex stopped 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.316+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.317+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46830:172.31.5.112:12000] 2023-10-21T04:09:18.325+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.325+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex] 2023-10-21T04:09:18.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex closed 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex] 2023-10-21T04:09:18.326+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:18.326+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex] 2023-10-21T04:09:18.326+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex] 2023-10-21T04:09:18.326+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:18.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex stopped 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.332+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex] 2023-10-21T04:09:18.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex closed 2023-10-21T04:09:18.332+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex ... 2023-10-21T04:09:18.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex ... 2023-10-21T04:09:18.334+05:30 [Info] switched currmeta from 73 -> 73 force true 2023-10-21T04:09:18.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex ... 2023-10-21T04:09:18.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex ... 2023-10-21T04:09:18.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index destroyed 2023-10-21T04:09:18.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.342+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.342+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.349+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.349+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 58 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:18.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.351+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:18.352+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex 2023-10-21T04:09:18.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/docIndex started 2023-10-21T04:09:18.352+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.352+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.355+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 65 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:18.355+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.355+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.355+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:18.355+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex 2023-10-21T04:09:18.355+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index/mainIndex started 2023-10-21T04:09:18.355+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 7 Warmup took 48.721075ms 2023-10-21T04:09:18.355+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 7 WriterThreads 1 2023-10-21T04:09:18.355+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_7.index Index: 2307463109245789631 Partition: 7 2023-10-21T04:09:18.355+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.356+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 8 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.356+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.356+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 12603 recovery UsedSpace 8236 2023-10-21T04:09:18.357+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.357+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.357+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.357+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13653 recovery UsedSpace 8236 2023-10-21T04:09:18.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:18.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.362+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:18.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex started 2023-10-21T04:09:18.362+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.362+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:18.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.363+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:18.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex started 2023-10-21T04:09:18.363+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:18.363+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.363+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.363+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.363+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex] 2023-10-21T04:09:18.363+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:18.363+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex] 2023-10-21T04:09:18.363+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex] 2023-10-21T04:09:18.363+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:18.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex stopped 2023-10-21T04:09:18.363+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.363+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.363+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.363+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.364+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.364+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex] 2023-10-21T04:09:18.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex closed 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex] 2023-10-21T04:09:18.369+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:18.369+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex] 2023-10-21T04:09:18.369+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex] 2023-10-21T04:09:18.369+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:18.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex stopped 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.369+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.376+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex] 2023-10-21T04:09:18.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex closed 2023-10-21T04:09:18.376+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex ... 2023-10-21T04:09:18.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex ... 2023-10-21T04:09:18.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex ... 2023-10-21T04:09:18.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex ... 2023-10-21T04:09:18.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index destroyed 2023-10-21T04:09:18.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.389+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.389+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.392+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.395+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46832:172.31.5.112:12000] 2023-10-21T04:09:18.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 66 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:18.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.411+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:18.411+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex 2023-10-21T04:09:18.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/mainIndex started 2023-10-21T04:09:18.411+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.411+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 59 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:18.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.415+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:18.415+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex 2023-10-21T04:09:18.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index/docIndex started 2023-10-21T04:09:18.415+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 8 Warmup took 59.590222ms 2023-10-21T04:09:18.416+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 8 WriterThreads 1 2023-10-21T04:09:18.416+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_8.index Index: 2307463109245789631 Partition: 8 2023-10-21T04:09:18.416+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.417+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 1 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.416+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.416+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.418+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.419+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13653 recovery UsedSpace 8236 2023-10-21T04:09:18.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.419+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.420+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 12800 recovery UsedSpace 8236 2023-10-21T04:09:18.429+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.429+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:18.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.433+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:18.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex started 2023-10-21T04:09:18.433+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.434+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.434+05:30 [Info] switched currmeta from 73 -> 74 force true 2023-10-21T04:09:18.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:18.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:18.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex started 2023-10-21T04:09:18.440+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:18.440+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex] 2023-10-21T04:09:18.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:18.440+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex] 2023-10-21T04:09:18.440+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex] 2023-10-21T04:09:18.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:18.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex stopped 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.440+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9924508484015576, throttleDelayMs (new, old, change): (9, 5, 4) 2023-10-21T04:09:18.455+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex] 2023-10-21T04:09:18.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex closed 2023-10-21T04:09:18.456+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.458+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.458+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.458+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex] 2023-10-21T04:09:18.458+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:18.458+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex] 2023-10-21T04:09:18.458+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex] 2023-10-21T04:09:18.458+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:18.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex stopped 2023-10-21T04:09:18.458+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.458+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.458+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.458+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.463+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex] 2023-10-21T04:09:18.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex closed 2023-10-21T04:09:18.463+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex ... 2023-10-21T04:09:18.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex ... 2023-10-21T04:09:18.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex ... 2023-10-21T04:09:18.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex ... 2023-10-21T04:09:18.480+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.480+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index destroyed 2023-10-21T04:09:18.487+05:30 [Info] switched currmeta from 74 -> 75 force true 2023-10-21T04:09:18.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.490+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.491+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.493+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.493+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 67 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:18.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.511+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:18.511+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex 2023-10-21T04:09:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/mainIndex started 2023-10-21T04:09:18.511+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.511+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 60 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.511+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:18.511+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex 2023-10-21T04:09:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index/docIndex started 2023-10-21T04:09:18.512+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 1 Warmup took 94.794267ms 2023-10-21T04:09:18.512+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.512+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.513+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 1 WriterThreads 1 2023-10-21T04:09:18.513+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_1.index Index: 2307463109245789631 Partition: 1 2023-10-21T04:09:18.514+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.514+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 2 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.516+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13884 recovery UsedSpace 8236 2023-10-21T04:09:18.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.517+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.518+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 12800 recovery UsedSpace 8236 2023-10-21T04:09:18.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:18.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.520+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:18.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex started 2023-10-21T04:09:18.520+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.520+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.523+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.523+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.533+05:30 [Info] switched currmeta from 75 -> 76 force true 2023-10-21T04:09:18.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:18.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.534+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:18.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex started 2023-10-21T04:09:18.534+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:18.535+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.535+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.535+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.535+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.535+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.535+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex] 2023-10-21T04:09:18.536+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:18.536+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex] 2023-10-21T04:09:18.536+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex] 2023-10-21T04:09:18.536+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:18.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex stopped 2023-10-21T04:09:18.537+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.537+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.537+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.537+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.546+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex] 2023-10-21T04:09:18.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex closed 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex] 2023-10-21T04:09:18.547+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:18.547+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex] 2023-10-21T04:09:18.547+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex] 2023-10-21T04:09:18.547+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:18.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex stopped 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.547+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.555+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex] 2023-10-21T04:09:18.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex closed 2023-10-21T04:09:18.556+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex ... 2023-10-21T04:09:18.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex ... 2023-10-21T04:09:18.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex ... 2023-10-21T04:09:18.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex ... 2023-10-21T04:09:18.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index destroyed 2023-10-21T04:09:18.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.566+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.582+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.582+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 68 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:18.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.588+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:18.588+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex 2023-10-21T04:09:18.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/mainIndex started 2023-10-21T04:09:18.588+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.588+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 61 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:18.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.593+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:18.593+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex 2023-10-21T04:09:18.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index/docIndex started 2023-10-21T04:09:18.594+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 2 Warmup took 78.991589ms 2023-10-21T04:09:18.594+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 2 WriterThreads 1 2023-10-21T04:09:18.594+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_2.index Index: 2307463109245789631 Partition: 2 2023-10-21T04:09:18.594+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2307463109245789631 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:18.594+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2307463109245789631 PartitionId: 3 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:18.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.594+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.595+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13003 recovery UsedSpace 8236 2023-10-21T04:09:18.595+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.595+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.595+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.596+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 14124 recovery UsedSpace 8236 2023-10-21T04:09:18.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:18.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.601+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:18.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex started 2023-10-21T04:09:18.601+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.602+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:18.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.602+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:18.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex started 2023-10-21T04:09:18.602+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:18.602+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.602+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.602+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.602+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex] 2023-10-21T04:09:18.602+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:18.602+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex] 2023-10-21T04:09:18.602+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex] 2023-10-21T04:09:18.602+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:18.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex stopped 2023-10-21T04:09:18.602+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.602+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.602+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.603+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.603+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.603+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex] 2023-10-21T04:09:18.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex closed 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex] 2023-10-21T04:09:18.608+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:18.608+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex] 2023-10-21T04:09:18.608+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex] 2023-10-21T04:09:18.608+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:18.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex stopped 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.608+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.614+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex] 2023-10-21T04:09:18.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex closed 2023-10-21T04:09:18.614+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex ... 2023-10-21T04:09:18.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex ... 2023-10-21T04:09:18.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex sucessfully destroyed 2023-10-21T04:09:18.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex ... 2023-10-21T04:09:18.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex ... 2023-10-21T04:09:18.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index destroyed 2023-10-21T04:09:18.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:18.622+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.622+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:18.625+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.625+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 62 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:18.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:18.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:18.627+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:18.627+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex 2023-10-21T04:09:18.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/docIndex started 2023-10-21T04:09:18.627+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.627+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#2307463109245789631:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 69 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:18.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:18.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:18.630+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:18.630+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex 2023-10-21T04:09:18.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index/mainIndex started 2023-10-21T04:09:18.630+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2307463109245789631 PartitionId 3 Warmup took 35.458108ms 2023-10-21T04:09:18.630+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2307463109245789631 partitionId 3 WriterThreads 1 2023-10-21T04:09:18.630+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2307463109245789631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_2307463109245789631_3.index Index: 2307463109245789631 Partition: 3 2023-10-21T04:09:18.630+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2307463109245789631, shardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:18.630+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2307463109245789631, partnShardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:18.630+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2307463109245789631 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:18.631+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2307463109245789631 2023-10-21T04:09:18.631+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2307463109245789631 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:18.631+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:18.631+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:18.631+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2307463109245789631 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:18.631+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2307463109245789631 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:18.631+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2307463109245789631 Partitions [5 6 7 8 1 2 3 4] 2023-10-21T04:09:18.631+05:30 [Info] StorageMgr::openSnapshot IndexInst:2307463109245789631 Partition:1 No Snapshot Found. 2023-10-21T04:09:18.631+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2307463109245789631 Adding Nil Snapshot. 2023-10-21T04:09:18.631+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.631+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#2307463109245789631:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.653+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.006754ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:18.653+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 100.065µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:18.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.687+05:30 [Info] switched currmeta from 76 -> 77 force true 2023-10-21T04:09:18.770+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2483169159760864439:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:18.770+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 2374548411389959576, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:18.770+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 2374548411389959576, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:18.805+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 249283441109236971,realInstId: 0, indexDefn DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:18.805+05:30 [Info] Indexer::handleRecoverIndex InstId: 249283441109236971 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:18.805+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 372.772µs 2023-10-21T04:09:18.805+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 421.16µs 2023-10-21T04:09:18.805+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:18.806+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 4 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:18.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:18.806+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.807+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 11872 recovery UsedSpace 8236 2023-10-21T04:09:18.807+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 249283441109236971, indexDefn DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:18.807+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 36.611339ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:18.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:18.807+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13212 recovery UsedSpace 8236 2023-10-21T04:09:18.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 59 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:18.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:18.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:18.835+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:18.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex started 2023-10-21T04:09:18.835+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.835+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:18.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:18.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:18.837+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:18.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex started 2023-10-21T04:09:18.837+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:18.837+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex] 2023-10-21T04:09:18.838+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex] 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex] 2023-10-21T04:09:18.838+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:18.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex stopped 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.838+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.839+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:18.839+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:18.842+05:30 [Info] switched currmeta from 77 -> 78 force true 2023-10-21T04:09:18.844+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex] 2023-10-21T04:09:18.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex closed 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex] 2023-10-21T04:09:18.845+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex] 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex] 2023-10-21T04:09:18.845+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:18.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex stopped 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.864+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex] 2023-10-21T04:09:18.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex closed 2023-10-21T04:09:18.864+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex ... 2023-10-21T04:09:18.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex ... 2023-10-21T04:09:18.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex sucessfully destroyed 2023-10-21T04:09:18.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex ... 2023-10-21T04:09:18.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 59 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex ... 2023-10-21T04:09:18.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index destroyed 2023-10-21T04:09:18.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:18.877+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.877+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:18.884+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.884+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 56 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:18.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:18.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:18.889+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:18.889+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex 2023-10-21T04:09:18.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/docIndex started 2023-10-21T04:09:18.889+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.889+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 63 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:18.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:18.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:18.891+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:18.891+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex 2023-10-21T04:09:18.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index/mainIndex started 2023-10-21T04:09:18.891+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 4 Warmup took 84.947392ms 2023-10-21T04:09:18.891+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 4 WriterThreads 1 2023-10-21T04:09:18.891+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_4.index Index: 249283441109236971 Partition: 4 2023-10-21T04:09:18.891+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:18.891+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 5 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:18.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:18.892+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.892+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 12047 recovery UsedSpace 8236 2023-10-21T04:09:18.892+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.892+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:18.893+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.893+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13429 recovery UsedSpace 8236 2023-10-21T04:09:18.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:18.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:18.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:18.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:18.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex started 2023-10-21T04:09:18.899+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.899+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 60 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:18.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:18.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:18.900+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:18.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex started 2023-10-21T04:09:18.900+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:18.900+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex] 2023-10-21T04:09:18.900+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:18.901+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex] 2023-10-21T04:09:18.901+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex] 2023-10-21T04:09:18.901+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:18.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex stopped 2023-10-21T04:09:18.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.905+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex] 2023-10-21T04:09:18.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex closed 2023-10-21T04:09:18.905+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:18.905+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.905+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.905+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex] 2023-10-21T04:09:18.905+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:18.905+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex] 2023-10-21T04:09:18.905+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex] 2023-10-21T04:09:18.906+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:18.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex stopped 2023-10-21T04:09:18.906+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.906+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.906+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.906+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:18.915+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:18.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex] 2023-10-21T04:09:18.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex closed 2023-10-21T04:09:18.915+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:18.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex ... 2023-10-21T04:09:18.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex ... 2023-10-21T04:09:18.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex sucessfully destroyed 2023-10-21T04:09:18.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex ... 2023-10-21T04:09:18.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 60 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex ... 2023-10-21T04:09:18.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index destroyed 2023-10-21T04:09:18.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:18.938+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.938+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:18.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:18.948+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:18.948+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:18.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 64 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:18.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:18.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:18.967+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:18.967+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex 2023-10-21T04:09:18.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/mainIndex started 2023-10-21T04:09:18.967+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.967+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 57 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:18.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:18.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:18.980+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:18.980+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex 2023-10-21T04:09:18.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index/docIndex started 2023-10-21T04:09:18.981+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 5 Warmup took 89.343669ms 2023-10-21T04:09:18.981+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.981+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.982+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 5 WriterThreads 1 2023-10-21T04:09:18.982+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_5.index Index: 249283441109236971 Partition: 5 2023-10-21T04:09:18.982+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:18.983+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 6 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:18.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:18.984+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.984+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13429 recovery UsedSpace 8236 2023-10-21T04:09:18.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:18.985+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:18.987+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 12226 recovery UsedSpace 8236 2023-10-21T04:09:18.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 61 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:18.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:18.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:18.989+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:18.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex started 2023-10-21T04:09:18.990+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.990+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:18.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:18.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:18.992+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:18.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex started 2023-10-21T04:09:18.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:18.993+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:18.992+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:18.994+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:18.994+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:18.994+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:18.995+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:18.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex] 2023-10-21T04:09:18.995+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:18.995+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex] 2023-10-21T04:09:18.995+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex] 2023-10-21T04:09:18.995+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:18.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex stopped 2023-10-21T04:09:18.995+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:18.995+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:18.995+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:18.995+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.000+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex] 2023-10-21T04:09:19.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex closed 2023-10-21T04:09:19.001+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:19.001+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.001+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.002+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex] 2023-10-21T04:09:19.002+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:19.002+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex] 2023-10-21T04:09:19.002+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex] 2023-10-21T04:09:19.002+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:19.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex stopped 2023-10-21T04:09:19.003+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.003+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.003+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.003+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.011+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2307463109245789631:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:19.011+05:30 [Info] Indexer::handleMergePartition Source 12845014165282783025 Target 2307463109245789631 2023-10-21T04:09:19.011+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2307463109245789631 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007e49ec0 2023-10-21T04:09:19.011+05:30 [Info] MergePartition: sent async request to update index instance 2307463109245789631 rstate moved to ACTIVE 2023-10-21T04:09:19.018+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex] 2023-10-21T04:09:19.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex closed 2023-10-21T04:09:19.019+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.44509ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:19.019+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 12845014165282783025, realInst: 2307463109245789631 2023-10-21T04:09:19.023+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:19.031+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 18323460706027642575, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:19.032+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 18323460706027642575, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:19.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex ... 2023-10-21T04:09:19.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex ... 2023-10-21T04:09:19.039+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.039+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex sucessfully destroyed 2023-10-21T04:09:19.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex ... 2023-10-21T04:09:19.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 61 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex ... 2023-10-21T04:09:19.082+05:30 [Info] switched currmeta from 78 -> 81 force true 2023-10-21T04:09:19.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index destroyed 2023-10-21T04:09:19.095+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2288923528993975684,realInstId: 0, indexDefn DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:19.096+05:30 [Info] Indexer::handleRecoverIndex InstId: 2288923528993975684 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:19.096+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 429.341µs 2023-10-21T04:09:19.096+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 508.324µs 2023-10-21T04:09:19.096+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2288923528993975684 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:19.097+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2288923528993975684, indexDefn DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:19.100+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 69.707849ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:19.100+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.101+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.105+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2288923528993975684 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:19.109+05:30 [Info] switched currmeta from 81 -> 82 force true 2023-10-21T04:09:19.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.116+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:19.117+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.118+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 13212 recovery UsedSpace 8236 2023-10-21T04:09:19.116+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:19.125+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.126+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 14124 recovery UsedSpace 8236 2023-10-21T04:09:19.127+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.128+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.133+05:30 [Info] switched currmeta from 82 -> 82 force true 2023-10-21T04:09:19.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.134+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.134+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:19.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:19.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:19.137+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:19.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex started 2023-10-21T04:09:19.139+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.141+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 58 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:19.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.144+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:19.145+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex 2023-10-21T04:09:19.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/docIndex started 2023-10-21T04:09:19.145+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.145+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 65 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:19.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.154+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:19.155+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex 2023-10-21T04:09:19.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index/mainIndex started 2023-10-21T04:09:19.155+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 6 Warmup took 171.850596ms 2023-10-21T04:09:19.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.159+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 6 WriterThreads 1 2023-10-21T04:09:19.159+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_6.index Index: 249283441109236971 Partition: 6 2023-10-21T04:09:19.159+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:19.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:19.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:19.172+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:19.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex started 2023-10-21T04:09:19.174+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 7 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.176+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.177+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.177+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.176+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2288923528993975684 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:19.177+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2288923528993975684 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:19.177+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.176+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.178+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex] 2023-10-21T04:09:19.178+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.179+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.179+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.187+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 0 InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:19.189+05:30 [Info] switched currmeta from 82 -> 83 force true 2023-10-21T04:09:19.188+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 0 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:19.190+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 0 InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:19.190+05:30 [Info] ShardRebalancer::computeProgress 0.7029407894736843 2023-10-21T04:09:19.192+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 12412 recovery UsedSpace 8236 2023-10-21T04:09:19.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:19.194+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex] 2023-10-21T04:09:19.194+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex] 2023-10-21T04:09:19.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:19.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex stopped 2023-10-21T04:09:19.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.196+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13653 recovery UsedSpace 8236 2023-10-21T04:09:19.202+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex] 2023-10-21T04:09:19.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex closed 2023-10-21T04:09:19.203+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:19.203+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.203+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.204+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex] 2023-10-21T04:09:19.204+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:19.204+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex] 2023-10-21T04:09:19.205+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex] 2023-10-21T04:09:19.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:19.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.205+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:19.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex started 2023-10-21T04:09:19.206+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.205+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:19.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex stopped 2023-10-21T04:09:19.206+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.207+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.206+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.207+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.207+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:19.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.208+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:19.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex started 2023-10-21T04:09:19.209+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.209+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.210+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:19.210+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:19.210+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.210+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.210+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex] 2023-10-21T04:09:19.211+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:19.211+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex] 2023-10-21T04:09:19.212+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex] 2023-10-21T04:09:19.212+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:19.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex stopped 2023-10-21T04:09:19.212+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.213+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.213+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.214+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex] 2023-10-21T04:09:19.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex closed 2023-10-21T04:09:19.213+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.217+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:19.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex ... 2023-10-21T04:09:19.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex ... 2023-10-21T04:09:19.218+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex] 2023-10-21T04:09:19.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex closed 2023-10-21T04:09:19.218+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:19.218+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.218+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.218+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.218+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno cache..., established connections: map[172.31.5.112:46892:172.31.5.112:12000] 2023-10-21T04:09:19.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex] 2023-10-21T04:09:19.218+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:19.219+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex] 2023-10-21T04:09:19.219+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex] 2023-10-21T04:09:19.219+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:19.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex stopped 2023-10-21T04:09:19.219+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.219+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.219+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.219+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.227+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex] 2023-10-21T04:09:19.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex closed 2023-10-21T04:09:19.228+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:19.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex ... 2023-10-21T04:09:19.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex ... 2023-10-21T04:09:19.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex sucessfully destroyed 2023-10-21T04:09:19.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex ... 2023-10-21T04:09:19.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex ... 2023-10-21T04:09:19.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex sucessfully destroyed 2023-10-21T04:09:19.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex sucessfully destroyed 2023-10-21T04:09:19.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex ... 2023-10-21T04:09:19.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex ... 2023-10-21T04:09:19.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index destroyed 2023-10-21T04:09:19.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.264+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:19.265+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.266+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#2288923528993975684:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.272+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:19.287+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.288+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#2288923528993975684:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 66 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:19.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:19.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex 2023-10-21T04:09:19.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/mainIndex started 2023-10-21T04:09:19.288+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.288+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 63 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:19.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:19.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:19.293+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:19.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 59 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:19.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.293+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:19.293+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex 2023-10-21T04:09:19.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index/docIndex started 2023-10-21T04:09:19.293+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 7 Warmup took 117.751554ms 2023-10-21T04:09:19.294+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.294+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 7 WriterThreads 1 2023-10-21T04:09:19.294+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.294+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_7.index Index: 249283441109236971 Partition: 7 2023-10-21T04:09:19.294+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.294+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 8 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 70 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:19.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:19.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:19.295+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:19.295+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex 2023-10-21T04:09:19.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/mainIndex started 2023-10-21T04:09:19.295+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#2288923528993975684:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.295+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.295+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#2288923528993975684:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.295+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 12412 recovery UsedSpace 8236 2023-10-21T04:09:19.294+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex 2023-10-21T04:09:19.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index/docIndex started 2023-10-21T04:09:19.296+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#2288923528993975684:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.296+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#2288923528993975684:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.296+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2288923528993975684 PartitionId 0 Warmup took 190.453147ms 2023-10-21T04:09:19.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.296+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.297+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2288923528993975684 partitionId 0 WriterThreads 4 2023-10-21T04:09:19.297+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2288923528993975684 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:19.297+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2288923528993975684 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_2288923528993975684_0.index Index: 2288923528993975684 Partition: 0 2023-10-21T04:09:19.297+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2288923528993975684, shardIdMap: map[0:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:19.297+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2288923528993975684, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:19.298+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2288923528993975684 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:19.297+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13653 recovery UsedSpace 8236 2023-10-21T04:09:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.300+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex started 2023-10-21T04:09:19.300+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.300+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.299+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2288923528993975684 2023-10-21T04:09:19.300+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2288923528993975684 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:19.300+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:19.300+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:19.300+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2288923528993975684 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:19.301+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2288923528993975684 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:19.301+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2288923528993975684 Partitions [0] 2023-10-21T04:09:19.301+05:30 [Info] StorageMgr::openSnapshot IndexInst:2288923528993975684 Partition:0 No Snapshot Found. 2023-10-21T04:09:19.301+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2288923528993975684 Adding Nil Snapshot. 2023-10-21T04:09:19.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 55 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:19.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.313+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:19.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex started 2023-10-21T04:09:19.313+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:19.313+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:19.313+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.313+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.313+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.313+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex] 2023-10-21T04:09:19.313+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:19.314+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex] 2023-10-21T04:09:19.314+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex] 2023-10-21T04:09:19.314+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:19.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex stopped 2023-10-21T04:09:19.314+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.314+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.314+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.314+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.314+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.311+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.546077ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:19.318+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 1.844776ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.325+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex] 2023-10-21T04:09:19.325+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex closed 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.325+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex] 2023-10-21T04:09:19.325+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex] 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex] 2023-10-21T04:09:19.325+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:19.325+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex stopped 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.325+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.330+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex] 2023-10-21T04:09:19.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex closed 2023-10-21T04:09:19.330+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:19.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex ... 2023-10-21T04:09:19.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex ... 2023-10-21T04:09:19.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex sucessfully destroyed 2023-10-21T04:09:19.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex ... 2023-10-21T04:09:19.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 55 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex ... 2023-10-21T04:09:19.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index destroyed 2023-10-21T04:09:19.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.341+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.341+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.347+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.347+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.361+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.361+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 60 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:19.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.367+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:19.367+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex 2023-10-21T04:09:19.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/docIndex started 2023-10-21T04:09:19.367+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.367+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 67 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:19.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.371+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:19.371+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex 2023-10-21T04:09:19.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index/mainIndex started 2023-10-21T04:09:19.371+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 8 Warmup took 77.034723ms 2023-10-21T04:09:19.372+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.372+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.372+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 8 WriterThreads 1 2023-10-21T04:09:19.372+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_8.index Index: 249283441109236971 Partition: 8 2023-10-21T04:09:19.372+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.372+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 1 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.373+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.373+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.373+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 12603 recovery UsedSpace 8236 2023-10-21T04:09:19.373+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13884 recovery UsedSpace 8236 2023-10-21T04:09:19.379+05:30 [Info] switched currmeta from 83 -> 84 force true 2023-10-21T04:09:19.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:19.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.403+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:19.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex started 2023-10-21T04:09:19.403+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.403+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 56 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:19.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.404+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:19.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex started 2023-10-21T04:09:19.404+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:19.404+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:19.404+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.404+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.404+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.404+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex] 2023-10-21T04:09:19.404+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:19.404+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex] 2023-10-21T04:09:19.404+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex] 2023-10-21T04:09:19.404+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:19.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex stopped 2023-10-21T04:09:19.405+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.405+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.405+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.405+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.405+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.416+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex] 2023-10-21T04:09:19.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex closed 2023-10-21T04:09:19.416+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:19.416+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.416+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.416+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex] 2023-10-21T04:09:19.416+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:19.416+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex] 2023-10-21T04:09:19.417+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex] 2023-10-21T04:09:19.417+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:19.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex stopped 2023-10-21T04:09:19.417+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.417+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.417+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.417+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.433+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex] 2023-10-21T04:09:19.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex closed 2023-10-21T04:09:19.433+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:19.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex ... 2023-10-21T04:09:19.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex ... 2023-10-21T04:09:19.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex sucessfully destroyed 2023-10-21T04:09:19.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex ... 2023-10-21T04:09:19.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 56 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex ... 2023-10-21T04:09:19.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index destroyed 2023-10-21T04:09:19.448+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029407894736843 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:19.449+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:19.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9897341832481235, throttleDelayMs (new, old, change): (10, 9, 1) 2023-10-21T04:09:19.453+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:09:19.453+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:09:19.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.454+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.454+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.461+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.461+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 61 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:19.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:19.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex 2023-10-21T04:09:19.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/docIndex started 2023-10-21T04:09:19.467+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.467+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 68 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:19.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.471+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:19.471+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex 2023-10-21T04:09:19.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index/mainIndex started 2023-10-21T04:09:19.472+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 1 Warmup took 99.024603ms 2023-10-21T04:09:19.472+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 1 WriterThreads 1 2023-10-21T04:09:19.472+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_1.index Index: 249283441109236971 Partition: 1 2023-10-21T04:09:19.472+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.472+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 2 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.472+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.472+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.473+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.473+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 12800 recovery UsedSpace 8236 2023-10-21T04:09:19.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.474+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.474+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13884 recovery UsedSpace 8236 2023-10-21T04:09:19.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:19.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.485+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:19.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex started 2023-10-21T04:09:19.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 57 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:19.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.486+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:19.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex started 2023-10-21T04:09:19.486+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:19.486+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex] 2023-10-21T04:09:19.487+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex] 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex] 2023-10-21T04:09:19.487+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:19.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex stopped 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.488+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex] 2023-10-21T04:09:19.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex closed 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex] 2023-10-21T04:09:19.493+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex] 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex] 2023-10-21T04:09:19.493+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:19.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex stopped 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.512+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.512+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.514+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex] 2023-10-21T04:09:19.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex closed 2023-10-21T04:09:19.515+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:19.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex ... 2023-10-21T04:09:19.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex ... 2023-10-21T04:09:19.519+05:30 [Info] switched currmeta from 84 -> 85 force true 2023-10-21T04:09:19.522+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46944:172.31.5.112:12000] 2023-10-21T04:09:19.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex sucessfully destroyed 2023-10-21T04:09:19.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex ... 2023-10-21T04:09:19.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 57 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex ... 2023-10-21T04:09:19.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index destroyed 2023-10-21T04:09:19.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.543+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.547+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.553+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46942:172.31.5.112:12000] 2023-10-21T04:09:19.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 62 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:19.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.563+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:19.563+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex 2023-10-21T04:09:19.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/docIndex started 2023-10-21T04:09:19.563+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.563+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 69 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:19.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.568+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:19.568+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex 2023-10-21T04:09:19.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index/mainIndex started 2023-10-21T04:09:19.568+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.568+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 2 Warmup took 95.751806ms 2023-10-21T04:09:19.568+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.568+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 2 WriterThreads 1 2023-10-21T04:09:19.568+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_2.index Index: 249283441109236971 Partition: 2 2023-10-21T04:09:19.568+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 249283441109236971 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.571+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 249283441109236971 PartitionId: 3 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.572+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.572+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 13003 recovery UsedSpace 8236 2023-10-21T04:09:19.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.573+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:19.573+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 14124 recovery UsedSpace 8236 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 58 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.588+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex started 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.588+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex started 2023-10-21T04:09:19.588+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:19.588+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex] 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.588+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex] 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex] 2023-10-21T04:09:19.588+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:19.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex stopped 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.588+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex] 2023-10-21T04:09:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex closed 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex] 2023-10-21T04:09:19.597+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex] 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex] 2023-10-21T04:09:19.597+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex stopped 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:19.602+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:19.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex] 2023-10-21T04:09:19.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex closed 2023-10-21T04:09:19.602+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:19.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex ... 2023-10-21T04:09:19.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex ... 2023-10-21T04:09:19.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex sucessfully destroyed 2023-10-21T04:09:19.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex ... 2023-10-21T04:09:19.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 58 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex ... 2023-10-21T04:09:19.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index destroyed 2023-10-21T04:09:19.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.615+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.615+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:19.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.624+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:19.624+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:19.624+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46946:172.31.5.112:12000] 2023-10-21T04:09:19.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 63 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:19.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.630+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:19.630+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex 2023-10-21T04:09:19.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/docIndex started 2023-10-21T04:09:19.631+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.631+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#249283441109236971:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 70 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:19.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.633+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:19.633+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex 2023-10-21T04:09:19.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index/mainIndex started 2023-10-21T04:09:19.633+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 249283441109236971 PartitionId 3 Warmup took 61.586792ms 2023-10-21T04:09:19.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#249283441109236971:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.634+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 249283441109236971 partitionId 3 WriterThreads 1 2023-10-21T04:09:19.634+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 249283441109236971 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_249283441109236971_3.index Index: 249283441109236971 Partition: 3 2023-10-21T04:09:19.634+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 249283441109236971, shardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:19.634+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 249283441109236971, partnShardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:19.634+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 249283441109236971 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:19.635+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 249283441109236971 2023-10-21T04:09:19.635+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 249283441109236971 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:19.635+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:19.635+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:19.635+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 249283441109236971 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:19.635+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 249283441109236971 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:19.635+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 249283441109236971 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:19.635+05:30 [Info] StorageMgr::openSnapshot IndexInst:249283441109236971 Partition:2 No Snapshot Found. 2023-10-21T04:09:19.635+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 249283441109236971 Adding Nil Snapshot. 2023-10-21T04:09:19.643+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46948:172.31.5.112:12000] 2023-10-21T04:09:19.666+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 30.716107ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:19.667+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 144.889µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:19.707+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.707+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.708+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46950:172.31.5.112:12000] 2023-10-21T04:09:19.711+05:30 [Info] switched currmeta from 85 -> 86 force true 2023-10-21T04:09:19.727+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46952:172.31.5.112:12000] 2023-10-21T04:09:19.765+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46954:172.31.5.112:12000] 2023-10-21T04:09:19.776+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46956:172.31.5.112:12000] 2023-10-21T04:09:19.805+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46958:172.31.5.112:12000] 2023-10-21T04:09:19.815+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46960:172.31.5.112:12000] 2023-10-21T04:09:19.828+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[249283441109236971:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:19.828+05:30 [Info] Indexer::handleMergePartition Source 1981123602637975601 Target 249283441109236971 2023-10-21T04:09:19.828+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 249283441109236971 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009e25bc0 2023-10-21T04:09:19.829+05:30 [Info] MergePartition: sent async request to update index instance 249283441109236971 rstate moved to ACTIVE 2023-10-21T04:09:19.848+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.822187ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:19.848+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 1981123602637975601, realInst: 249283441109236971 2023-10-21T04:09:19.849+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 15991242037348127852, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:19.849+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 15991242037348127852, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:19.880+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.880+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.883+05:30 [Info] switched currmeta from 86 -> 87 force true 2023-10-21T04:09:19.888+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 9664682240473767939,realInstId: 0, indexDefn DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:19.888+05:30 [Info] Indexer::handleRecoverIndex InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:19.889+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 563.468µs 2023-10-21T04:09:19.889+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 645.982µs 2023-10-21T04:09:19.889+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.889+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 4 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.890+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:09:19.890+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 130 metaSz: 2510 rp [1] sn: 4 count: 130 metaSz: 2575 2023-10-21T04:09:19.890+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 23933 recovery UsedSpace 21228 2023-10-21T04:09:19.893+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 9664682240473767939, indexDefn DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:19.893+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 44.230268ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:19.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.896+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:19.896+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 130 metaSz: 2510 rp [1] sn: 4 count: 130 metaSz: 2575 2023-10-21T04:09:19.896+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 25384 recovery UsedSpace 20910 2023-10-21T04:09:19.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex 2023-10-21T04:09:19.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.913+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex 2023-10-21T04:09:19.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/docIndex started 2023-10-21T04:09:19.913+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.913+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex 2023-10-21T04:09:19.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.916+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex 2023-10-21T04:09:19.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index/mainIndex started 2023-10-21T04:09:19.916+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:19.916+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.916+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.920+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:19.920+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:19.936+05:30 [Info] switched currmeta from 87 -> 88 force true 2023-10-21T04:09:19.962+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 4 Warmup took 72.089582ms 2023-10-21T04:09:19.962+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 4 WriterThreads 1 2023-10-21T04:09:19.962+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 4 Stopped Billing WUs 2023-10-21T04:09:19.962+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_4.index Index: 9664682240473767939 Partition: 4 2023-10-21T04:09:19.962+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.962+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 5 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.963+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:19.963+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 128 metaSz: 2510 rp [1] sn: 4 count: 128 metaSz: 2575 2023-10-21T04:09:19.963+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 24595 recovery UsedSpace 21137 2023-10-21T04:09:19.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.964+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:19.964+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 128 metaSz: 2510 rp [1] sn: 4 count: 128 metaSz: 2575 2023-10-21T04:09:19.964+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 25901 recovery UsedSpace 20910 2023-10-21T04:09:19.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex 2023-10-21T04:09:19.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.973+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex 2023-10-21T04:09:19.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/mainIndex started 2023-10-21T04:09:19.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex 2023-10-21T04:09:19.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.974+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex 2023-10-21T04:09:19.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index/docIndex started 2023-10-21T04:09:19.974+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:19.976+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.976+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.990+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 5 Warmup took 27.338794ms 2023-10-21T04:09:19.990+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 5 WriterThreads 1 2023-10-21T04:09:19.990+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 5 Stopped Billing WUs 2023-10-21T04:09:19.990+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_5.index Index: 9664682240473767939 Partition: 5 2023-10-21T04:09:19.990+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:19.991+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 6 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:19.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:19.991+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:09:19.991+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 106 metaSz: 2509 rp [1] sn: 4 count: 106 metaSz: 2574 2023-10-21T04:09:19.991+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 24001 recovery UsedSpace 20910 2023-10-21T04:09:19.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:19.992+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:19.992+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 106 metaSz: 2509 rp [1] sn: 4 count: 106 metaSz: 2574 2023-10-21T04:09:19.992+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 25657 recovery UsedSpace 20910 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:19.996+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/docIndex started 2023-10-21T04:09:19.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.996+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:19.996+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex 2023-10-21T04:09:19.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index/mainIndex started 2023-10-21T04:09:19.996+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:19.998+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:19.998+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.024+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 6 Warmup took 33.464505ms 2023-10-21T04:09:20.025+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 6 WriterThreads 1 2023-10-21T04:09:20.025+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 6 Stopped Billing WUs 2023-10-21T04:09:20.025+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_6.index Index: 9664682240473767939 Partition: 6 2023-10-21T04:09:20.025+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:20.025+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 7 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:20.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:20.025+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:20.025+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 132 metaSz: 2510 rp [1] sn: 4 count: 132 metaSz: 2575 2023-10-21T04:09:20.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 25693 recovery UsedSpace 21232 2023-10-21T04:09:20.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:20.026+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:20.026+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 132 metaSz: 2510 rp [1] sn: 4 count: 132 metaSz: 2575 2023-10-21T04:09:20.026+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 26907 recovery UsedSpace 20910 2023-10-21T04:09:20.030+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.030+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex 2023-10-21T04:09:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:20.034+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex 2023-10-21T04:09:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/mainIndex started 2023-10-21T04:09:20.034+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.034+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex 2023-10-21T04:09:20.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:20.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:20.035+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex 2023-10-21T04:09:20.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index/docIndex started 2023-10-21T04:09:20.035+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:20.035+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.035+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.038+05:30 [Info] switched currmeta from 88 -> 89 force true 2023-10-21T04:09:20.054+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 7 Warmup took 29.379285ms 2023-10-21T04:09:20.056+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 7 WriterThreads 1 2023-10-21T04:09:20.056+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 7 Stopped Billing WUs 2023-10-21T04:09:20.056+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_7.index Index: 9664682240473767939 Partition: 7 2023-10-21T04:09:20.057+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:20.057+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 8 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:20.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:20.058+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:20.058+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 123 metaSz: 2510 rp [1] sn: 4 count: 123 metaSz: 2575 2023-10-21T04:09:20.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:20.059+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 25346 recovery UsedSpace 20910 2023-10-21T04:09:20.059+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:20.059+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 123 metaSz: 2510 rp [1] sn: 4 count: 123 metaSz: 2575 2023-10-21T04:09:20.059+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 27148 recovery UsedSpace 20910 2023-10-21T04:09:20.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex 2023-10-21T04:09:20.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:20.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:20.069+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex 2023-10-21T04:09:20.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/docIndex started 2023-10-21T04:09:20.069+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.069+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex 2023-10-21T04:09:20.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:20.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:20.076+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex 2023-10-21T04:09:20.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index/mainIndex started 2023-10-21T04:09:20.076+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:20.076+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.076+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.105+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2288923528993975684:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:20.105+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2288923528993975684 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00a697d40 2023-10-21T04:09:20.106+05:30 [Info] handleUpdateIndexRState: Index instance 2288923528993975684 rstate moved to ACTIVE 2023-10-21T04:09:20.107+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 8 Warmup took 49.129571ms 2023-10-21T04:09:20.107+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 8 WriterThreads 1 2023-10-21T04:09:20.107+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 8 Stopped Billing WUs 2023-10-21T04:09:20.107+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_8.index Index: 9664682240473767939 Partition: 8 2023-10-21T04:09:20.107+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:20.108+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 1 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:20.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:20.108+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:09:20.108+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 120 metaSz: 2510 rp [1] sn: 4 count: 120 metaSz: 2575 2023-10-21T04:09:20.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:20.108+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:20.109+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 120 metaSz: 2510 rp [1] sn: 4 count: 120 metaSz: 2575 2023-10-21T04:09:20.109+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 25698 recovery UsedSpace 20910 2023-10-21T04:09:20.109+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 27567 recovery UsedSpace 20910 2023-10-21T04:09:20.119+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.968101ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:20.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex 2023-10-21T04:09:20.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:20.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:20.121+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex 2023-10-21T04:09:20.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/mainIndex started 2023-10-21T04:09:20.121+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.121+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.123+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16013578430151540990, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:20.123+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16013578430151540990, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:20.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex 2023-10-21T04:09:20.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:20.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:20.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex 2023-10-21T04:09:20.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index/docIndex started 2023-10-21T04:09:20.130+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:20.130+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.130+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.149+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.149+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.153+05:30 [Info] switched currmeta from 89 -> 91 force true 2023-10-21T04:09:20.165+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.165+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.169+05:30 [Info] switched currmeta from 91 -> 91 force true 2023-10-21T04:09:20.177+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 1 Warmup took 69.210448ms 2023-10-21T04:09:20.177+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 1 WriterThreads 1 2023-10-21T04:09:20.177+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 1 Stopped Billing WUs 2023-10-21T04:09:20.177+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_1.index Index: 9664682240473767939 Partition: 1 2023-10-21T04:09:20.177+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:20.178+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 2 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:20.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:20.178+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:20.178+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 129 metaSz: 2510 rp [1] sn: 4 count: 129 metaSz: 2575 2023-10-21T04:09:20.179+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 27017 recovery UsedSpace 21232 2023-10-21T04:09:20.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:20.180+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:20.180+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 129 metaSz: 2510 rp [1] sn: 4 count: 129 metaSz: 2575 2023-10-21T04:09:20.181+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 27835 recovery UsedSpace 20910 2023-10-21T04:09:20.191+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 16726320877690809560,realInstId: 0, indexDefn DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:20.191+05:30 [Info] Indexer::handleRecoverIndex InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:20.191+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 555.005µs 2023-10-21T04:09:20.191+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 611.966µs 2023-10-21T04:09:20.192+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.194+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 16726320877690809560, indexDefn DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:20.194+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 71.068772ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:20.195+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 1 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 256 2023-10-21T04:09:20.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 134 metaSz: 2511 rp [1] sn: 4 count: 134 metaSz: 2576 rp [2] sn: 5 count: 134 metaSz: 2575 2023-10-21T04:09:20.195+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 24959 recovery UsedSpace 29584 2023-10-21T04:09:20.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.196+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:20.196+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 134 metaSz: 2576 rp [1] sn: 5 count: 134 metaSz: 2575 2023-10-21T04:09:20.196+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 26080 recovery UsedSpace 37480 2023-10-21T04:09:20.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex 2023-10-21T04:09:20.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:20.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:20.204+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex 2023-10-21T04:09:20.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/docIndex started 2023-10-21T04:09:20.204+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.205+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex 2023-10-21T04:09:20.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:20.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:20.206+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex 2023-10-21T04:09:20.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index/mainIndex started 2023-10-21T04:09:20.206+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:20.208+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.208+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.209+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.213+05:30 [Info] switched currmeta from 91 -> 92 force true 2023-10-21T04:09:20.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex 2023-10-21T04:09:20.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.216+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex 2023-10-21T04:09:20.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/mainIndex started 2023-10-21T04:09:20.216+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.216+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex 2023-10-21T04:09:20.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.230+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex 2023-10-21T04:09:20.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index/docIndex started 2023-10-21T04:09:20.230+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:20.230+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.230+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 2 Warmup took 56.04865ms 2023-10-21T04:09:20.234+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 2 WriterThreads 1 2023-10-21T04:09:20.234+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 2 Stopped Billing WUs 2023-10-21T04:09:20.234+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_2.index Index: 9664682240473767939 Partition: 2 2023-10-21T04:09:20.234+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9664682240473767939 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:20.235+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9664682240473767939 PartitionId: 3 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:20.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:20.235+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 238 2023-10-21T04:09:20.235+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 132 metaSz: 2510 rp [1] sn: 4 count: 132 metaSz: 2575 2023-10-21T04:09:20.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:20.235+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:20.235+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 132 metaSz: 2510 rp [1] sn: 4 count: 132 metaSz: 2575 2023-10-21T04:09:20.236+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 27529 recovery UsedSpace 21224 2023-10-21T04:09:20.236+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 28459 recovery UsedSpace 20910 2023-10-21T04:09:20.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex 2023-10-21T04:09:20.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:20.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:20.248+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex 2023-10-21T04:09:20.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/mainIndex started 2023-10-21T04:09:20.248+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.248+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex 2023-10-21T04:09:20.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:20.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:20.251+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex 2023-10-21T04:09:20.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index/docIndex started 2023-10-21T04:09:20.251+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:20.251+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.251+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.259+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 1 Warmup took 64.454982ms 2023-10-21T04:09:20.260+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 1 WriterThreads 1 2023-10-21T04:09:20.260+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_1.index Index: 16726320877690809560 Partition: 1 2023-10-21T04:09:20.260+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.263+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 2 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.264+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:09:20.264+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 120 metaSz: 2511 rp [1] sn: 4 count: 120 metaSz: 2576 rp [2] sn: 5 count: 120 metaSz: 2575 2023-10-21T04:09:20.264+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 24370 recovery UsedSpace 29266 2023-10-21T04:09:20.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.264+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:20.264+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 120 metaSz: 2576 rp [1] sn: 5 count: 120 metaSz: 2575 2023-10-21T04:09:20.265+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 25584 recovery UsedSpace 37480 2023-10-21T04:09:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex 2023-10-21T04:09:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.282+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex 2023-10-21T04:09:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/docIndex started 2023-10-21T04:09:20.282+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.282+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex 2023-10-21T04:09:20.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.283+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex 2023-10-21T04:09:20.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index/mainIndex started 2023-10-21T04:09:20.283+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:20.283+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.283+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.286+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9664682240473767939 PartitionId 3 Warmup took 50.802236ms 2023-10-21T04:09:20.286+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9664682240473767939 partitionId 3 WriterThreads 1 2023-10-21T04:09:20.286+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9664682240473767939 PartitionId 3 Stopped Billing WUs 2023-10-21T04:09:20.286+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9664682240473767939 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_9664682240473767939_3.index Index: 9664682240473767939 Partition: 3 2023-10-21T04:09:20.286+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 9664682240473767939, shardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:20.286+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 9664682240473767939, partnShardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:20.286+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:20.287+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 9664682240473767939 2023-10-21T04:09:20.287+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9664682240473767939 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:20.287+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:20.287+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:20.287+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9664682240473767939 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:20.288+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:20.288+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 9664682240473767939 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:09:20.291+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:1 Attempting to open snapshot (SnapshotInfo: count:120 committed:false) 2023-10-21T04:09:20.292+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:2 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:09:20.293+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:3 Attempting to open snapshot (SnapshotInfo: count:132 committed:false) 2023-10-21T04:09:20.294+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:4 Attempting to open snapshot (SnapshotInfo: count:130 committed:false) 2023-10-21T04:09:20.295+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:5 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:09:20.297+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:6 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:09:20.298+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:7 Attempting to open snapshot (SnapshotInfo: count:132 committed:false) 2023-10-21T04:09:20.299+05:30 [Info] StorageMgr::openSnapshot IndexInst:9664682240473767939 Partition:8 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:09:20.304+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.409229ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:20.305+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 176.616µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:20.311+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 2 Warmup took 47.318936ms 2023-10-21T04:09:20.313+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 2 WriterThreads 1 2023-10-21T04:09:20.314+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_2.index Index: 16726320877690809560 Partition: 2 2023-10-21T04:09:20.314+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.317+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 3 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.319+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:20.319+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 139 metaSz: 2576 rp [1] sn: 5 count: 139 metaSz: 2575 2023-10-21T04:09:20.320+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 26660 recovery UsedSpace 37480 2023-10-21T04:09:20.324+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.324+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:09:20.324+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 139 metaSz: 2511 rp [1] sn: 4 count: 139 metaSz: 2576 rp [2] sn: 5 count: 139 metaSz: 2575 2023-10-21T04:09:20.325+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 25903 recovery UsedSpace 29588 2023-10-21T04:09:20.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex 2023-10-21T04:09:20.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.329+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex 2023-10-21T04:09:20.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/mainIndex started 2023-10-21T04:09:20.330+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.330+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex 2023-10-21T04:09:20.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.339+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex 2023-10-21T04:09:20.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index/docIndex started 2023-10-21T04:09:20.339+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.339+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.339+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:20.341+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.341+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.346+05:30 [Info] switched currmeta from 92 -> 93 force true 2023-10-21T04:09:20.380+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 3 Warmup took 61.985239ms 2023-10-21T04:09:20.381+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 3 WriterThreads 1 2023-10-21T04:09:20.381+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_3.index Index: 16726320877690809560 Partition: 3 2023-10-21T04:09:20.381+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.385+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 4 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.385+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:20.385+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:20.385+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 112 metaSz: 2511 rp [1] sn: 4 count: 112 metaSz: 2576 rp [2] sn: 5 count: 112 metaSz: 2575 2023-10-21T04:09:20.385+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 112 metaSz: 2576 rp [1] sn: 5 count: 112 metaSz: 2575 2023-10-21T04:09:20.386+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 24939 recovery UsedSpace 29266 2023-10-21T04:09:20.386+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 26278 recovery UsedSpace 37480 2023-10-21T04:09:20.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex 2023-10-21T04:09:20.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex 2023-10-21T04:09:20.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.404+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex 2023-10-21T04:09:20.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/mainIndex started 2023-10-21T04:09:20.404+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.404+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.404+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex 2023-10-21T04:09:20.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index/docIndex started 2023-10-21T04:09:20.404+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.404+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:20.404+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.432+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 4 Warmup took 46.648837ms 2023-10-21T04:09:20.432+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 4 WriterThreads 1 2023-10-21T04:09:20.432+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_4.index Index: 16726320877690809560 Partition: 4 2023-10-21T04:09:20.432+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.432+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 5 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.433+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:20.433+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 133 metaSz: 2511 rp [1] sn: 4 count: 133 metaSz: 2576 rp [2] sn: 5 count: 133 metaSz: 2575 2023-10-21T04:09:20.433+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 26594 recovery UsedSpace 29588 2023-10-21T04:09:20.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.434+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:20.434+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 133 metaSz: 2576 rp [1] sn: 5 count: 133 metaSz: 2575 2023-10-21T04:09:20.434+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 27466 recovery UsedSpace 37480 2023-10-21T04:09:20.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex 2023-10-21T04:09:20.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.441+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex 2023-10-21T04:09:20.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/mainIndex started 2023-10-21T04:09:20.441+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.442+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex 2023-10-21T04:09:20.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.442+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex 2023-10-21T04:09:20.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index/docIndex started 2023-10-21T04:09:20.442+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:20.443+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.443+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.450+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9651234895113427 2023-10-21T04:09:20.470+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 5 Warmup took 37.287952ms 2023-10-21T04:09:20.470+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 5 WriterThreads 1 2023-10-21T04:09:20.470+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_5.index Index: 16726320877690809560 Partition: 5 2023-10-21T04:09:20.471+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.471+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 6 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.472+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:09:20.472+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 121 metaSz: 2511 rp [1] sn: 4 count: 121 metaSz: 2576 rp [2] sn: 5 count: 121 metaSz: 2575 2023-10-21T04:09:20.473+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 26157 recovery UsedSpace 29266 2023-10-21T04:09:20.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.473+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:20.474+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 121 metaSz: 2576 rp [1] sn: 5 count: 121 metaSz: 2575 2023-10-21T04:09:20.474+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 27622 recovery UsedSpace 37480 2023-10-21T04:09:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex 2023-10-21T04:09:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.498+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex 2023-10-21T04:09:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/docIndex started 2023-10-21T04:09:20.498+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.498+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex 2023-10-21T04:09:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.499+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex 2023-10-21T04:09:20.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index/mainIndex started 2023-10-21T04:09:20.499+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:20.499+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.499+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.538+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 6 Warmup took 66.270621ms 2023-10-21T04:09:20.539+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 6 WriterThreads 1 2023-10-21T04:09:20.539+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_6.index Index: 16726320877690809560 Partition: 6 2023-10-21T04:09:20.539+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.540+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 7 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.540+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:20.540+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 119 metaSz: 2511 rp [1] sn: 4 count: 119 metaSz: 2576 rp [2] sn: 5 count: 119 metaSz: 2575 2023-10-21T04:09:20.541+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 26580 recovery UsedSpace 29266 2023-10-21T04:09:20.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.541+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:20.541+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 119 metaSz: 2576 rp [1] sn: 5 count: 119 metaSz: 2575 2023-10-21T04:09:20.541+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 27498 recovery UsedSpace 37480 2023-10-21T04:09:20.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex 2023-10-21T04:09:20.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.553+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex 2023-10-21T04:09:20.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/docIndex started 2023-10-21T04:09:20.554+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.554+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.556+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.557+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex 2023-10-21T04:09:20.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.566+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex 2023-10-21T04:09:20.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index/mainIndex started 2023-10-21T04:09:20.566+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:20.567+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.567+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.594+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 7 Warmup took 53.462182ms 2023-10-21T04:09:20.594+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 7 WriterThreads 1 2023-10-21T04:09:20.594+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_7.index Index: 16726320877690809560 Partition: 7 2023-10-21T04:09:20.594+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16726320877690809560 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:20.594+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16726320877690809560 PartitionId: 8 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:20.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:20.595+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:20.595+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 122 metaSz: 2511 rp [1] sn: 4 count: 122 metaSz: 2576 rp [2] sn: 5 count: 122 metaSz: 2575 2023-10-21T04:09:20.595+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 27200 recovery UsedSpace 29266 2023-10-21T04:09:20.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:20.597+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:20.597+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 122 metaSz: 2576 rp [1] sn: 5 count: 122 metaSz: 2575 2023-10-21T04:09:20.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 28155 recovery UsedSpace 37480 2023-10-21T04:09:20.598+05:30 [Info] switched currmeta from 93 -> 94 force true 2023-10-21T04:09:20.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex 2023-10-21T04:09:20.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:20.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:20.606+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex 2023-10-21T04:09:20.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/docIndex started 2023-10-21T04:09:20.606+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.607+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.609+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.609+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex 2023-10-21T04:09:20.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:20.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:20.613+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex 2023-10-21T04:09:20.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index/mainIndex started 2023-10-21T04:09:20.613+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.613+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:20.613+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.616+05:30 [Info] switched currmeta from 94 -> 95 force true 2023-10-21T04:09:20.658+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16726320877690809560 PartitionId 8 Warmup took 64.033558ms 2023-10-21T04:09:20.659+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16726320877690809560 partitionId 8 WriterThreads 1 2023-10-21T04:09:20.659+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16726320877690809560 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16726320877690809560_8.index Index: 16726320877690809560 Partition: 8 2023-10-21T04:09:20.659+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 16726320877690809560, shardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:20.659+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 16726320877690809560, partnShardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:20.659+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:20.660+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 16726320877690809560 2023-10-21T04:09:20.660+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16726320877690809560 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:20.660+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:20.660+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:20.660+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16726320877690809560 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:20.661+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:20.661+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 16726320877690809560 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:09:20.662+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:5 Attempting to open snapshot (SnapshotInfo: count:133 committed:false) 2023-10-21T04:09:20.663+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:6 Attempting to open snapshot (SnapshotInfo: count:121 committed:false) 2023-10-21T04:09:20.664+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:7 Attempting to open snapshot (SnapshotInfo: count:119 committed:false) 2023-10-21T04:09:20.667+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:8 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:09:20.668+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:1 Attempting to open snapshot (SnapshotInfo: count:134 committed:false) 2023-10-21T04:09:20.669+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:2 Attempting to open snapshot (SnapshotInfo: count:120 committed:false) 2023-10-21T04:09:20.671+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:3 Attempting to open snapshot (SnapshotInfo: count:139 committed:false) 2023-10-21T04:09:20.672+05:30 [Info] StorageMgr::openSnapshot IndexInst:16726320877690809560 Partition:4 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:09:20.673+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.920146ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:20.674+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 108.439µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:20.700+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.700+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.711+05:30 [Info] switched currmeta from 95 -> 96 force true 2023-10-21T04:09:20.752+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.752+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.757+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.757+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.757+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.757+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Info] switched currmeta from 96 -> 97 force true 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.758+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:09:20.897+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2483169159760864439:INDEX_STATE_RECOVERED 9664682240473767939:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:20.898+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 1844709904168202817, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:20.898+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 1844709904168202817, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:20.915+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17037906325662037075,realInstId: 0, indexDefn DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:20.915+05:30 [Info] Indexer::handleRecoverIndex InstId: 17037906325662037075 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:20.915+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 135.841µs 2023-10-21T04:09:20.915+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 184.058µs 2023-10-21T04:09:20.915+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17037906325662037075 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:20.915+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17037906325662037075 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:20.917+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17037906325662037075, indexDefn DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:20.917+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 19.687782ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:20.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:20.924+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:20.927+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 16062 recovery UsedSpace 8236 2023-10-21T04:09:20.932+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:20.932+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:20.935+05:30 [Info] switched currmeta from 97 -> 98 force true 2023-10-21T04:09:20.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:20.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:20.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:20.943+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:20.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex started 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.944+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17037906325662037075 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:20.944+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17037906325662037075 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:20.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex] 2023-10-21T04:09:20.944+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex] 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex] 2023-10-21T04:09:20.944+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:20.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex stopped 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:20.944+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:20.954+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:20.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex] 2023-10-21T04:09:20.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex closed 2023-10-21T04:09:20.954+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:20.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex ... 2023-10-21T04:09:20.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex ... 2023-10-21T04:09:20.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:20.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index destroyed 2023-10-21T04:09:20.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:20.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:20.966+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:20.966+05:30 [Info] bucket_%2/#primary_defer/Mainstore#17037906325662037075:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:20.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 71 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:20.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:20.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:20.975+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:20.975+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex 2023-10-21T04:09:20.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index/mainIndex started 2023-10-21T04:09:20.975+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17037906325662037075 PartitionId 0 Warmup took 59.305017ms 2023-10-21T04:09:20.975+05:30 [Info] bucket_%2/#primary_defer/Mainstore#17037906325662037075:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:20.975+05:30 [Info] bucket_%2/#primary_defer/Mainstore#17037906325662037075:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:20.975+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17037906325662037075 partitionId 0 WriterThreads 4 2023-10-21T04:09:20.976+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17037906325662037075 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_17037906325662037075_0.index Index: 17037906325662037075 Partition: 0 2023-10-21T04:09:20.976+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17037906325662037075, shardIdMap: map[0:[6611724767191156837]] err: 2023-10-21T04:09:20.976+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17037906325662037075, partnShardIdMap: map[0:[6611724767191156837]], err: 2023-10-21T04:09:20.977+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17037906325662037075 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:20.977+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17037906325662037075 2023-10-21T04:09:20.977+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17037906325662037075 Name: #primary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:20.978+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:20.978+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:20.978+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17037906325662037075 Name: #primary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:20.978+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17037906325662037075 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:20.979+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17037906325662037075 Partitions [0] 2023-10-21T04:09:20.979+05:30 [Info] StorageMgr::openSnapshot IndexInst:17037906325662037075 Partition:0 No Snapshot Found. 2023-10-21T04:09:20.979+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17037906325662037075 Adding Nil Snapshot. 2023-10-21T04:09:20.991+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.634784ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:20.992+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 99.196µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:21.005+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.005+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.009+05:30 [Info] switched currmeta from 98 -> 99 force true 2023-10-21T04:09:21.153+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.153+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.157+05:30 [Info] switched currmeta from 99 -> 100 force true 2023-10-21T04:09:21.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.195+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.197+05:30 [Info] switched currmeta from 100 -> 101 force true 2023-10-21T04:09:21.199+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[16726320877690809560:INDEX_STATE_RECOVERED 18201836893981255880:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:21.200+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 17489537288579938718, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:21.200+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 17489537288579938718, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:21.229+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3985672602331234807,realInstId: 0, indexDefn DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:21.229+05:30 [Info] Indexer::handleRecoverIndex InstId: 3985672602331234807 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:21.229+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 138.837µs 2023-10-21T04:09:21.230+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 179.853µs 2023-10-21T04:09:21.230+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3985672602331234807 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:21.230+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3985672602331234807 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:21.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:21.230+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:21.231+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 16062 recovery UsedSpace 8236 2023-10-21T04:09:21.232+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3985672602331234807, indexDefn DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:21.236+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 36.477023ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:21.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:21.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:21.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:21.243+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:21.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex started 2023-10-21T04:09:21.244+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3985672602331234807 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:21.244+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3985672602331234807 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:21.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex] 2023-10-21T04:09:21.244+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:21.244+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex] 2023-10-21T04:09:21.244+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex] 2023-10-21T04:09:21.244+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:21.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex stopped 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:21.244+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:21.249+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.249+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.253+05:30 [Info] switched currmeta from 101 -> 102 force true 2023-10-21T04:09:21.255+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:21.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex] 2023-10-21T04:09:21.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex closed 2023-10-21T04:09:21.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:21.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex ... 2023-10-21T04:09:21.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex ... 2023-10-21T04:09:21.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:21.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index destroyed 2023-10-21T04:09:21.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:21.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:21.269+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:21.269+05:30 [Info] bucket_1/#primary_defer/Mainstore#3985672602331234807:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:21.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 71 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:21.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:21.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:21.277+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:21.277+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex 2023-10-21T04:09:21.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index/mainIndex started 2023-10-21T04:09:21.277+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3985672602331234807 PartitionId 0 Warmup took 46.928921ms 2023-10-21T04:09:21.277+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3985672602331234807 partitionId 0 WriterThreads 4 2023-10-21T04:09:21.277+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3985672602331234807 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:21.277+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3985672602331234807 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_3985672602331234807_0.index Index: 3985672602331234807 Partition: 0 2023-10-21T04:09:21.277+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3985672602331234807, shardIdMap: map[0:[10563635432419104314]] err: 2023-10-21T04:09:21.277+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3985672602331234807, partnShardIdMap: map[0:[10563635432419104314]], err: 2023-10-21T04:09:21.277+05:30 [Info] bucket_1/#primary_defer/Mainstore#3985672602331234807:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:21.277+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3985672602331234807 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:21.278+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3985672602331234807 2023-10-21T04:09:21.278+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3985672602331234807 Name: #primary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:21.278+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:21.278+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:21.278+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3985672602331234807 Name: #primary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:21.278+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3985672602331234807 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:21.278+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3985672602331234807 Partitions [0] 2023-10-21T04:09:21.278+05:30 [Info] StorageMgr::openSnapshot IndexInst:3985672602331234807 Partition:0 No Snapshot Found. 2023-10-21T04:09:21.278+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3985672602331234807 Adding Nil Snapshot. 2023-10-21T04:09:21.277+05:30 [Info] bucket_1/#primary_defer/Mainstore#3985672602331234807:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:21.291+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.914557ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:21.297+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 87.663µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:21.315+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.324+05:30 [Info] switched currmeta from 102 -> 103 force true 2023-10-21T04:09:21.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8371664075738414, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:09:21.631+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.631+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.639+05:30 [Info] switched currmeta from 103 -> 105 force true 2023-10-21T04:09:21.665+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.665+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.673+05:30 [Info] switched currmeta from 105 -> 105 force true 2023-10-21T04:09:21.709+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.710+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.714+05:30 [Info] switched currmeta from 105 -> 106 force true 2023-10-21T04:09:21.922+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[17037906325662037075:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:21.922+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17037906325662037075 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0051f2540 2023-10-21T04:09:21.922+05:30 [Info] handleUpdateIndexRState: Index instance 17037906325662037075 rstate moved to ACTIVE 2023-10-21T04:09:21.928+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.276517ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:21.929+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 7874378322029374196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:21.929+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 7874378322029374196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:21.943+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.943+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.947+05:30 [Info] switched currmeta from 106 -> 107 force true 2023-10-21T04:09:21.956+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17128008689111072444,realInstId: 0, indexDefn DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:21.957+05:30 [Info] Indexer::handleRecoverIndex InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:21.957+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 287.028µs 2023-10-21T04:09:21.957+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 419.872µs 2023-10-21T04:09:21.958+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17128008689111072444 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:21.958+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17128008689111072444, indexDefn DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:21.959+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.806046ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:21.964+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17128008689111072444 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:21.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:21.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:21.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:21.967+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1054724 memSz 770 2023-10-21T04:09:21.968+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 1000 metaSz: 2490 rp [1] sn: 4 count: 1000 metaSz: 2555 2023-10-21T04:09:21.970+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 45266 recovery UsedSpace 22293 2023-10-21T04:09:21.975+05:30 [Info] switched currmeta from 107 -> 108 force true 2023-10-21T04:09:21.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex 2023-10-21T04:09:21.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:21.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:21.990+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex 2023-10-21T04:09:21.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index/mainIndex started 2023-10-21T04:09:21.990+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17128008689111072444 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:21.991+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:21.991+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:22.006+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17128008689111072444 PartitionId 0 Warmup took 41.430792ms 2023-10-21T04:09:22.006+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17128008689111072444 partitionId 0 WriterThreads 4 2023-10-21T04:09:22.006+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17128008689111072444 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:22.006+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17128008689111072444 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17128008689111072444_0.index Index: 17128008689111072444 Partition: 0 2023-10-21T04:09:22.006+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17128008689111072444, shardIdMap: map[0:[6611724767191156837]] err: 2023-10-21T04:09:22.006+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17128008689111072444, partnShardIdMap: map[0:[6611724767191156837]], err: 2023-10-21T04:09:22.006+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:22.007+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17128008689111072444 2023-10-21T04:09:22.007+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17128008689111072444 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:22.007+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:22.007+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:22.007+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17128008689111072444 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:22.007+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:22.008+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17128008689111072444 Partitions [0] 2023-10-21T04:09:22.008+05:30 [Info] StorageMgr::openSnapshot IndexInst:17128008689111072444 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:22.020+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.918995ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:22.020+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 70.713µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:22.027+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.028+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.031+05:30 [Info] switched currmeta from 108 -> 109 force true 2023-10-21T04:09:22.178+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.178+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.182+05:30 [Info] switched currmeta from 109 -> 110 force true 2023-10-21T04:09:22.227+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.227+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.242+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3985672602331234807:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:22.243+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3985672602331234807 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0056b9f20 2023-10-21T04:09:22.243+05:30 [Info] handleUpdateIndexRState: Index instance 3985672602331234807 rstate moved to ACTIVE 2023-10-21T04:09:22.248+05:30 [Info] switched currmeta from 110 -> 111 force true 2023-10-21T04:09:22.251+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.712141ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:22.260+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.277+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 1571318351652255745, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:22.280+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 1571318351652255745, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:22.283+05:30 [Info] switched currmeta from 111 -> 112 force true 2023-10-21T04:09:22.296+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17148287981272169168,realInstId: 0, indexDefn DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:22.296+05:30 [Info] Indexer::handleRecoverIndex InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:22.296+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 184.055µs 2023-10-21T04:09:22.296+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 230.355µs 2023-10-21T04:09:22.297+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17148287981272169168 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:22.297+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17148287981272169168 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:22.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:22.297+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1054716 memSz 788 2023-10-21T04:09:22.297+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 5 count: 1000 metaSz: 2557 rp [1] sn: 6 count: 1000 metaSz: 2556 2023-10-21T04:09:22.298+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 45523 recovery UsedSpace 38871 2023-10-21T04:09:22.302+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17148287981272169168, indexDefn DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:22.302+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 24.828142ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:22.312+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex 2023-10-21T04:09:22.312+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:22.312+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:22.312+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex 2023-10-21T04:09:22.312+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index/mainIndex started 2023-10-21T04:09:22.312+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17148287981272169168 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:22.313+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:22.313+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:22.314+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.314+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.324+05:30 [Info] switched currmeta from 112 -> 113 force true 2023-10-21T04:09:22.342+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17148287981272169168 PartitionId 0 Warmup took 44.899273ms 2023-10-21T04:09:22.342+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17148287981272169168 partitionId 0 WriterThreads 4 2023-10-21T04:09:22.342+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17148287981272169168 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:22.342+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17148287981272169168 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_17148287981272169168_0.index Index: 17148287981272169168 Partition: 0 2023-10-21T04:09:22.342+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17148287981272169168, shardIdMap: map[0:[10563635432419104314]] err: 2023-10-21T04:09:22.342+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17148287981272169168, partnShardIdMap: map[0:[10563635432419104314]], err: 2023-10-21T04:09:22.342+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:22.343+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17148287981272169168 2023-10-21T04:09:22.343+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17148287981272169168 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:22.343+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:22.343+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:22.343+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17148287981272169168 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:22.343+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:22.343+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17148287981272169168 Partitions [0] 2023-10-21T04:09:22.344+05:30 [Info] StorageMgr::openSnapshot IndexInst:17148287981272169168 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:22.364+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.829682ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:22.364+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 107.643µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:22.401+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.402+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.406+05:30 [Info] switched currmeta from 113 -> 114 force true 2023-10-21T04:09:22.457+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7098063805273219, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:09:22.712+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.712+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.717+05:30 [Info] switched currmeta from 114 -> 116 force true 2023-10-21T04:09:22.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.761+05:30 [Info] switched currmeta from 116 -> 117 force true 2023-10-21T04:09:22.769+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:22.769+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:22.772+05:30 [Info] switched currmeta from 117 -> 117 force true 2023-10-21T04:09:22.963+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2483169159760864439:INDEX_STATE_RECOVERED 9664682240473767939:INDEX_STATE_RECOVERED 17128008689111072444:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:22.964+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [14741949230702299196 15991242037348127852 7874378322029374196] belonging to collectionId: d, ttid: ShardToken80:53:e0:a8:fa:c4:c4:85. Initiating build 2023-10-21T04:09:23.007+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.007+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.008+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [2483169159760864439 9664682240473767939 17128008689111072444] &{2} 2023-10-21T04:09:23.008+05:30 [Info] Indexer::handleBuildRecoveredIndexes [2483169159760864439 9664682240473767939 17128008689111072444] 2023-10-21T04:09:23.008+05:30 [Info] bulkUpdateRState: Index instance 2483169159760864439 rstate moved to PENDING 2023-10-21T04:09:23.009+05:30 [Info] bulkUpdateRState: Index instance 9664682240473767939 rstate moved to PENDING 2023-10-21T04:09:23.009+05:30 [Info] bulkUpdateRState: Index instance 17128008689111072444 rstate moved to PENDING 2023-10-21T04:09:23.009+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [2483169159760864439 9664682240473767939 17128008689111072444] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:23.009+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2483169159760864439 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 9664682240473767939 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17128008689111072444 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:23.009+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:23.009+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2483169159760864439 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 9664682240473767939 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17128008689111072444 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:23.033+05:30 [Info] switched currmeta from 117 -> 119 force true 2023-10-21T04:09:23.041+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:09:23.044+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.045+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.049+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:23.049+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:09:23.051+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:09:23.051+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:23.052+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:23.052+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:09:23.052+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c2% reset bucket state to NIL 2023-10-21T04:09:23.052+05:30 [Info] switched currmeta from 119 -> 120 force true 2023-10-21T04:09:23.052+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:09:23.052+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% SessionId 16730 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:09:23.053+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c2% Added for Stream INIT_STREAM 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c2%:1]] 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 9664682240473767939 stream INIT_STREAM keyspaceId bucket_%2:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c2%:2]] 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17128008689111072444 stream INIT_STREAM keyspaceId bucket_%2:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c2%:3]] 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 2483169159760864439 stream INIT_STREAM keyspaceId bucket_%2:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:23.053+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:09:23.053+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 16730 CollectionId: d CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:09:23.054+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:09:23.054+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:09:23.055+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:23.056+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 47.232128ms 2023-10-21T04:09:23.056+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 47.274931ms 2023-10-21T04:09:23.056+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [2483169159760864439 9664682240473767939 17128008689111072444] 2023-10-21T04:09:23.056+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 90.675529ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.057+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[2483169159760864439:true 9664682240473767939:true 17128008689111072444:true] 2023-10-21T04:09:23.062+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c2% Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:09:23.063+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c2% Check pending stream merge. 2023-10-21T04:09:23.067+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.656243ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.087+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.73697ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.105+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.105+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.107+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.605779ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.125+05:30 [Info] switched currmeta from 120 -> 123 force true 2023-10-21T04:09:23.133+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:58480" doAuth() ... 2023-10-21T04:09:23.133+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:58480" auth successful 2023-10-21T04:09:23.133+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:58480" +1 2023-10-21T04:09:23.148+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.148+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.151+05:30 [Info] switched currmeta from 123 -> 123 force true 2023-10-21T04:09:23.200+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.200+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.209+05:30 [Info] switched currmeta from 123 -> 124 force true 2023-10-21T04:09:23.240+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.241+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.249+05:30 [Info] switched currmeta from 124 -> 127 force true 2023-10-21T04:09:23.281+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.283+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.323+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[16726320877690809560:INDEX_STATE_RECOVERED 17148287981272169168:INDEX_STATE_RECOVERED 18201836893981255880:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:23.323+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [14889401051926000176 16013578430151540990 1571318351652255745] belonging to collectionId: c, ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4. Initiating build 2023-10-21T04:09:23.333+05:30 [Info] switched currmeta from 127 -> 127 force true 2023-10-21T04:09:23.403+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.404+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.438+05:30 [Info] switched currmeta from 127 -> 131 force true 2023-10-21T04:09:23.475+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [18201836893981255880 16726320877690809560 17148287981272169168] &{2} 2023-10-21T04:09:23.475+05:30 [Info] Indexer::handleBuildRecoveredIndexes [18201836893981255880 16726320877690809560 17148287981272169168] 2023-10-21T04:09:23.478+05:30 [Info] bulkUpdateRState: Index instance 18201836893981255880 rstate moved to PENDING 2023-10-21T04:09:23.479+05:30 [Info] bulkUpdateRState: Index instance 16726320877690809560 rstate moved to PENDING 2023-10-21T04:09:23.481+05:30 [Info] bulkUpdateRState: Index instance 17148287981272169168 rstate moved to PENDING 2023-10-21T04:09:23.483+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [18201836893981255880 16726320877690809560 17148287981272169168] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:23.483+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.486+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16726320877690809560 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17148287981272169168 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:23.489+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:23.490+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16726320877690809560 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17148287981272169168 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:23.496+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:23.498+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.529+05:30 [Info] switched currmeta from 131 -> 133 force true 2023-10-21T04:09:23.555+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.556+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.562+05:30 [Info] switched currmeta from 133 -> 133 force true 2023-10-21T04:09:23.569+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.569+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.572+05:30 [Info] switched currmeta from 133 -> 133 force true 2023-10-21T04:09:23.575+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:09:23.576+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:23.576+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:23.576+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:23.576+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:09:23.576+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:23.576+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:23.576+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:23.576+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c2%:{0xc0084946c0 }] 2023-10-21T04:09:23.577+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:23.577+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:23.577+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:09:23.577+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:09:23.577+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId 57460 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:23.578+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:23.578+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:23.578+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:09:23.578+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c2%:3 bucket_1:_default:c1:1]] 2023-10-21T04:09:23.578+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17148287981272169168 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:23.578+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c2%:3 bucket_1:_default:c1:2]] 2023-10-21T04:09:23.578+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 18201836893981255880 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:23.579+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c2%:3 bucket_1:_default:c1:3]] 2023-10-21T04:09:23.579+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 16726320877690809560 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:23.579+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:09:23.579+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 57460 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:09:23.581+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c2%:{0xc0084946c0 } bucket_1:_default:c1:{0xc006085380 }] 2023-10-21T04:09:23.582+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:23.584+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [18201836893981255880 16726320877690809560 17148287981272169168] 2023-10-21T04:09:23.584+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 259.495076ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.585+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 109.845418ms 2023-10-21T04:09:23.585+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:09:23.586+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[16726320877690809560:true 17148287981272169168:true 18201836893981255880:true] 2023-10-21T04:09:23.589+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:09:23.591+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 109.18458ms 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 16726320877690809560 RealInstId: 16726320877690809560 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 17148287981272169168 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9664682240473767939 RealInstId: 9664682240473767939 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 17128008689111072444 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 15710672812788661570 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:23.599+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:23.600+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:23.600+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:23.600+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:23.600+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:09:23.617+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 31.643603ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.675+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.676+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.685+05:30 [Info] switched currmeta from 133 -> 134 force true 2023-10-21T04:09:23.690+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 71.735407ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.725+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 35.113672ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:23.810+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.810+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.824+05:30 [Info] switched currmeta from 134 -> 139 force true 2023-10-21T04:09:23.844+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.855+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.859+05:30 [Info] switched currmeta from 139 -> 139 force true 2023-10-21T04:09:23.936+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.936+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.940+05:30 [Info] switched currmeta from 139 -> 142 force true 2023-10-21T04:09:23.945+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:23.946+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:23.949+05:30 [Info] switched currmeta from 142 -> 142 force true 2023-10-21T04:09:24.042+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 0 263268055970459 52 16730 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:24.044+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 2 35403626585217 50 16730 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:24.044+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 1 176639880977228 58 16730 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:24.046+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 3 215749515705641 45 16730 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:24.046+05:30 [Info] NewTsVbuuidCached: bucket_1, 64 TsVbuuidPoolMap is updated from: map[] to: map[64:0xc00c155410] 2023-10-21T04:09:24.052+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 6 261493429196201 60 16730 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:24.052+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 5 3874519919278 54 16730 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:24.053+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 7 170288557775498 52 16730 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:24.053+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 4 159113185399719 58 16730 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:24.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 10 206533515052028 50 16730 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:24.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 9 236398693961229 62 16730 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:24.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 8 264975169139761 56 16730 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:24.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 11 234758907181738 56 16730 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:24.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 12 116104661063478 55 16730 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:24.064+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 13 102350060311343 53 16730 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:24.067+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 14 140236896721046 60 16730 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:24.070+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 15 162158720181435 67 16730 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:24.073+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 17 98820809009123 44 16730 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:09:24.075+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 18 207134287529256 46 16730 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:24.076+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 16 200752780087280 51 16730 172.31.5.112:9000. HWT [49-51,51]. 2023-10-21T04:09:24.077+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 19 208836052671833 45 16730 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:24.080+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 20 92956357968546 50 16730 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:24.083+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 22 192215088742243 55 16730 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:24.083+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 21 27979933591532 54 16730 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:24.086+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 24 271667630317209 52 16730 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:24.086+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 23 154934968850942 51 16730 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:24.087+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 25 168740244208559 59 16730 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:24.098+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 32 67362620867705 67 16730 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:24.098+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 26 120989102157727 59 16730 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:24.099+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 27 5715487545729 54 16730 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:24.099+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 30 60545336618717 46 16730 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:24.099+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 29 130231725986856 49 16730 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:24.100+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 31 73432771436458 60 16730 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:24.100+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 28 236953356346174 54 16730 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:24.100+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 34 51776324008509 60 16730 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:24.101+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 35 24368176313355 50 16730 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:24.104+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 33 278426329668882 54 16730 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:24.107+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 36 43594649874772 52 16730 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:24.107+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 37 83647976044922 59 16730 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:24.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 38 181267443431019 53 16730 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:24.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 39 100659545510465 49 16730 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:09:24.109+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 40 153478150002150 68 16730 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:09:24.111+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 41 116785907755771 55 16730 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:24.111+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 42 45479338375128 55 16730 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:24.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 43 249919763471169 64 16730 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:24.117+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 44 166675095997741 57 16730 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:24.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 46 124194574848641 54 16730 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:24.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 45 49796766757346 58 16730 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:24.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 48 19274425379955 55 16730 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:24.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 47 149022976193937 55 16730 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:24.128+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 49 184487117398647 55 16730 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:24.132+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 51 196777743741606 51 16730 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:24.132+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 50 199728167534433 66 16730 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:09:24.137+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 52 158076890855634 54 16730 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:24.153+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 53 138935694816848 61 16730 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:24.155+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 54 24856128127173 60 16730 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:24.159+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 55 61798873988101 67 16730 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:24.162+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 56 29071415952852 61 16730 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:24.170+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 57 191719959781498 63 16730 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:24.173+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 58 103336286678362 46 16730 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:24.173+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 59 140942236111856 63 16730 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:24.175+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 60 56626109228904 46 16730 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:24.177+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 61 119871837460216 39 16730 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:09:24.180+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 62 177105814270799 58 16730 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:24.182+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 63 252537041863196 73 16730 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:09:24.185+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c2% InstanceIds [9664682240473767939 17128008689111072444 2483169159760864439] 2023-10-21T04:09:24.185+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% Cid d SessionId 16730 2023-10-21T04:09:24.186+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 16730 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 9664682240473767939 INDEX_STATE_CATCHUP 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 17128008689111072444 INDEX_STATE_CATCHUP 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 2483169159760864439 INDEX_STATE_CATCHUP 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 2483169159760864439 true 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 9664682240473767939 true 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 17128008689111072444 true 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 9664682240473767939 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% Session: 16730 BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c2%, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:09:24.186+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:09:24.186+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 16730 STREAM_ACTIVE 2023-10-21T04:09:24.187+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c2% Stream: INIT_STREAM SessionId: 16730 2023-10-21T04:09:24.187+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2:_default:c2% 16730 2023-10-21T04:09:24.187+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9664682240473767939 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17128008689111072444 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2483169159760864439 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:24.187+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:24.188+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9664682240473767939 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17128008689111072444 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2483169159760864439 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:24.189+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:09:24.188+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:24.189+05:30 [Info] send progress stats to clients 2023-10-21T04:09:24.189+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 16730 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:24.189+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17128008689111072444 from stream INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:09:24.189+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 2483169159760864439 from stream INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:09:24.189+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 9664682240473767939 from stream INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:09:24.189+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:09:24.189+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:09:24.189+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:24.189+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 16730 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:24.190+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_default:c1:{0xc006085380 }] 2023-10-21T04:09:24.190+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:24.190+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:24.190+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:24.190+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:24.191+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c2%] 2023-10-21T04:09:24.192+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2:_default:c2% 2023-10-21T04:09:24.205+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.303716ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:24.233+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.18746ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:24.245+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:24.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:24.248+05:30 [Info] switched currmeta from 142 -> 144 force true 2023-10-21T04:09:24.267+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:24.267+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:24.266+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 29.332305ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:24.271+05:30 [Info] switched currmeta from 144 -> 145 force true 2023-10-21T04:09:24.284+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:24.284+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:24.292+05:30 [Info] switched currmeta from 145 -> 145 force true 2023-10-21T04:09:24.448+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 2] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:24.448+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:24.458+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:09:24.458+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:09:24.688+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:24.688+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:24.698+05:30 [Info] switched currmeta from 145 -> 148 force true 2023-10-21T04:09:24.710+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:24.712+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:24.734+05:30 [Info] switched currmeta from 148 -> 148 force true 2023-10-21T04:09:25.096+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 186517626268899 55 57460 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:25.098+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 178840478062605 54 57460 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:25.100+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 148075163102199 54 57460 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:25.103+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 28605411869539 58 57460 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:25.103+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 215374859564337 35 57460 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:09:25.106+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 39552632604610 52 57460 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:25.107+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 242327631330054 62 57460 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:25.111+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 191903564840858 63 57460 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:25.111+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 50961953886801 45 57460 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:25.114+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 225280996447309 53 57460 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:25.114+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 274410200699189 54 57460 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:25.118+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 4429827396609 42 57460 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:09:25.118+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 66881023717063 51 57460 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:25.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 183148377680315 46 57460 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:25.126+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 257031483891250 55 57460 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:25.126+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 226814108379280 72 57460 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:09:25.126+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 127185710837163 50 57460 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:25.128+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 55290066231400 51 57460 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:25.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 242578104468179 63 57460 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:25.133+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 210306088158507 48 57460 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:09:25.134+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 230239359684242 50 57460 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:25.136+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 209771427122473 47 57460 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:25.139+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 216462437971699 57 57460 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:25.140+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 260036488774725 51 57460 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:25.142+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 160961818008256 59 57460 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:25.147+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 219132834127088 52 57460 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:25.147+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 280874278672777 71 57460 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:09:25.150+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 184056281482149 49 57460 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:25.152+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 102578907139247 58 57460 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:25.154+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 45957426777107 57 57460 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:25.157+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 99433130332778 58 57460 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:25.160+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 217381278987664 67 57460 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:25.163+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 82469662608686 60 57460 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:25.163+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 136958029391999 65 57460 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:09:25.165+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 28417669987133 58 57460 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:25.167+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 70213628416850 47 57460 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:25.170+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 253845164956592 58 57460 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:25.172+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 253494845579900 54 57460 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:25.172+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 188107826286720 60 57460 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:25.179+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 222555662236724 67 57460 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:25.179+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 5915864305121 60 57460 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:25.186+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 252931432434966 46 57460 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:25.186+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 259381390638966 58 57460 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:25.193+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:09:25.249+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 146267938201125 55 57460 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:25.249+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 158545118983293 67 57460 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:25.249+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 12301831552869 63 57460 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:25.253+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 242994859770444 54 57460 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:25.253+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 271074273345339 53 57460 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:25.257+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 227378777358533 57 57460 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:25.259+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 215467668600630 34 57460 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:09:25.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 207367350302991 64 57460 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:25.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 200436537417141 54 57460 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:25.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 188718367400378 56 57460 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:25.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 204867593700407 56 57460 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:25.275+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 279362078165491 55 57460 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:25.275+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 170056388266420 70 57460 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:09:25.276+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 264835102027079 52 57460 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:25.282+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 129411666769864 55 57460 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:25.283+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 16044440367455 53 57460 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:25.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 125009072873642 53 57460 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:25.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 75566139929862 56 57460 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:25.289+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 10527297481171 57 57460 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:25.289+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 106932309485550 53 57460 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:25.294+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 133748675726909 56 57460 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:25.322+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c1 InstanceIds [9664682240473767939 17128008689111072444 2483169159760864439 18201836893981255880 16726320877690809560 17148287981272169168] 2023-10-21T04:09:25.323+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 57460 2023-10-21T04:09:25.323+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57460 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 17148287981272169168 INDEX_STATE_CATCHUP 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 18201836893981255880 INDEX_STATE_CATCHUP 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 16726320877690809560 INDEX_STATE_CATCHUP 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 17148287981272169168 true 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 18201836893981255880 true 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 16726320877690809560 true 2023-10-21T04:09:25.323+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 18201836893981255880 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 57460 BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:09:25.324+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:09:25.324+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:09:25.324+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57460 STREAM_ACTIVE 2023-10-21T04:09:25.324+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 57460 2023-10-21T04:09:25.324+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1:_default:c1 57460 2023-10-21T04:09:25.325+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16726320877690809560 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17148287981272169168 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:25.325+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:25.325+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16726320877690809560 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17148287981272169168 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:25.325+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:25.328+05:30 [Info] send progress stats to clients 2023-10-21T04:09:25.328+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57460 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:25.328+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17148287981272169168 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:09:25.328+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 18201836893981255880 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:09:25.328+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 16726320877690809560 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:09:25.328+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:09:25.328+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:25.328+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:25.328+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57460 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:25.328+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:09:25.329+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:25.329+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:25.329+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c2% total received key-versions: map[12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:25.330+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c2% latest sequence numbers: map[12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:25.330+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:09:25.332+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:58480: use of closed network connection 2023-10-21T04:09:25.332+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:58480" exit: read tcp4 127.0.0.1:9121->127.0.0.1:58480: use of closed network connection 2023-10-21T04:09:25.332+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:58480" closed ! 2023-10-21T04:09:25.332+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:09:25.332+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:58480" already gone 2023-10-21T04:09:25.333+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:09:25.333+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 4 159113185399719 58 2023-10-21T04:09:25.334+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.334+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 1 176639880977228 58 2023-10-21T04:09:25.334+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.334+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 0 263268055970459 52 2023-10-21T04:09:25.334+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 2 35403626585217 50 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 5 3874519919278 54 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 6 261493429196201 60 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 7 170288557775498 52 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 3 215749515705641 45 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1:_default:c1 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 11 234758907181738 56 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 8 264975169139761 56 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 9 236398693961229 62 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.335+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 10 206533515052028 50 2023-10-21T04:09:25.335+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:25.338+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.979381ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.362+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.362+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.366+05:30 [Info] switched currmeta from 148 -> 149 force true 2023-10-21T04:09:25.377+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 39.348267ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.392+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.09933ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.406+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.407+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.412+05:30 [Info] switched currmeta from 149 -> 151 force true 2023-10-21T04:09:25.432+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c1]. Err projector.topicMissing 2023-10-21T04:09:25.432+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:09:25.432+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating projector.topicMissing As Success 2023-10-21T04:09:25.432+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57460 2023-10-21T04:09:25.432+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1:_default:c1 2023-10-21T04:09:25.432+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:09:25.432+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:25.432+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:09:25.432+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:09:25.432+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:25.432+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:25.432+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:09:25.432+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:09:25.432+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:09:25.432+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c2%] 2023-10-21T04:09:25.433+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 16730 2023-10-21T04:09:25.432+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 62064 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:25.433+05:30 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2023-10-21T04:09:25.433+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:09:25.434+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:09:25.434+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:09:25.434+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:09:25.434+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:09:25.435+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 62064 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:09:25.437+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:09:25.438+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:09:25.443+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.443+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.446+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:09:25.452+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:09:25.449+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:09:25.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9822842797959064, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:09:25.451+05:30 [Info] switched currmeta from 151 -> 151 force true 2023-10-21T04:09:25.453+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:25.455+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:25.455+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:25.455+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:09:25.456+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:09:25.456+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:25.456+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:25.456+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc00c43dec0 }] 2023-10-21T04:09:25.457+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:25.457+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:25.457+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:09:25.457+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:09:25.458+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 9997 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:25.459+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:25.459+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:25.460+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:09:25.460+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1 bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:09:25.461+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:09:25.461+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:09:25.461+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:09:25.461+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 9997 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:09:25.465+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc00d15a180 } bucket_1:{0xc00c43dec0 }] 2023-10-21T04:09:25.469+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:09:25.476+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:54394" doAuth() ... 2023-10-21T04:09:25.476+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:54394" auth successful 2023-10-21T04:09:25.477+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:54394" +1 2023-10-21T04:09:25.661+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00aa680c0 2023-10-21T04:09:25.662+05:30 [Info] handleUpdateIndexRState: Index instance 18201836893981255880 rstate moved to ACTIVE 2023-10-21T04:09:25.688+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.453138ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.688+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17148287981272169168 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00a7901e0 2023-10-21T04:09:25.688+05:30 [Info] handleUpdateIndexRState: Index instance 17148287981272169168 rstate moved to ACTIVE 2023-10-21T04:09:25.688+05:30 [Info] Indexer::handleMergePartition Source 10225476268021138633 Target 16726320877690809560 2023-10-21T04:09:25.688+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16726320877690809560 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00d0e63c0 2023-10-21T04:09:25.688+05:30 [Info] MergePartition: sent async request to update index instance 16726320877690809560 rstate moved to ACTIVE 2023-10-21T04:09:25.696+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.696+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.701+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.142045ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.705+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 13258659765240018889. Retrying... 2023-10-21T04:09:25.705+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 7597064108789640455. Retrying... 2023-10-21T04:09:25.705+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[16726320877690809560:true 17148287981272169168:true 18201836893981255880:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:25.734+05:30 [Info] switched currmeta from 151 -> 154 force true 2023-10-21T04:09:25.750+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.750+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.754+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 53.831051ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.754+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 10225476268021138633, realInst: 16726320877690809560 2023-10-21T04:09:25.777+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12140458060295645238, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:25.777+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12140458060295645238, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:25.798+05:30 [Info] switched currmeta from 154 -> 156 force true 2023-10-21T04:09:25.823+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4004220452657849790,realInstId: 0, indexDefn DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:25.823+05:30 [Info] Indexer::handleRecoverIndex InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:25.824+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 765.881µs 2023-10-21T04:09:25.825+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 1.304135ms 2023-10-21T04:09:25.825+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4004220452657849790 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:25.842+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4004220452657849790 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:25.825+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4004220452657849790, indexDefn DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:25.844+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 67.669119ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:25.845+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1080 2023-10-21T04:09:25.845+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 1000 metaSz: 2516 rp [1] sn: 4 count: 1000 metaSz: 2581 2023-10-21T04:09:25.845+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 51312 recovery UsedSpace 22977 2023-10-21T04:09:25.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:25.846+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1056222 memSz 1862 2023-10-21T04:09:25.847+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 3 count: 1000 metaSz: 2516 rp [1] sn: 4 count: 1000 metaSz: 2581 2023-10-21T04:09:25.852+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.852+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex 2023-10-21T04:09:25.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:25.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:25.857+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex 2023-10-21T04:09:25.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/mainIndex started 2023-10-21T04:09:25.857+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:25.857+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:25.858+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 49264 recovery UsedSpace 24605 2023-10-21T04:09:25.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex 2023-10-21T04:09:25.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:25.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:25.871+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex 2023-10-21T04:09:25.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index/docIndex started 2023-10-21T04:09:25.872+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4004220452657849790 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:25.872+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:25.872+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:25.890+05:30 [Info] switched currmeta from 156 -> 158 force true 2023-10-21T04:09:25.903+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.903+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.915+05:30 [Info] switched currmeta from 158 -> 158 force true 2023-10-21T04:09:25.937+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4004220452657849790 PartitionId 0 Warmup took 93.650525ms 2023-10-21T04:09:25.938+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4004220452657849790 partitionId 0 WriterThreads 4 2023-10-21T04:09:25.938+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4004220452657849790 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:25.938+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4004220452657849790 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4004220452657849790_0.index Index: 4004220452657849790 Partition: 0 2023-10-21T04:09:25.938+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4004220452657849790, shardIdMap: map[0:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:25.938+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4004220452657849790, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:25.938+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:25.939+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4004220452657849790 2023-10-21T04:09:25.939+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:25.939+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:25.939+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:25.939+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:25.940+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:25.940+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4004220452657849790 Partitions [0] 2023-10-21T04:09:25.944+05:30 [Info] StorageMgr::openSnapshot IndexInst:4004220452657849790 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:25.951+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.126256ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:25.951+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 79.067µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:25.968+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.968+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:25.976+05:30 [Info] switched currmeta from 158 -> 159 force true 2023-10-21T04:09:25.999+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:25.999+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:26.023+05:30 [Info] switched currmeta from 159 -> 162 force true 2023-10-21T04:09:26.058+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:26.058+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:26.079+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2483169159760864439 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00b042de0 2023-10-21T04:09:26.080+05:30 [Info] handleUpdateIndexRState: Index instance 2483169159760864439 rstate moved to ACTIVE 2023-10-21T04:09:26.117+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 36.904696ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:26.117+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17128008689111072444 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00aec4ae0 2023-10-21T04:09:26.117+05:30 [Info] handleUpdateIndexRState: Index instance 17128008689111072444 rstate moved to ACTIVE 2023-10-21T04:09:26.117+05:30 [Info] Indexer::handleMergePartition Source 10428614901675085295 Target 9664682240473767939 2023-10-21T04:09:26.117+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9664682240473767939 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00dbf7da0 2023-10-21T04:09:26.118+05:30 [Info] MergePartition: sent async request to update index instance 9664682240473767939 rstate moved to ACTIVE 2023-10-21T04:09:26.139+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.841491ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:26.139+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[2483169159760864439:true 9664682240473767939:true 17128008689111072444:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:26.151+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.355086ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:26.151+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 10428614901675085295, realInst: 9664682240473767939 2023-10-21T04:09:26.168+05:30 [Info] switched currmeta from 162 -> 166 force true 2023-10-21T04:09:26.168+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18201836893981255880:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:26.169+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:26.169+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:26.169+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16726320877690809560:[3 4 5 6 7 8 1 2] 17148287981272169168:[0] 18201836893981255880:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 {Index inst : [partitions]}: map[2483169159760864439:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:26.175+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12027054384338714804, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:26.175+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12027054384338714804, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:26.190+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:09:26.212+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:26.212+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:26.234+05:30 [Info] switched currmeta from 166 -> 167 force true 2023-10-21T04:09:26.240+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11400459767051699632,realInstId: 0, indexDefn DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:26.241+05:30 [Info] Indexer::handleRecoverIndex InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:26.241+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 479.596µs 2023-10-21T04:09:26.241+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 521.405µs 2023-10-21T04:09:26.241+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11400459767051699632 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:26.241+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11400459767051699632, indexDefn DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:26.241+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11400459767051699632 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:26.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:26.242+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1056218 memSz 2076 2023-10-21T04:09:26.242+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 25 count: 1000 metaSz: 2515 rp [1] sn: 26 count: 1000 metaSz: 2581 rp [2] sn: 27 count: 1000 metaSz: 2579 2023-10-21T04:09:26.242+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 50242 recovery UsedSpace 33166 2023-10-21T04:09:26.241+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 66.826219ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:26.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:26.243+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1055958 memSz 884 2023-10-21T04:09:26.243+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 26 count: 1000 metaSz: 2581 rp [1] sn: 27 count: 1000 metaSz: 2579 2023-10-21T04:09:26.244+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 53014 recovery UsedSpace 39145 2023-10-21T04:09:26.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex 2023-10-21T04:09:26.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:26.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:26.249+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex 2023-10-21T04:09:26.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/mainIndex started 2023-10-21T04:09:26.249+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:26.249+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:26.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex 2023-10-21T04:09:26.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:26.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:26.257+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex 2023-10-21T04:09:26.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index/docIndex started 2023-10-21T04:09:26.257+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11400459767051699632 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:26.258+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:26.258+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:26.282+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11400459767051699632 PartitionId 0 Warmup took 40.368308ms 2023-10-21T04:09:26.283+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11400459767051699632 partitionId 0 WriterThreads 4 2023-10-21T04:09:26.283+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11400459767051699632 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:26.283+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11400459767051699632 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11400459767051699632_0.index Index: 11400459767051699632 Partition: 0 2023-10-21T04:09:26.283+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11400459767051699632, shardIdMap: map[0:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:26.283+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11400459767051699632, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:26.284+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:26.285+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11400459767051699632 2023-10-21T04:09:26.286+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:26.286+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:26.286+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:26.296+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.932292ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:26.303+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:26.303+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:26.304+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11400459767051699632 Partitions [0] 2023-10-21T04:09:26.306+05:30 [Info] StorageMgr::openSnapshot IndexInst:11400459767051699632 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:26.307+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 165µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:26.316+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:26.316+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:26.328+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:09:26.330+05:30 [Info] switched currmeta from 167 -> 169 force true 2023-10-21T04:09:26.372+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:26.372+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:26.387+05:30 [Info] switched currmeta from 169 -> 169 force true 2023-10-21T04:09:26.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9839379458591322, throttleDelayMs (new, old, change): (6, 3, 3) 2023-10-21T04:09:26.512+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 186517626268899 55 62064 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:26.514+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 178840478062605 54 62064 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:26.520+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 215374859564337 35 62064 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:09:26.523+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 28605411869539 58 62064 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:26.527+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 148075163102199 54 62064 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:26.527+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 39552632604610 52 62064 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:26.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 191903564840858 63 62064 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:26.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 242327631330054 62 62064 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:26.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 225280996447309 53 62064 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:26.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 50961953886801 45 62064 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:26.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 183148377680315 46 62064 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:26.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 66881023717063 51 62064 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:26.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 4429827396609 42 62064 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:09:26.541+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 127185710837163 50 62064 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:26.545+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 274410200699189 54 62064 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:26.545+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 226814108379280 72 62064 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:09:26.548+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 55290066231400 51 62064 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:26.548+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 257031483891250 55 62064 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:26.553+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 210306088158507 48 62064 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:09:26.553+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 230239359684242 50 62064 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:26.555+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 242578104468179 63 62064 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:26.556+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 209771427122473 47 62064 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:26.558+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 260036488774725 51 62064 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:26.559+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 216462437971699 57 62064 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:26.562+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 160961818008256 59 62064 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:26.563+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 280874278672777 71 62064 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:09:26.566+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 184056281482149 49 62064 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:26.567+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 219132834127088 52 62064 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:26.568+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 102578907139247 58 62064 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:26.571+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 99433130332778 58 62064 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:26.571+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 45957426777107 57 62064 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:26.573+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 217381278987664 67 62064 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:26.575+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 82469662608686 60 62064 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:26.577+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 136958029391999 65 62064 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:09:26.577+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 28417669987133 58 62064 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:26.579+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 70213628416850 47 62064 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:26.581+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 253845164956592 58 62064 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:26.583+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 253494845579900 54 62064 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:26.586+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 188107826286720 60 62064 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:26.591+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 5915864305121 60 62064 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:26.591+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 222555662236724 67 62064 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:26.592+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 259381390638966 58 62064 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:26.595+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 146267938201125 55 62064 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:26.596+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17128008689111072444:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:26.596+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17128008689111072444:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:26.596+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17128008689111072444:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:26.596+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17128008689111072444:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:26.596+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17128008689111072444:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:26.596+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9664682240473767939:[1 2 3 4 5 6 7 8] 17128008689111072444:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:26.596+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 12301831552869 63 62064 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:26.596+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 158545118983293 67 62064 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:26.600+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 271074273345339 53 62064 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:26.604+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 227378777358533 57 62064 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:26.606+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 215467668600630 34 62064 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:09:26.611+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 252931432434966 46 62064 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:26.611+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 242994859770444 54 62064 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:26.611+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 200436537417141 54 62064 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:26.611+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 188718367400378 56 62064 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:26.611+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 207367350302991 64 62064 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:26.614+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 204867593700407 56 62064 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:26.617+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 279362078165491 55 62064 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:26.617+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 170056388266420 70 62064 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:09:26.620+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 129411666769864 55 62064 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:26.620+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 264835102027079 52 62064 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:26.630+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 106932309485550 53 62064 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:26.630+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 16044440367455 53 62064 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:26.630+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 125009072873642 53 62064 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:26.630+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 75566139929862 56 62064 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:26.631+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 133748675726909 56 62064 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:26.631+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 10527297481171 57 62064 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:26.651+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1 InstanceIds [16726320877690809560 17148287981272169168 18201836893981255880] 2023-10-21T04:09:26.651+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 62064 2023-10-21T04:09:26.652+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 62064 2023-10-21T04:09:26.652+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:09:26.652+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:09:26.652+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:09:26.652+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:09:26.652+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 62064 2023-10-21T04:09:26.652+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 62064 STREAM_ACTIVE 2023-10-21T04:09:26.652+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:09:26.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3513301448177380059:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:26.722+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3513301448177380059:[0] 16983881020783103544:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:26.722+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3513301448177380059:[0] 6393061446487756912:[7 8 1 2 3 4 5 6] 16983881020783103544:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:26.855+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4004220452657849790:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:26.855+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 6985049432225870291, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:26.855+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 6985049432225870291, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:26.879+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11756349460704871322,realInstId: 0, indexDefn DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:26.879+05:30 [Info] Indexer::handleRecoverIndex InstId: 11756349460704871322 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:26.879+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 305.673µs 2023-10-21T04:09:26.879+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 352.215µs 2023-10-21T04:09:26.880+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11756349460704871322 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:26.880+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11756349460704871322 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:26.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:26.880+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:26.881+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 15456 recovery UsedSpace 8236 2023-10-21T04:09:26.882+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11756349460704871322, indexDefn DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:26.882+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 26.737422ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:26.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:26.906+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:26.906+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 16384 recovery UsedSpace 8236 2023-10-21T04:09:26.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:26.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:26.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:26.923+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:26.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex started 2023-10-21T04:09:26.924+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:26.924+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:26.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:26.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:26.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:26.933+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:26.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex started 2023-10-21T04:09:26.933+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11756349460704871322 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:26.933+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11756349460704871322 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:26.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:26.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:26.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:26.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex] 2023-10-21T04:09:26.933+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:26.933+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex] 2023-10-21T04:09:26.933+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex] 2023-10-21T04:09:26.933+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:26.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex stopped 2023-10-21T04:09:26.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:26.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:26.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:26.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:26.935+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:26.935+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:26.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:26.944+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:26.954+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:26.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex] 2023-10-21T04:09:26.955+05:30 [Info] switched currmeta from 169 -> 170 force true 2023-10-21T04:09:26.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex closed 2023-10-21T04:09:26.958+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:26.958+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:26.958+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:26.958+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:26.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex] 2023-10-21T04:09:26.959+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:26.959+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex] 2023-10-21T04:09:26.959+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex] 2023-10-21T04:09:26.959+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:26.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex stopped 2023-10-21T04:09:26.959+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:26.959+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:26.959+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:26.960+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:26.967+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:26.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex] 2023-10-21T04:09:26.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex closed 2023-10-21T04:09:26.968+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:26.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex ... 2023-10-21T04:09:26.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex ... 2023-10-21T04:09:26.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:26.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex sucessfully destroyed 2023-10-21T04:09:26.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex ... 2023-10-21T04:09:26.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex ... 2023-10-21T04:09:26.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:26.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index destroyed 2023-10-21T04:09:26.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:26.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:26.989+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:26.989+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#11756349460704871322:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:26.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:26.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:26.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:26.994+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#11756349460704871322:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:27.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 64 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:27.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:27.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:27.002+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:27.002+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex 2023-10-21T04:09:27.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/docIndex started 2023-10-21T04:09:27.002+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#11756349460704871322:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.002+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#11756349460704871322:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 72 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:27.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:27.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:27.006+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:27.006+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex 2023-10-21T04:09:27.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index/mainIndex started 2023-10-21T04:09:27.006+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11756349460704871322 PartitionId 0 Warmup took 125.578048ms 2023-10-21T04:09:27.006+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11756349460704871322 partitionId 0 WriterThreads 4 2023-10-21T04:09:27.006+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11756349460704871322 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11756349460704871322_0.index Index: 11756349460704871322 Partition: 0 2023-10-21T04:09:27.006+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11756349460704871322, shardIdMap: map[0:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:27.006+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11756349460704871322, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:27.006+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11756349460704871322 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:27.007+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11756349460704871322 2023-10-21T04:09:27.007+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11756349460704871322 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:27.007+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:27.008+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:27.008+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11756349460704871322 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:27.009+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#11756349460704871322:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.009+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#11756349460704871322:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.010+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11756349460704871322 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:27.011+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11756349460704871322 Partitions [0] 2023-10-21T04:09:27.011+05:30 [Info] StorageMgr::openSnapshot IndexInst:11756349460704871322 Partition:0 No Snapshot Found. 2023-10-21T04:09:27.011+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11756349460704871322 Adding Nil Snapshot. 2023-10-21T04:09:27.023+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.116789ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:27.023+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 108.389µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:27.064+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.064+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.070+05:30 [Info] switched currmeta from 170 -> 171 force true 2023-10-21T04:09:27.184+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.184+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.190+05:30 [Info] switched currmeta from 171 -> 172 force true 2023-10-21T04:09:27.249+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11400459767051699632:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:27.251+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16162012140118277065, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:27.251+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16162012140118277065, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:27.272+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 80911002600697866,realInstId: 0, indexDefn DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:27.275+05:30 [Info] Indexer::handleRecoverIndex InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:27.276+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 793.294µs 2023-10-21T04:09:27.276+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 1.857449ms 2023-10-21T04:09:27.276+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.276+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 80911002600697866, indexDefn DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:27.276+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 25.414134ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:27.278+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 4 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.279+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:09:27.279+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 15 count: 140 metaSz: 2506 rp [1] sn: 16 count: 140 metaSz: 2571 rp [2] sn: 17 count: 140 metaSz: 2570 2023-10-21T04:09:27.279+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 29049 recovery UsedSpace 29584 2023-10-21T04:09:27.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.280+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:27.280+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 15 count: 140 metaSz: 2506 rp [1] sn: 16 count: 140 metaSz: 2571 rp [2] sn: 17 count: 140 metaSz: 2570 2023-10-21T04:09:27.280+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 30138 recovery UsedSpace 29266 2023-10-21T04:09:27.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex 2023-10-21T04:09:27.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.289+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex 2023-10-21T04:09:27.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/docIndex started 2023-10-21T04:09:27.289+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.289+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex 2023-10-21T04:09:27.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:27.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:27.294+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex 2023-10-21T04:09:27.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index/mainIndex started 2023-10-21T04:09:27.294+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:27.297+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.297+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.317+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 4 Warmup took 38.378431ms 2023-10-21T04:09:27.317+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 4 WriterThreads 1 2023-10-21T04:09:27.318+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_4.index Index: 80911002600697866 Partition: 4 2023-10-21T04:09:27.318+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.318+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 5 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.319+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:27.319+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 17 count: 116 metaSz: 2506 rp [1] sn: 18 count: 116 metaSz: 2571 rp [2] sn: 19 count: 116 metaSz: 2570 2023-10-21T04:09:27.319+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 27590 recovery UsedSpace 29266 2023-10-21T04:09:27.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.320+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:27.320+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 18 count: 116 metaSz: 2571 rp [1] sn: 19 count: 116 metaSz: 2570 2023-10-21T04:09:27.320+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 29842 recovery UsedSpace 37480 2023-10-21T04:09:27.329+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.329+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex 2023-10-21T04:09:27.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:27.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:27.331+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex 2023-10-21T04:09:27.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/mainIndex started 2023-10-21T04:09:27.332+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.332+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.334+05:30 [Info] switched currmeta from 172 -> 173 force true 2023-10-21T04:09:27.343+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex 2023-10-21T04:09:27.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.344+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex 2023-10-21T04:09:27.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index/docIndex started 2023-10-21T04:09:27.346+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.346+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:27.345+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.377+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 5 Warmup took 58.807139ms 2023-10-21T04:09:27.378+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 5 WriterThreads 1 2023-10-21T04:09:27.378+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_5.index Index: 80911002600697866 Partition: 5 2023-10-21T04:09:27.378+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.379+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 6 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.379+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:09:27.379+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 16 count: 127 metaSz: 2506 rp [1] sn: 17 count: 127 metaSz: 2571 rp [2] sn: 18 count: 127 metaSz: 2570 2023-10-21T04:09:27.380+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 28900 recovery UsedSpace 29490 2023-10-21T04:09:27.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.385+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:27.385+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 17 count: 127 metaSz: 2571 rp [1] sn: 18 count: 127 metaSz: 2570 2023-10-21T04:09:27.389+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 30897 recovery UsedSpace 37480 2023-10-21T04:09:27.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex 2023-10-21T04:09:27.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.403+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex 2023-10-21T04:09:27.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/docIndex started 2023-10-21T04:09:27.404+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.404+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex 2023-10-21T04:09:27.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:27.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:27.417+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex 2023-10-21T04:09:27.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index/mainIndex started 2023-10-21T04:09:27.417+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:27.417+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.418+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.445+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 6 Warmup took 65.873617ms 2023-10-21T04:09:27.445+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 6 WriterThreads 1 2023-10-21T04:09:27.445+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_6.index Index: 80911002600697866 Partition: 6 2023-10-21T04:09:27.445+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.445+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 7 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.446+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 256 2023-10-21T04:09:27.446+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 16 count: 125 metaSz: 2506 rp [1] sn: 17 count: 125 metaSz: 2571 rp [2] sn: 18 count: 125 metaSz: 2570 2023-10-21T04:09:27.446+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 29307 recovery UsedSpace 29396 2023-10-21T04:09:27.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:27.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 17 count: 125 metaSz: 2571 rp [1] sn: 18 count: 125 metaSz: 2570 2023-10-21T04:09:27.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 30879 recovery UsedSpace 37480 2023-10-21T04:09:27.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9876437544331625, throttleDelayMs (new, old, change): (10, 6, 4) 2023-10-21T04:09:27.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex 2023-10-21T04:09:27.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:27.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:27.463+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex 2023-10-21T04:09:27.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/mainIndex started 2023-10-21T04:09:27.463+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.463+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.465+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 263268055970459 52 9997 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:27.468+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 35403626585217 50 9997 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:27.468+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 176639880977228 58 9997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:27.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex 2023-10-21T04:09:27.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.469+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex 2023-10-21T04:09:27.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index/docIndex started 2023-10-21T04:09:27.469+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:27.469+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.469+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.471+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 215749515705641 45 9997 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:27.473+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 159113185399719 58 9997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:27.478+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 3874519919278 54 9997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:27.480+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 261493429196201 60 9997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:27.481+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 264975169139761 56 9997 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:27.481+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 170288557775498 52 9997 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:27.483+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 236398693961229 62 9997 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:27.490+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 206533515052028 50 9997 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:27.490+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 234758907181738 56 9997 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:27.504+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 7 Warmup took 59.090692ms 2023-10-21T04:09:27.515+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 7 WriterThreads 1 2023-10-21T04:09:27.515+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_7.index Index: 80911002600697866 Partition: 7 2023-10-21T04:09:27.515+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.515+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 8 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.516+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:27.516+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 15 count: 124 metaSz: 2506 rp [1] sn: 16 count: 124 metaSz: 2571 rp [2] sn: 17 count: 124 metaSz: 2570 2023-10-21T04:09:27.516+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 29552 recovery UsedSpace 29266 2023-10-21T04:09:27.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.517+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:27.519+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 116104661063478 55 9997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:27.519+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 102350060311343 53 9997 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:27.524+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 92956357968546 50 9997 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:27.524+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 162158720181435 67 9997 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:27.524+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 140236896721046 60 9997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:27.524+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 200752780087280 51 9997 172.31.5.112:9000. HWT [49-51,51]. 2023-10-21T04:09:27.524+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 98820809009123 44 9997 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:09:27.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 168740244208559 59 9997 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:27.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 154934968850942 51 9997 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:27.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 207134287529256 46 9997 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:27.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 208836052671833 45 9997 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:27.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 27979933591532 54 9997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:27.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 192215088742243 55 9997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:27.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 120989102157727 59 9997 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:27.533+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 271667630317209 52 9997 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:27.533+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 5715487545729 54 9997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:27.533+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 130231725986856 49 9997 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:27.517+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 16 count: 124 metaSz: 2571 rp [1] sn: 17 count: 124 metaSz: 2570 2023-10-21T04:09:27.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 31453 recovery UsedSpace 37480 2023-10-21T04:09:27.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex 2023-10-21T04:09:27.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.553+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex 2023-10-21T04:09:27.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/docIndex started 2023-10-21T04:09:27.553+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.553+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.554+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.554+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.565+05:30 [Info] switched currmeta from 173 -> 174 force true 2023-10-21T04:09:27.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex 2023-10-21T04:09:27.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:27.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:27.624+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex 2023-10-21T04:09:27.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index/mainIndex started 2023-10-21T04:09:27.622+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 100659545510465 49 9997 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:09:27.624+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 236953356346174 54 9997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:27.624+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 24368176313355 50 9997 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:27.624+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 51776324008509 60 9997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:27.624+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 60545336618717 46 9997 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:27.624+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 67362620867705 67 9997 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:27.628+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.630+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.630+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:27.630+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 181267443431019 53 9997 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 116785907755771 55 9997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 49796766757346 58 9997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 149022976193937 55 9997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 249919763471169 64 9997 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 184487117398647 55 9997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 196777743741606 51 9997 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 278426329668882 54 9997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:27.644+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 43594649874772 52 9997 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:27.644+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 73432771436458 60 9997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:27.644+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 83647976044922 59 9997 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:27.667+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.682+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:09:27.682+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:09:27.684+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 199728167534433 66 9997 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:09:27.684+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 45479338375128 55 9997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:27.685+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 124194574848641 54 9997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:27.688+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 153478150002150 68 9997 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:09:27.689+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 166675095997741 57 9997 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:27.690+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 19274425379955 55 9997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:27.692+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 158076890855634 54 9997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:27.693+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2 InstanceIds [18201836893981255880 16726320877690809560 17148287981272169168 2483169159760864439 9664682240473767939 17128008689111072444] 2023-10-21T04:09:27.693+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 9997 2023-10-21T04:09:27.693+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 9997 2023-10-21T04:09:27.693+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:09:27.693+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:09:27.693+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:09:27.694+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 9997 2023-10-21T04:09:27.694+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 9997 STREAM_ACTIVE 2023-10-21T04:09:27.706+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 56626109228904 46 9997 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:27.707+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 119871837460216 39 9997 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:09:27.707+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 138935694816848 61 9997 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:27.708+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 191719959781498 63 9997 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:27.708+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 140942236111856 63 9997 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:27.708+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 61798873988101 67 9997 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:27.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 252537041863196 73 9997 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:09:27.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 177105814270799 58 9997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:27.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 24856128127173 60 9997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:27.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 103336286678362 46 9997 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:27.710+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 29071415952852 61 9997 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:27.671+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.756+05:30 [Info] switched currmeta from 174 -> 177 force true 2023-10-21T04:09:27.785+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.788+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.788+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 8 Warmup took 272.591246ms 2023-10-21T04:09:27.797+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 8 WriterThreads 1 2023-10-21T04:09:27.799+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_8.index Index: 80911002600697866 Partition: 8 2023-10-21T04:09:27.800+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.811+05:30 [Info] switched currmeta from 177 -> 177 force true 2023-10-21T04:09:27.822+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 1 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.841+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 272 2023-10-21T04:09:27.841+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 16 count: 129 metaSz: 2506 rp [1] sn: 17 count: 129 metaSz: 2571 rp [2] sn: 18 count: 129 metaSz: 2570 2023-10-21T04:09:27.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.846+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:27.846+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 17 count: 129 metaSz: 2571 rp [1] sn: 18 count: 129 metaSz: 2570 2023-10-21T04:09:27.847+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 30871 recovery UsedSpace 29584 2023-10-21T04:09:27.847+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 31582 recovery UsedSpace 37480 2023-10-21T04:09:27.849+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.849+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex 2023-10-21T04:09:27.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.858+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex 2023-10-21T04:09:27.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/docIndex started 2023-10-21T04:09:27.858+05:30 [Info] switched currmeta from 177 -> 178 force true 2023-10-21T04:09:27.858+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.858+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex 2023-10-21T04:09:27.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:27.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:27.867+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex 2023-10-21T04:09:27.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index/mainIndex started 2023-10-21T04:09:27.867+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.867+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.868+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:27.903+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 1 Warmup took 62.970703ms 2023-10-21T04:09:27.905+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 1 WriterThreads 1 2023-10-21T04:09:27.905+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_1.index Index: 80911002600697866 Partition: 1 2023-10-21T04:09:27.905+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.906+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 2 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.906+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:27.907+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:27.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.907+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:27.907+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 16 count: 110 metaSz: 2571 rp [1] sn: 17 count: 110 metaSz: 2570 2023-10-21T04:09:27.908+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 31491 recovery UsedSpace 37480 2023-10-21T04:09:27.913+05:30 [Info] switched currmeta from 178 -> 179 force true 2023-10-21T04:09:27.915+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:27.915+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 15 count: 110 metaSz: 2506 rp [1] sn: 16 count: 110 metaSz: 2571 rp [2] sn: 17 count: 110 metaSz: 2570 2023-10-21T04:09:27.919+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 29585 recovery UsedSpace 29266 2023-10-21T04:09:27.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex 2023-10-21T04:09:27.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:27.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:27.921+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex 2023-10-21T04:09:27.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/mainIndex started 2023-10-21T04:09:27.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex 2023-10-21T04:09:27.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.936+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex 2023-10-21T04:09:27.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index/docIndex started 2023-10-21T04:09:27.936+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:27.937+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.937+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.938+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11756349460704871322:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:27.938+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11756349460704871322 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0097e2fc0 2023-10-21T04:09:27.942+05:30 [Info] handleUpdateIndexRState: Index instance 11756349460704871322 rstate moved to ACTIVE 2023-10-21T04:09:27.969+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 26.541591ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:27.974+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 14026476026078401783, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:27.976+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 14026476026078401783, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:27.978+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 2 Warmup took 71.764727ms 2023-10-21T04:09:27.978+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 2 WriterThreads 1 2023-10-21T04:09:27.978+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_2.index Index: 80911002600697866 Partition: 2 2023-10-21T04:09:27.978+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 80911002600697866 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:27.978+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 80911002600697866 PartitionId: 3 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:27.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:27.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:27.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 15 count: 129 metaSz: 2506 rp [1] sn: 16 count: 129 metaSz: 2571 rp [2] sn: 17 count: 129 metaSz: 2570 2023-10-21T04:09:27.979+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 31606 recovery UsedSpace 29588 2023-10-21T04:09:27.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:27.984+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:27.985+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 16 count: 129 metaSz: 2571 rp [1] sn: 17 count: 129 metaSz: 2570 2023-10-21T04:09:27.986+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 33002 recovery UsedSpace 37480 2023-10-21T04:09:27.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex 2023-10-21T04:09:27.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:27.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:27.994+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex 2023-10-21T04:09:27.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/docIndex started 2023-10-21T04:09:27.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:27.994+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:27.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex 2023-10-21T04:09:27.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:28.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:28.001+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex 2023-10-21T04:09:28.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index/mainIndex started 2023-10-21T04:09:28.001+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:28.002+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.002+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.007+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2170849387178940761,realInstId: 0, indexDefn DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:28.008+05:30 [Info] Indexer::handleRecoverIndex InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:28.009+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 1.354757ms 2023-10-21T04:09:28.009+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 1.561735ms 2023-10-21T04:09:28.009+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.010+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2170849387178940761, indexDefn DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:28.010+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 36.640807ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:28.010+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 8 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.011+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:09:28.011+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 128 metaSz: 2510 rp [1] sn: 2 count: 128 metaSz: 2575 2023-10-21T04:09:28.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.013+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:28.013+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 128 metaSz: 2510 rp [1] sn: 2 count: 128 metaSz: 2575 2023-10-21T04:09:28.016+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 28896 recovery UsedSpace 21228 2023-10-21T04:09:28.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 30059 recovery UsedSpace 20910 2023-10-21T04:09:28.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex 2023-10-21T04:09:28.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.026+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex 2023-10-21T04:09:28.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/docIndex started 2023-10-21T04:09:28.027+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.027+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex 2023-10-21T04:09:28.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.028+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex 2023-10-21T04:09:28.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index/mainIndex started 2023-10-21T04:09:28.028+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:28.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.031+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.031+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.041+05:30 [Info] switched currmeta from 179 -> 181 force true 2023-10-21T04:09:28.070+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 8 Warmup took 59.502078ms 2023-10-21T04:09:28.070+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 8 WriterThreads 1 2023-10-21T04:09:28.070+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_8.index Index: 2170849387178940761 Partition: 8 2023-10-21T04:09:28.071+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.071+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 1 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.072+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:09:28.072+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 135 metaSz: 2510 rp [1] sn: 2 count: 135 metaSz: 2575 2023-10-21T04:09:28.073+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 29739 recovery UsedSpace 21228 2023-10-21T04:09:28.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.076+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:28.076+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 135 metaSz: 2510 rp [1] sn: 2 count: 135 metaSz: 2575 2023-10-21T04:09:28.076+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 30306 recovery UsedSpace 20910 2023-10-21T04:09:28.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex 2023-10-21T04:09:28.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.080+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex 2023-10-21T04:09:28.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/docIndex started 2023-10-21T04:09:28.080+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.080+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.083+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 80911002600697866 PartitionId 3 Warmup took 104.673608ms 2023-10-21T04:09:28.085+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 80911002600697866 partitionId 3 WriterThreads 1 2023-10-21T04:09:28.086+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 80911002600697866 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_80911002600697866_3.index Index: 80911002600697866 Partition: 3 2023-10-21T04:09:28.086+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 80911002600697866, shardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:28.086+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 80911002600697866, partnShardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:28.086+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:28.088+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 80911002600697866 2023-10-21T04:09:28.092+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:28.093+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:28.095+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:28.096+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:28.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex 2023-10-21T04:09:28.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.123+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex 2023-10-21T04:09:28.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index/mainIndex started 2023-10-21T04:09:28.123+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:28.098+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:28.132+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 80911002600697866 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:09:28.137+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.137+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.138+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:2 Attempting to open snapshot (SnapshotInfo: count:110 committed:false) 2023-10-21T04:09:28.142+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:3 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:09:28.139+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 52.173053ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:28.143+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 151.078µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:28.140+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.143+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.159+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:4 Attempting to open snapshot (SnapshotInfo: count:140 committed:false) 2023-10-21T04:09:28.164+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:5 Attempting to open snapshot (SnapshotInfo: count:116 committed:false) 2023-10-21T04:09:28.165+05:30 [Info] switched currmeta from 181 -> 182 force true 2023-10-21T04:09:28.168+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:09:28.173+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:7 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:09:28.174+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.176+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.178+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:8 Attempting to open snapshot (SnapshotInfo: count:124 committed:false) 2023-10-21T04:09:28.185+05:30 [Info] StorageMgr::openSnapshot IndexInst:80911002600697866 Partition:1 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:09:28.185+05:30 [Error] ScanCoordinator: Unable to compute index items_count for bucket_1/idx_partitioned/2170849387178940761 state 0 (Index not found) 2023-10-21T04:09:28.186+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:09:28.187+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 1 Warmup took 115.603234ms 2023-10-21T04:09:28.187+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 1 WriterThreads 1 2023-10-21T04:09:28.187+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_1.index Index: 2170849387178940761 Partition: 1 2023-10-21T04:09:28.187+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.187+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 2 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:28.188+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 107 metaSz: 2510 rp [1] sn: 2 count: 107 metaSz: 2575 2023-10-21T04:09:28.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.188+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:28.188+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 107 metaSz: 2510 rp [1] sn: 2 count: 107 metaSz: 2575 2023-10-21T04:09:28.189+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 28160 recovery UsedSpace 20910 2023-10-21T04:09:28.189+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 29862 recovery UsedSpace 20910 2023-10-21T04:09:28.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex 2023-10-21T04:09:28.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.203+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex 2023-10-21T04:09:28.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/docIndex started 2023-10-21T04:09:28.203+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.203+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex 2023-10-21T04:09:28.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.204+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex 2023-10-21T04:09:28.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index/mainIndex started 2023-10-21T04:09:28.205+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.205+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:28.205+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 4004220452657849790 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 16726320877690809560 RealInstId: 16726320877690809560 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 17148287981272169168 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 11400459767051699632 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9664682240473767939 RealInstId: 9664682240473767939 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 17128008689111072444 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:28.238+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 15710672812788661570 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 0 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:28.244+05:30 [Info] ShardRebalancer::computeProgress 0.7029802631578949 2023-10-21T04:09:28.252+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 2 Warmup took 64.413825ms 2023-10-21T04:09:28.253+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 2 WriterThreads 1 2023-10-21T04:09:28.253+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_2.index Index: 2170849387178940761 Partition: 2 2023-10-21T04:09:28.253+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.254+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 3 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.255+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:28.255+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 138 metaSz: 2510 rp [1] sn: 2 count: 138 metaSz: 2575 2023-10-21T04:09:28.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.256+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:28.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 138 metaSz: 2510 rp [1] sn: 2 count: 138 metaSz: 2575 2023-10-21T04:09:28.256+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 30482 recovery UsedSpace 21232 2023-10-21T04:09:28.257+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 31661 recovery UsedSpace 20910 2023-10-21T04:09:28.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex 2023-10-21T04:09:28.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.262+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex 2023-10-21T04:09:28.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/mainIndex started 2023-10-21T04:09:28.263+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.263+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.265+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.265+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex 2023-10-21T04:09:28.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.280+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex 2023-10-21T04:09:28.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index/docIndex started 2023-10-21T04:09:28.281+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:28.282+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.282+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.283+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[80911002600697866:INDEX_STATE_RECOVERED 11400459767051699632:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:28.283+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 18046371353728341033, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:28.283+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 18046371353728341033, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:28.304+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 3 Warmup took 49.60695ms 2023-10-21T04:09:28.305+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 3 WriterThreads 1 2023-10-21T04:09:28.305+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_3.index Index: 2170849387178940761 Partition: 3 2023-10-21T04:09:28.305+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.306+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 4 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.306+05:30 [Info] switched currmeta from 182 -> 183 force true 2023-10-21T04:09:28.307+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13718648355797887482:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:28.307+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13718648355797887482:[0] 15710672812788661570:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:28.307+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[6 7 8 1 2 3 4 5] 13718648355797887482:[0] 15710672812788661570:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:28.307+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[6 7 8 1 2 3 4 5] 13718648355797887482:[0] 15710672812788661570:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:28.308+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[6 7 8 1 2 3 4 5] 13718648355797887482:[0] 15710672812788661570:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:28.308+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[6 7 8 1 2 3 4 5] 13718648355797887482:[0] 15710672812788661570:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:28.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.308+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:28.308+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 137 metaSz: 2510 rp [1] sn: 2 count: 137 metaSz: 2575 2023-10-21T04:09:28.309+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 31678 recovery UsedSpace 20910 2023-10-21T04:09:28.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.309+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 258 2023-10-21T04:09:28.309+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 137 metaSz: 2510 rp [1] sn: 2 count: 137 metaSz: 2575 2023-10-21T04:09:28.310+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 31063 recovery UsedSpace 21232 2023-10-21T04:09:28.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex 2023-10-21T04:09:28.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.317+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex 2023-10-21T04:09:28.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/mainIndex started 2023-10-21T04:09:28.317+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.317+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex 2023-10-21T04:09:28.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.318+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex 2023-10-21T04:09:28.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index/docIndex started 2023-10-21T04:09:28.318+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:28.318+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.318+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.370+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 4 Warmup took 63.571668ms 2023-10-21T04:09:28.370+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 4 WriterThreads 1 2023-10-21T04:09:28.370+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_4.index Index: 2170849387178940761 Partition: 4 2023-10-21T04:09:28.370+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.371+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 5 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.371+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:28.371+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 131 metaSz: 2510 rp [1] sn: 2 count: 131 metaSz: 2575 2023-10-21T04:09:28.371+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:28.371+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 131 metaSz: 2510 rp [1] sn: 2 count: 131 metaSz: 2575 2023-10-21T04:09:28.372+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 32130 recovery UsedSpace 20910 2023-10-21T04:09:28.373+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 31404 recovery UsedSpace 21232 2023-10-21T04:09:28.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex 2023-10-21T04:09:28.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.397+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex 2023-10-21T04:09:28.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/mainIndex started 2023-10-21T04:09:28.397+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.397+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex 2023-10-21T04:09:28.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.398+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex 2023-10-21T04:09:28.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index/docIndex started 2023-10-21T04:09:28.398+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:28.399+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.399+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.406+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8446412942681787061,realInstId: 0, indexDefn DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:28.406+05:30 [Info] Indexer::handleRecoverIndex InstId: 8446412942681787061 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:28.407+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 238.855µs 2023-10-21T04:09:28.407+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 280.531µs 2023-10-21T04:09:28.408+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8446412942681787061 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:28.408+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8446412942681787061 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:28.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:28.408+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:28.411+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 17429 recovery UsedSpace 8236 2023-10-21T04:09:28.415+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8446412942681787061, indexDefn DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:28.415+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 131.399391ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:28.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:28.415+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:28.415+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:28.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:28.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:28.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:28.429+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:28.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex started 2023-10-21T04:09:28.429+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.429+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.429+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.429+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:28.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:28.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:28.434+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:28.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex started 2023-10-21T04:09:28.435+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8446412942681787061 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.435+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8446412942681787061 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:28.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex] 2023-10-21T04:09:28.435+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex] 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex] 2023-10-21T04:09:28.435+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:28.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex stopped 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:28.435+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:28.435+05:30 [Info] switched currmeta from 183 -> 184 force true 2023-10-21T04:09:28.443+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:28.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex] 2023-10-21T04:09:28.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex closed 2023-10-21T04:09:28.444+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:28.444+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:28.444+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:28.444+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:28.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex] 2023-10-21T04:09:28.445+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:28.445+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex] 2023-10-21T04:09:28.445+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex] 2023-10-21T04:09:28.445+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:28.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex stopped 2023-10-21T04:09:28.445+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:28.445+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:28.445+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:28.445+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:28.451+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9919552327719938 2023-10-21T04:09:28.465+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 5 Warmup took 94.068206ms 2023-10-21T04:09:28.465+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 5 WriterThreads 1 2023-10-21T04:09:28.465+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_5.index Index: 2170849387178940761 Partition: 5 2023-10-21T04:09:28.465+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.465+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 6 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.466+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:28.466+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 115 metaSz: 2510 rp [1] sn: 2 count: 115 metaSz: 2575 2023-10-21T04:09:28.466+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 30147 recovery UsedSpace 20910 2023-10-21T04:09:28.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.467+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:28.467+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 115 metaSz: 2510 rp [1] sn: 2 count: 115 metaSz: 2575 2023-10-21T04:09:28.467+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 31445 recovery UsedSpace 20910 2023-10-21T04:09:28.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:28.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex] 2023-10-21T04:09:28.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex closed 2023-10-21T04:09:28.471+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:28.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex ... 2023-10-21T04:09:28.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex ... 2023-10-21T04:09:28.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:28.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex sucessfully destroyed 2023-10-21T04:09:28.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex ... 2023-10-21T04:09:28.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex ... 2023-10-21T04:09:28.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex 2023-10-21T04:09:28.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.479+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex 2023-10-21T04:09:28.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/docIndex started 2023-10-21T04:09:28.479+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.480+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex 2023-10-21T04:09:28.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.480+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex 2023-10-21T04:09:28.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index/mainIndex started 2023-10-21T04:09:28.480+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:28.481+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.481+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:28.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index destroyed 2023-10-21T04:09:28.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:28.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:28.488+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:28.488+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8446412942681787061:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:28.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:28.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:28.496+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:28.497+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8446412942681787061:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:28.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 64 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:28.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:28.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:28.506+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:28.506+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex 2023-10-21T04:09:28.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/docIndex started 2023-10-21T04:09:28.506+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8446412942681787061:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.506+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8446412942681787061:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.509+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 6 Warmup took 43.530799ms 2023-10-21T04:09:28.509+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 6 WriterThreads 1 2023-10-21T04:09:28.509+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_6.index Index: 2170849387178940761 Partition: 6 2023-10-21T04:09:28.509+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2170849387178940761 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:28.509+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2170849387178940761 PartitionId: 7 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:28.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:28.510+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:28.510+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 109 metaSz: 2510 rp [1] sn: 2 count: 109 metaSz: 2575 2023-10-21T04:09:28.510+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 30538 recovery UsedSpace 20910 2023-10-21T04:09:28.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:28.511+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:28.511+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 109 metaSz: 2510 rp [1] sn: 2 count: 109 metaSz: 2575 2023-10-21T04:09:28.511+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 31913 recovery UsedSpace 20910 2023-10-21T04:09:28.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 72 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:28.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:28.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:28.512+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:28.512+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex 2023-10-21T04:09:28.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index/mainIndex started 2023-10-21T04:09:28.512+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8446412942681787061 PartitionId 0 Warmup took 103.830989ms 2023-10-21T04:09:28.512+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8446412942681787061 partitionId 0 WriterThreads 4 2023-10-21T04:09:28.512+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8446412942681787061 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8446412942681787061_0.index Index: 8446412942681787061 Partition: 0 2023-10-21T04:09:28.512+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8446412942681787061, shardIdMap: map[0:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:28.512+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8446412942681787061, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:28.512+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8446412942681787061 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:28.513+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8446412942681787061 2023-10-21T04:09:28.513+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8446412942681787061 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:28.513+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:28.513+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:28.513+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8446412942681787061 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:28.513+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8446412942681787061 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:28.514+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8446412942681787061 Partitions [0] 2023-10-21T04:09:28.514+05:30 [Info] StorageMgr::openSnapshot IndexInst:8446412942681787061 Partition:0 No Snapshot Found. 2023-10-21T04:09:28.514+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8446412942681787061 Adding Nil Snapshot. 2023-10-21T04:09:28.514+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8446412942681787061:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.514+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8446412942681787061:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex 2023-10-21T04:09:28.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:28.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:28.522+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex 2023-10-21T04:09:28.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/docIndex started 2023-10-21T04:09:28.522+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.522+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex 2023-10-21T04:09:28.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:28.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:28.523+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex 2023-10-21T04:09:28.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index/mainIndex started 2023-10-21T04:09:28.523+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:28.523+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:28.523+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:28.523+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.181575ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:28.523+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 80.566µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:28.533+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.533+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.539+05:30 [Info] switched currmeta from 184 -> 185 force true 2023-10-21T04:09:28.553+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2170849387178940761 PartitionId 7 Warmup took 43.157549ms 2023-10-21T04:09:28.553+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2170849387178940761 partitionId 7 WriterThreads 1 2023-10-21T04:09:28.553+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2170849387178940761 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_2170849387178940761_7.index Index: 2170849387178940761 Partition: 7 2023-10-21T04:09:28.553+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2170849387178940761, shardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:28.553+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2170849387178940761, partnShardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:28.553+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:28.554+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2170849387178940761 2023-10-21T04:09:28.554+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:28.554+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:28.554+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:28.554+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:28.554+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:28.556+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2170849387178940761 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:09:28.557+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:2 Attempting to open snapshot (SnapshotInfo: count:107 committed:false) 2023-10-21T04:09:28.558+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:3 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:09:28.559+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:4 Attempting to open snapshot (SnapshotInfo: count:137 committed:false) 2023-10-21T04:09:28.560+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:5 Attempting to open snapshot (SnapshotInfo: count:131 committed:false) 2023-10-21T04:09:28.561+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:6 Attempting to open snapshot (SnapshotInfo: count:115 committed:false) 2023-10-21T04:09:28.564+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:7 Attempting to open snapshot (SnapshotInfo: count:109 committed:false) 2023-10-21T04:09:28.566+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:8 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:09:28.567+05:30 [Info] StorageMgr::openSnapshot IndexInst:2170849387178940761 Partition:1 Attempting to open snapshot (SnapshotInfo: count:135 committed:false) 2023-10-21T04:09:28.579+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 24.172539ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:28.586+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 6.146579ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:28.596+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.603+05:30 [Info] switched currmeta from 185 -> 186 force true 2023-10-21T04:09:28.647+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:28.653+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:09:28.665+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.665+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.669+05:30 [Info] switched currmeta from 186 -> 186 force true 2023-10-21T04:09:28.694+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:09:28.694+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:09:28.729+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.729+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.738+05:30 [Info] switched currmeta from 186 -> 187 force true 2023-10-21T04:09:28.784+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:28.784+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:28.788+05:30 [Info] switched currmeta from 187 -> 188 force true 2023-10-21T04:09:29.001+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.001+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.004+05:30 [Info] switched currmeta from 188 -> 189 force true 2023-10-21T04:09:29.019+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2170849387178940761:INDEX_STATE_RECOVERED 4004220452657849790:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:29.020+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 10585832761158485697, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:29.020+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 10585832761158485697, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:29.039+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7755999278497385093,realInstId: 0, indexDefn DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:29.042+05:30 [Info] Indexer::handleRecoverIndex InstId: 7755999278497385093 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:29.044+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.044+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 2 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.043+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 1.806552ms 2023-10-21T04:09:29.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.046+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.044+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7755999278497385093, indexDefn DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:29.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.045+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.29588ms 2023-10-21T04:09:29.048+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.048+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.048+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.049+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.151459ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:29.053+05:30 [Info] switched currmeta from 189 -> 190 force true 2023-10-21T04:09:29.056+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:29.057+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 17808 recovery UsedSpace 8236 2023-10-21T04:09:29.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 56 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:29.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.075+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:29.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex started 2023-10-21T04:09:29.075+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.075+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:29.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.077+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:29.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex started 2023-10-21T04:09:29.078+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:29.078+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex] 2023-10-21T04:09:29.078+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.078+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex] 2023-10-21T04:09:29.078+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex] 2023-10-21T04:09:29.078+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:29.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex stopped 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.078+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex] 2023-10-21T04:09:29.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex closed 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex] 2023-10-21T04:09:29.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:29.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex] 2023-10-21T04:09:29.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex] 2023-10-21T04:09:29.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:29.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex stopped 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.102+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.115+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex] 2023-10-21T04:09:29.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex closed 2023-10-21T04:09:29.115+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex ... 2023-10-21T04:09:29.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 56 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex ... 2023-10-21T04:09:29.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex ... 2023-10-21T04:09:29.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex ... 2023-10-21T04:09:29.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index destroyed 2023-10-21T04:09:29.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.138+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.138+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.143+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.143+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 65 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:29.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.147+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:29.147+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex 2023-10-21T04:09:29.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/docIndex started 2023-10-21T04:09:29.147+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.147+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 73 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:29.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.151+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:29.151+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex 2023-10-21T04:09:29.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index/mainIndex started 2023-10-21T04:09:29.151+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 2 Warmup took 106.807052ms 2023-10-21T04:09:29.151+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 2 WriterThreads 1 2023-10-21T04:09:29.151+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 2 Stopped Billing WUs 2023-10-21T04:09:29.151+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_2.index Index: 7755999278497385093 Partition: 2 2023-10-21T04:09:29.151+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.151+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 3 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.152+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.152+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.152+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.152+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 17808 recovery UsedSpace 8236 2023-10-21T04:09:29.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.152+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.153+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:29.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.168+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:29.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex started 2023-10-21T04:09:29.168+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.168+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 57 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:29.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.168+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:29.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex started 2023-10-21T04:09:29.169+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:29.169+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.169+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.169+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.169+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.169+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.169+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex] 2023-10-21T04:09:29.170+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:29.171+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex] 2023-10-21T04:09:29.171+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex] 2023-10-21T04:09:29.171+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:29.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex stopped 2023-10-21T04:09:29.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.197+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex] 2023-10-21T04:09:29.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex closed 2023-10-21T04:09:29.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.198+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.198+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.199+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex] 2023-10-21T04:09:29.199+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:29.199+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex] 2023-10-21T04:09:29.199+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex] 2023-10-21T04:09:29.199+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:29.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex stopped 2023-10-21T04:09:29.199+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.199+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.199+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.199+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.223+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex] 2023-10-21T04:09:29.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex closed 2023-10-21T04:09:29.225+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex ... 2023-10-21T04:09:29.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 57 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex ... 2023-10-21T04:09:29.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex ... 2023-10-21T04:09:29.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex ... 2023-10-21T04:09:29.246+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.246+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.249+05:30 [Info] switched currmeta from 190 -> 191 force true 2023-10-21T04:09:29.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index destroyed 2023-10-21T04:09:29.277+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.277+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.280+05:30 [Info] switched currmeta from 191 -> 192 force true 2023-10-21T04:09:29.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.282+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.282+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.291+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.291+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 66 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:29.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.301+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:29.301+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex 2023-10-21T04:09:29.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/docIndex started 2023-10-21T04:09:29.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 74 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:29.304+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.304+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.305+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:29.304+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.305+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.305+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex 2023-10-21T04:09:29.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index/mainIndex started 2023-10-21T04:09:29.307+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 3 Warmup took 155.978211ms 2023-10-21T04:09:29.307+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.307+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.308+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 3 WriterThreads 1 2023-10-21T04:09:29.310+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 3 Stopped Billing WUs 2023-10-21T04:09:29.310+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_3.index Index: 7755999278497385093 Partition: 3 2023-10-21T04:09:29.311+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.313+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 4 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.315+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.315+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.317+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 17808 recovery UsedSpace 8236 2023-10-21T04:09:29.316+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.335+05:30 [Info] switched currmeta from 192 -> 193 force true 2023-10-21T04:09:29.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:29.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.336+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:29.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex started 2023-10-21T04:09:29.336+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.336+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 58 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:29.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.336+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:29.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex started 2023-10-21T04:09:29.336+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:29.336+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.336+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.336+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.336+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.337+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex] 2023-10-21T04:09:29.337+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:29.337+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex] 2023-10-21T04:09:29.337+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex] 2023-10-21T04:09:29.337+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:29.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex stopped 2023-10-21T04:09:29.337+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.337+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.337+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.337+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.337+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.352+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex] 2023-10-21T04:09:29.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex closed 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex] 2023-10-21T04:09:29.353+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:29.353+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex] 2023-10-21T04:09:29.353+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex] 2023-10-21T04:09:29.353+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:29.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex stopped 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.353+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.361+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex] 2023-10-21T04:09:29.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex closed 2023-10-21T04:09:29.361+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex ... 2023-10-21T04:09:29.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 58 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex ... 2023-10-21T04:09:29.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex ... 2023-10-21T04:09:29.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex ... 2023-10-21T04:09:29.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index destroyed 2023-10-21T04:09:29.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.374+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.374+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.378+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.378+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 67 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:29.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.380+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:29.380+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex 2023-10-21T04:09:29.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/docIndex started 2023-10-21T04:09:29.380+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.381+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 75 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:29.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.384+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:29.384+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex 2023-10-21T04:09:29.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index/mainIndex started 2023-10-21T04:09:29.384+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 4 Warmup took 70.840391ms 2023-10-21T04:09:29.385+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.385+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.385+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 4 WriterThreads 1 2023-10-21T04:09:29.385+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 4 Stopped Billing WUs 2023-10-21T04:09:29.385+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_4.index Index: 7755999278497385093 Partition: 4 2023-10-21T04:09:29.385+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.385+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 5 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.385+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.385+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.385+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:29.386+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:29.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.396+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:29.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex started 2023-10-21T04:09:29.396+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.396+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 59 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:29.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.400+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:29.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex started 2023-10-21T04:09:29.400+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:29.400+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex] 2023-10-21T04:09:29.400+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:29.400+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex] 2023-10-21T04:09:29.400+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex] 2023-10-21T04:09:29.400+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:29.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex stopped 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.400+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.401+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.407+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex] 2023-10-21T04:09:29.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex closed 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex] 2023-10-21T04:09:29.408+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:29.408+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex] 2023-10-21T04:09:29.408+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex] 2023-10-21T04:09:29.408+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:29.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex stopped 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.408+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.416+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex] 2023-10-21T04:09:29.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex closed 2023-10-21T04:09:29.417+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex ... 2023-10-21T04:09:29.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 59 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex ... 2023-10-21T04:09:29.418+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.418+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.424+05:30 [Info] switched currmeta from 193 -> 194 force true 2023-10-21T04:09:29.426+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8446412942681787061:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:29.426+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8446412942681787061 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc008373140 2023-10-21T04:09:29.426+05:30 [Info] handleUpdateIndexRState: Index instance 8446412942681787061 rstate moved to ACTIVE 2023-10-21T04:09:29.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex ... 2023-10-21T04:09:29.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex ... 2023-10-21T04:09:29.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index destroyed 2023-10-21T04:09:29.447+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:29.447+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 3] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029802631578949 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:29.448+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.175433ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:29.450+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 6] 2023-10-21T04:09:29.451+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 6] 2023-10-21T04:09:29.451+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 15431181238515168303, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:29.451+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 15431181238515168303, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:29.452+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9844931626502713 2023-10-21T04:09:29.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.457+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.457+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.459+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.459+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.461+05:30 [Info] switched currmeta from 194 -> 195 force true 2023-10-21T04:09:29.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.465+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.466+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 68 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:29.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.469+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:29.469+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex 2023-10-21T04:09:29.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/docIndex started 2023-10-21T04:09:29.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 76 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:29.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.473+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:29.473+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex 2023-10-21T04:09:29.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index/mainIndex started 2023-10-21T04:09:29.473+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 5 Warmup took 88.528555ms 2023-10-21T04:09:29.473+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.474+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.474+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 5 WriterThreads 1 2023-10-21T04:09:29.474+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 5 Stopped Billing WUs 2023-10-21T04:09:29.474+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_5.index Index: 7755999278497385093 Partition: 5 2023-10-21T04:09:29.474+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.474+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 6 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.474+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.475+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.475+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:29.475+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:29.483+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2921784364458449578,realInstId: 0, indexDefn DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:29.487+05:30 [Info] Indexer::handleRecoverIndex InstId: 2921784364458449578 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:29.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:29.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.494+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:29.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex started 2023-10-21T04:09:29.490+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 2.982406ms 2023-10-21T04:09:29.490+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.019706ms 2023-10-21T04:09:29.490+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:29.494+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.490+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2921784364458449578, indexDefn DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:29.494+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 43.522391ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:29.494+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.493+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.494+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.495+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 1 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:29.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.495+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.497+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 17429 recovery UsedSpace 8236 2023-10-21T04:09:29.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.498+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.498+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:29.499+05:30 [Info] switched currmeta from 195 -> 196 force true 2023-10-21T04:09:29.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 60 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:29.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.503+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:29.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex started 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.503+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:29.503+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex] 2023-10-21T04:09:29.503+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:29.503+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex] 2023-10-21T04:09:29.503+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex] 2023-10-21T04:09:29.503+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:29.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex stopped 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.503+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:29.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.506+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:29.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex started 2023-10-21T04:09:29.507+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.507+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.508+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex] 2023-10-21T04:09:29.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex closed 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex] 2023-10-21T04:09:29.509+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:29.509+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex] 2023-10-21T04:09:29.509+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex] 2023-10-21T04:09:29.509+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:29.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex stopped 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.509+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.515+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex] 2023-10-21T04:09:29.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex closed 2023-10-21T04:09:29.516+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:29.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.517+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:29.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex started 2023-10-21T04:09:29.517+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:29.517+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.517+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.517+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex] 2023-10-21T04:09:29.518+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex] 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex] 2023-10-21T04:09:29.518+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:29.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex stopped 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex ... 2023-10-21T04:09:29.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 60 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex ... 2023-10-21T04:09:29.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.524+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex] 2023-10-21T04:09:29.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex closed 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex] 2023-10-21T04:09:29.525+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex] 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex] 2023-10-21T04:09:29.525+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:29.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex stopped 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.525+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex ... 2023-10-21T04:09:29.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex ... 2023-10-21T04:09:29.531+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex] 2023-10-21T04:09:29.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex closed 2023-10-21T04:09:29.532+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex ... 2023-10-21T04:09:29.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex ... 2023-10-21T04:09:29.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index destroyed 2023-10-21T04:09:29.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex sucessfully destroyed 2023-10-21T04:09:29.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex ... 2023-10-21T04:09:29.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex ... 2023-10-21T04:09:29.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index destroyed 2023-10-21T04:09:29.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.554+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.554+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.554+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.554+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.567+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.567+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.567+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.567+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 73 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:29.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.572+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:29.572+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex 2023-10-21T04:09:29.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/mainIndex started 2023-10-21T04:09:29.573+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.573+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 69 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:29.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.573+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:29.573+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex 2023-10-21T04:09:29.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/docIndex started 2023-10-21T04:09:29.573+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.574+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 65 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.581+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:29.581+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index/docIndex started 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 77 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.581+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:29.581+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex 2023-10-21T04:09:29.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index/mainIndex started 2023-10-21T04:09:29.581+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 1 Warmup took 86.091246ms 2023-10-21T04:09:29.581+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 6 Warmup took 107.468032ms 2023-10-21T04:09:29.581+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.582+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.582+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 6 WriterThreads 1 2023-10-21T04:09:29.582+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 6 Stopped Billing WUs 2023-10-21T04:09:29.582+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_6.index Index: 7755999278497385093 Partition: 6 2023-10-21T04:09:29.582+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.582+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 1 WriterThreads 1 2023-10-21T04:09:29.582+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 7 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.582+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 1 Stopped Billing WUs 2023-10-21T04:09:29.582+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_1.index Index: 2921784364458449578 Partition: 1 2023-10-21T04:09:29.582+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:29.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.582+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.582+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.583+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.582+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 2 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:29.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.583+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.583+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:29.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.583+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.583+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 17429 recovery UsedSpace 8236 2023-10-21T04:09:29.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 61 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:29.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.598+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:29.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex started 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:29.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.598+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:29.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex started 2023-10-21T04:09:29.598+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:29.598+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex] 2023-10-21T04:09:29.598+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:29.598+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex] 2023-10-21T04:09:29.598+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex] 2023-10-21T04:09:29.598+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:29.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex stopped 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.602+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex started 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.602+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex started 2023-10-21T04:09:29.602+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:29.602+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex] 2023-10-21T04:09:29.602+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex] 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex] 2023-10-21T04:09:29.602+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:29.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex stopped 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.610+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex] 2023-10-21T04:09:29.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex closed 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex] 2023-10-21T04:09:29.611+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:29.611+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex] 2023-10-21T04:09:29.611+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex] 2023-10-21T04:09:29.611+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:29.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex stopped 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.611+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.612+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex] 2023-10-21T04:09:29.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex closed 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex] 2023-10-21T04:09:29.613+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex] 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex] 2023-10-21T04:09:29.613+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:29.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex stopped 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.621+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex] 2023-10-21T04:09:29.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex closed 2023-10-21T04:09:29.621+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex ... 2023-10-21T04:09:29.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex ... 2023-10-21T04:09:29.621+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex] 2023-10-21T04:09:29.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex closed 2023-10-21T04:09:29.622+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex sucessfully destroyed 2023-10-21T04:09:29.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex ... 2023-10-21T04:09:29.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex ... 2023-10-21T04:09:29.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index destroyed 2023-10-21T04:09:29.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex ... 2023-10-21T04:09:29.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 61 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex ... 2023-10-21T04:09:29.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex ... 2023-10-21T04:09:29.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex ... 2023-10-21T04:09:29.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index destroyed 2023-10-21T04:09:29.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.639+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.645+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.645+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.646+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.646+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 66 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:29.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.649+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:29.649+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex 2023-10-21T04:09:29.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/docIndex started 2023-10-21T04:09:29.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.662+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.662+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 70 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:29.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.674+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:29.674+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex 2023-10-21T04:09:29.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/docIndex started 2023-10-21T04:09:29.674+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.674+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 74 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:29.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.679+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:29.679+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex 2023-10-21T04:09:29.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index/mainIndex started 2023-10-21T04:09:29.679+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 2 Warmup took 96.224431ms 2023-10-21T04:09:29.679+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.679+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.679+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 2 WriterThreads 1 2023-10-21T04:09:29.679+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 2 Stopped Billing WUs 2023-10-21T04:09:29.679+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_2.index Index: 2921784364458449578 Partition: 2 2023-10-21T04:09:29.679+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:29.679+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 3 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:29.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.680+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.680+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 17808 recovery UsedSpace 8236 2023-10-21T04:09:29.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.681+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 78 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:29.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.681+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:29.681+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex 2023-10-21T04:09:29.681+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index/mainIndex started 2023-10-21T04:09:29.681+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 7 Warmup took 99.014603ms 2023-10-21T04:09:29.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.681+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 7 WriterThreads 1 2023-10-21T04:09:29.681+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 7 Stopped Billing WUs 2023-10-21T04:09:29.681+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_7.index Index: 7755999278497385093 Partition: 7 2023-10-21T04:09:29.681+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.684+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 8 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.685+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.685+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.686+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.686+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.697+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex started 2023-10-21T04:09:29.697+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.697+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.697+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex started 2023-10-21T04:09:29.697+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:29.697+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.697+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.697+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.697+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex] 2023-10-21T04:09:29.698+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex] 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex] 2023-10-21T04:09:29.698+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:29.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex stopped 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.698+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:29.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.703+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:29.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex started 2023-10-21T04:09:29.703+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:29.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.704+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:29.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex started 2023-10-21T04:09:29.704+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:29.704+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex] 2023-10-21T04:09:29.704+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:29.704+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex] 2023-10-21T04:09:29.704+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex] 2023-10-21T04:09:29.704+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:29.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex stopped 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.704+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex] 2023-10-21T04:09:29.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex closed 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex] 2023-10-21T04:09:29.709+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex] 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex] 2023-10-21T04:09:29.709+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:29.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex stopped 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex] 2023-10-21T04:09:29.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex closed 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex] 2023-10-21T04:09:29.714+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:29.714+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex] 2023-10-21T04:09:29.714+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex] 2023-10-21T04:09:29.714+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:29.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex stopped 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.714+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.718+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex] 2023-10-21T04:09:29.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex closed 2023-10-21T04:09:29.718+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex ... 2023-10-21T04:09:29.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex ... 2023-10-21T04:09:29.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex sucessfully destroyed 2023-10-21T04:09:29.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex ... 2023-10-21T04:09:29.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex ... 2023-10-21T04:09:29.726+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex] 2023-10-21T04:09:29.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex closed 2023-10-21T04:09:29.727+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index destroyed 2023-10-21T04:09:29.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex ... 2023-10-21T04:09:29.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex ... 2023-10-21T04:09:29.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex ... 2023-10-21T04:09:29.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex ... 2023-10-21T04:09:29.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index destroyed 2023-10-21T04:09:29.741+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.741+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.741+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.741+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.743+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.743+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.746+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.746+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.749+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 79 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:29.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.750+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:29.750+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex 2023-10-21T04:09:29.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/mainIndex started 2023-10-21T04:09:29.750+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.750+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 67 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:29.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.751+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:29.751+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex 2023-10-21T04:09:29.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/docIndex started 2023-10-21T04:09:29.751+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.751+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 71 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:29.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.757+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:29.757+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex 2023-10-21T04:09:29.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index/docIndex started 2023-10-21T04:09:29.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.757+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 8 Warmup took 72.601482ms 2023-10-21T04:09:29.757+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 8 WriterThreads 1 2023-10-21T04:09:29.757+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 8 Stopped Billing WUs 2023-10-21T04:09:29.758+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_8.index Index: 7755999278497385093 Partition: 8 2023-10-21T04:09:29.758+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7755999278497385093 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:29.758+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7755999278497385093 PartitionId: 1 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:29.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.758+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.759+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.759+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 75 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:29.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.759+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:29.759+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex 2023-10-21T04:09:29.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index/mainIndex started 2023-10-21T04:09:29.759+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 3 Warmup took 79.977495ms 2023-10-21T04:09:29.760+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 3 WriterThreads 1 2023-10-21T04:09:29.760+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 3 Stopped Billing WUs 2023-10-21T04:09:29.760+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_3.index Index: 2921784364458449578 Partition: 3 2023-10-21T04:09:29.760+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:29.760+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 4 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:29.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.760+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.761+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.761+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.762+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.762+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:29.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.763+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:29.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex started 2023-10-21T04:09:29.763+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.763+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:29.763+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.763+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 17808 recovery UsedSpace 8236 2023-10-21T04:09:29.763+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:29.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:29.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.768+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:29.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex started 2023-10-21T04:09:29.769+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.769+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 55 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:29.770+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.770+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex started 2023-10-21T04:09:29.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex started 2023-10-21T04:09:29.771+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:29.771+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex] 2023-10-21T04:09:29.771+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:29.771+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex] 2023-10-21T04:09:29.771+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex] 2023-10-21T04:09:29.771+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:29.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex stopped 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.771+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.771+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:29.772+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex] 2023-10-21T04:09:29.772+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex] 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex] 2023-10-21T04:09:29.772+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:29.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex stopped 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.791+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex] 2023-10-21T04:09:29.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex closed 2023-10-21T04:09:29.792+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.792+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.792+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.792+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex] 2023-10-21T04:09:29.792+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:29.792+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex] 2023-10-21T04:09:29.792+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex] 2023-10-21T04:09:29.792+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:29.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex stopped 2023-10-21T04:09:29.792+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.792+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.792+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.793+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.795+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex] 2023-10-21T04:09:29.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex closed 2023-10-21T04:09:29.795+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.795+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.795+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.795+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex] 2023-10-21T04:09:29.795+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:29.795+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex] 2023-10-21T04:09:29.795+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex] 2023-10-21T04:09:29.795+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:29.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex stopped 2023-10-21T04:09:29.796+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.796+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.796+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.796+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.808+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex] 2023-10-21T04:09:29.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex closed 2023-10-21T04:09:29.809+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex ... 2023-10-21T04:09:29.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 55 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex ... 2023-10-21T04:09:29.811+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex] 2023-10-21T04:09:29.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex closed 2023-10-21T04:09:29.812+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex ... 2023-10-21T04:09:29.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex ... 2023-10-21T04:09:29.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex sucessfully destroyed 2023-10-21T04:09:29.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex ... 2023-10-21T04:09:29.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex ... 2023-10-21T04:09:29.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex sucessfully destroyed 2023-10-21T04:09:29.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex ... 2023-10-21T04:09:29.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex ... 2023-10-21T04:09:29.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index destroyed 2023-10-21T04:09:29.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index destroyed 2023-10-21T04:09:29.848+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.848+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:29.857+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.857+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.859+05:30 [Info] switched currmeta from 196 -> 197 force true 2023-10-21T04:09:29.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:29.863+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.863+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.865+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.865+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 72 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:29.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:29.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:29.866+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:29.866+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex 2023-10-21T04:09:29.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/docIndex started 2023-10-21T04:09:29.866+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.866+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7755999278497385093:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.871+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.871+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 80 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:29.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:29.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:29.872+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:29.872+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex 2023-10-21T04:09:29.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index/mainIndex started 2023-10-21T04:09:29.872+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.872+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7755999278497385093:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.872+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7755999278497385093 PartitionId 1 Warmup took 114.401072ms 2023-10-21T04:09:29.873+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7755999278497385093 partitionId 1 WriterThreads 1 2023-10-21T04:09:29.873+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7755999278497385093 PartitionId 1 Stopped Billing WUs 2023-10-21T04:09:29.873+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7755999278497385093 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7755999278497385093_1.index Index: 7755999278497385093 Partition: 1 2023-10-21T04:09:29.873+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7755999278497385093, shardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:29.873+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7755999278497385093, partnShardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:29.873+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7755999278497385093 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:29.874+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7755999278497385093 2023-10-21T04:09:29.874+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7755999278497385093 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:29.874+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:29.874+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:29.874+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7755999278497385093 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:29.874+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7755999278497385093 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:29.875+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7755999278497385093 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:09:29.875+05:30 [Info] StorageMgr::openSnapshot IndexInst:7755999278497385093 Partition:3 No Snapshot Found. 2023-10-21T04:09:29.875+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7755999278497385093 Adding Nil Snapshot. 2023-10-21T04:09:29.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 76 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:29.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.886+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:29.886+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex 2023-10-21T04:09:29.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/mainIndex started 2023-10-21T04:09:29.886+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.886+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.888+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.746321ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:29.888+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 251.016µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:29.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 68 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:29.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.892+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:29.892+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex 2023-10-21T04:09:29.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index/docIndex started 2023-10-21T04:09:29.892+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 4 Warmup took 132.144249ms 2023-10-21T04:09:29.892+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.892+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.893+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 4 WriterThreads 1 2023-10-21T04:09:29.893+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 4 Stopped Billing WUs 2023-10-21T04:09:29.893+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_4.index Index: 2921784364458449578 Partition: 4 2023-10-21T04:09:29.893+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:29.893+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 5 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:29.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.894+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.894+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.899+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:29.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:29.917+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.917+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.920+05:30 [Info] switched currmeta from 197 -> 199 force true 2023-10-21T04:09:29.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:29.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.921+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:29.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex started 2023-10-21T04:09:29.921+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.921+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:29.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:29.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex started 2023-10-21T04:09:29.922+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:29.922+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex] 2023-10-21T04:09:29.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex] 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex] 2023-10-21T04:09:29.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:29.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex stopped 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.927+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:29.927+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:29.929+05:30 [Info] switched currmeta from 199 -> 199 force true 2023-10-21T04:09:29.933+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex] 2023-10-21T04:09:29.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex closed 2023-10-21T04:09:29.933+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.933+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.933+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.934+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex] 2023-10-21T04:09:29.934+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:29.934+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex] 2023-10-21T04:09:29.934+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex] 2023-10-21T04:09:29.934+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:29.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex stopped 2023-10-21T04:09:29.934+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.934+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.934+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.934+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.938+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex] 2023-10-21T04:09:29.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex closed 2023-10-21T04:09:29.939+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex ... 2023-10-21T04:09:29.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex ... 2023-10-21T04:09:29.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex sucessfully destroyed 2023-10-21T04:09:29.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex ... 2023-10-21T04:09:29.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex ... 2023-10-21T04:09:29.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index destroyed 2023-10-21T04:09:29.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.950+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.951+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:29.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:29.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:29.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 69 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:29.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.964+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:29.964+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex 2023-10-21T04:09:29.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/docIndex started 2023-10-21T04:09:29.964+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.964+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 77 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:29.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.967+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:29.967+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex 2023-10-21T04:09:29.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index/mainIndex started 2023-10-21T04:09:29.967+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 5 Warmup took 73.693414ms 2023-10-21T04:09:29.967+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 5 WriterThreads 1 2023-10-21T04:09:29.967+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 5 Stopped Billing WUs 2023-10-21T04:09:29.967+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_5.index Index: 2921784364458449578 Partition: 5 2023-10-21T04:09:29.967+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:29.967+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 6 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:29.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:29.968+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:29.968+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:29.969+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:29.969+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:29.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:29.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:29.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:29.978+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:29.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex started 2023-10-21T04:09:29.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:29.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:29.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:29.979+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:29.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex started 2023-10-21T04:09:29.979+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:29.979+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:29.979+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.979+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:29.979+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex] 2023-10-21T04:09:29.980+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex] 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex] 2023-10-21T04:09:29.980+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:29.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex stopped 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.980+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.986+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex] 2023-10-21T04:09:29.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex closed 2023-10-21T04:09:29.986+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:29.986+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:29.986+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:29.986+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:29.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex] 2023-10-21T04:09:29.987+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:29.987+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex] 2023-10-21T04:09:29.987+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex] 2023-10-21T04:09:29.987+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:29.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex stopped 2023-10-21T04:09:29.987+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:29.987+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:29.987+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:29.987+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:29.991+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:29.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex] 2023-10-21T04:09:29.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex closed 2023-10-21T04:09:29.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:29.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex ... 2023-10-21T04:09:29.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex ... 2023-10-21T04:09:30.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex sucessfully destroyed 2023-10-21T04:09:30.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex ... 2023-10-21T04:09:30.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex ... 2023-10-21T04:09:30.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index destroyed 2023-10-21T04:09:30.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:30.009+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.009+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:30.015+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.015+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 78 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:30.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:30.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:30.018+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:30.019+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex 2023-10-21T04:09:30.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/mainIndex started 2023-10-21T04:09:30.019+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.019+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 70 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:30.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:30.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:30.026+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:30.026+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex 2023-10-21T04:09:30.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index/docIndex started 2023-10-21T04:09:30.026+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 6 Warmup took 58.343423ms 2023-10-21T04:09:30.026+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 6 WriterThreads 1 2023-10-21T04:09:30.026+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 6 Stopped Billing WUs 2023-10-21T04:09:30.026+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_6.index Index: 2921784364458449578 Partition: 6 2023-10-21T04:09:30.026+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:30.026+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 7 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:30.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:30.027+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:30.027+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18204 recovery UsedSpace 8236 2023-10-21T04:09:30.029+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:30.029+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:30.029+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.031+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:30.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:30.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:30.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:30.040+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:30.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex started 2023-10-21T04:09:30.040+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.040+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:30.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:30.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:30.046+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:30.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex started 2023-10-21T04:09:30.046+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:30.046+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:30.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex] 2023-10-21T04:09:30.046+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex] 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex] 2023-10-21T04:09:30.046+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:30.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex stopped 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:30.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:30.047+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:30.056+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[7755999278497385093:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:30.056+05:30 [Info] Indexer::handleMergePartition Source 4248339762727765604 Target 7755999278497385093 2023-10-21T04:09:30.056+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7755999278497385093 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00b4fd140 2023-10-21T04:09:30.057+05:30 [Info] MergePartition: sent async request to update index instance 7755999278497385093 rstate moved to ACTIVE 2023-10-21T04:09:30.060+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:30.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex] 2023-10-21T04:09:30.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex closed 2023-10-21T04:09:30.060+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:30.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex] 2023-10-21T04:09:30.061+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex] 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex] 2023-10-21T04:09:30.061+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:30.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex stopped 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:30.061+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:30.075+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.024553ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:30.075+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 4248339762727765604, realInst: 7755999278497385093 2023-10-21T04:09:30.075+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 9619308266130016313, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:30.075+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 9619308266130016313, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:30.080+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:30.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex] 2023-10-21T04:09:30.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex closed 2023-10-21T04:09:30.080+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:30.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex ... 2023-10-21T04:09:30.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex ... 2023-10-21T04:09:30.092+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.092+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex sucessfully destroyed 2023-10-21T04:09:30.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex ... 2023-10-21T04:09:30.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex ... 2023-10-21T04:09:30.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index destroyed 2023-10-21T04:09:30.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:30.102+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.102+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.103+05:30 [Info] switched currmeta from 199 -> 200 force true 2023-10-21T04:09:30.104+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4435591218553988516,realInstId: 0, indexDefn DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:30.104+05:30 [Info] Indexer::handleRecoverIndex InstId: 4435591218553988516 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:30.105+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 319.394µs 2023-10-21T04:09:30.105+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4435591218553988516 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:30.107+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4435591218553988516, indexDefn DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:30.108+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 32.433429ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:30.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.107+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 2.715282ms 2023-10-21T04:09:30.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:30.108+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.113+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4435591218553988516 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:30.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:30.114+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:30.115+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 79 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:30.117+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 71 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:30.117+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/mainIndex started 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:30.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.117+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:30.117+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex 2023-10-21T04:09:30.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index/docIndex started 2023-10-21T04:09:30.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.118+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.118+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.118+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 7 Warmup took 91.023054ms 2023-10-21T04:09:30.118+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 7 WriterThreads 1 2023-10-21T04:09:30.118+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 7 Stopped Billing WUs 2023-10-21T04:09:30.118+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_7.index Index: 2921784364458449578 Partition: 7 2023-10-21T04:09:30.118+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2921784364458449578 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:30.118+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2921784364458449578 PartitionId: 8 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:30.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:30.122+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:30.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:30.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:30.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:30.123+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:30.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex started 2023-10-21T04:09:30.124+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4435591218553988516 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:30.124+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4435591218553988516 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:30.124+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:30.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:30.125+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:30.125+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:30.124+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:30.124+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.124+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.126+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.129+05:30 [Info] switched currmeta from 200 -> 201 force true 2023-10-21T04:09:30.129+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:30.129+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:30.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex] 2023-10-21T04:09:30.129+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:30.129+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex] 2023-10-21T04:09:30.129+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex] 2023-10-21T04:09:30.130+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:30.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex stopped 2023-10-21T04:09:30.130+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:30.130+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:30.131+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:30.131+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:30.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:30.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:30.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:30.134+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:30.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex started 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:30.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:30.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:30.135+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:30.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex started 2023-10-21T04:09:30.135+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:30.135+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:30.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex] 2023-10-21T04:09:30.135+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex] 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex] 2023-10-21T04:09:30.135+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:30.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex stopped 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:30.135+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:30.136+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.136+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.140+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:30.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex] 2023-10-21T04:09:30.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex closed 2023-10-21T04:09:30.141+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:30.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex] 2023-10-21T04:09:30.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex closed 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:30.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex] 2023-10-21T04:09:30.142+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex] 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex] 2023-10-21T04:09:30.142+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:30.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex stopped 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:30.142+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:30.141+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:30.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex ... 2023-10-21T04:09:30.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex ... 2023-10-21T04:09:30.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index destroyed 2023-10-21T04:09:30.150+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:30.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex] 2023-10-21T04:09:30.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex closed 2023-10-21T04:09:30.151+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:30.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex ... 2023-10-21T04:09:30.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex ... 2023-10-21T04:09:30.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:30.152+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.152+05:30 [Info] bucket_1/#primary_defer/Mainstore#4435591218553988516:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex sucessfully destroyed 2023-10-21T04:09:30.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex ... 2023-10-21T04:09:30.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex ... 2023-10-21T04:09:30.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 81 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:30.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:30.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:30.161+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:30.161+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex 2023-10-21T04:09:30.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index/mainIndex started 2023-10-21T04:09:30.161+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4435591218553988516 PartitionId 0 Warmup took 47.146311ms 2023-10-21T04:09:30.161+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4435591218553988516 partitionId 0 WriterThreads 4 2023-10-21T04:09:30.161+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4435591218553988516 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_4435591218553988516_0.index Index: 4435591218553988516 Partition: 0 2023-10-21T04:09:30.161+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4435591218553988516, shardIdMap: map[0:[10563635432419104314]] err: 2023-10-21T04:09:30.161+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4435591218553988516, partnShardIdMap: map[0:[10563635432419104314]], err: 2023-10-21T04:09:30.161+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4435591218553988516 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:30.162+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4435591218553988516 2023-10-21T04:09:30.162+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4435591218553988516 Name: #primary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:30.162+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:30.162+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:30.162+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4435591218553988516 Name: #primary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:30.163+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4435591218553988516 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:30.163+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4435591218553988516 Partitions [0] 2023-10-21T04:09:30.163+05:30 [Info] StorageMgr::openSnapshot IndexInst:4435591218553988516 Partition:0 No Snapshot Found. 2023-10-21T04:09:30.163+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4435591218553988516 Adding Nil Snapshot. 2023-10-21T04:09:30.163+05:30 [Info] bucket_1/#primary_defer/Mainstore#4435591218553988516:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.163+05:30 [Info] bucket_1/#primary_defer/Mainstore#4435591218553988516:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index destroyed 2023-10-21T04:09:30.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.178+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.197677ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:30.178+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 69.862µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:30.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:30.185+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:30.190+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.190+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.195+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.200+05:30 [Info] switched currmeta from 201 -> 202 force true 2023-10-21T04:09:30.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 80 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:30.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:30.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:30.201+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:30.201+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex 2023-10-21T04:09:30.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/mainIndex started 2023-10-21T04:09:30.201+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.201+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#2921784364458449578:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 72 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:30.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:30.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:30.208+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:30.208+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex 2023-10-21T04:09:30.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index/docIndex started 2023-10-21T04:09:30.208+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.208+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#2921784364458449578:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.208+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2921784364458449578 PartitionId 8 Warmup took 86.529098ms 2023-10-21T04:09:30.209+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2921784364458449578 partitionId 8 WriterThreads 1 2023-10-21T04:09:30.209+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2921784364458449578 PartitionId 8 Stopped Billing WUs 2023-10-21T04:09:30.209+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2921784364458449578 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_2921784364458449578_8.index Index: 2921784364458449578 Partition: 8 2023-10-21T04:09:30.209+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2921784364458449578, shardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:30.209+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2921784364458449578, partnShardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:30.210+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2921784364458449578 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:30.213+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2921784364458449578 2023-10-21T04:09:30.213+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2921784364458449578 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:30.213+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:30.214+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:30.214+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2921784364458449578 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:30.216+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2921784364458449578 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:30.216+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2921784364458449578 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:30.216+05:30 [Info] StorageMgr::openSnapshot IndexInst:2921784364458449578 Partition:1 No Snapshot Found. 2023-10-21T04:09:30.216+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2921784364458449578 Adding Nil Snapshot. 2023-10-21T04:09:30.223+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.436293ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:30.223+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 99.203µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:30.244+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.244+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.251+05:30 [Info] switched currmeta from 202 -> 203 force true 2023-10-21T04:09:30.319+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.319+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.323+05:30 [Info] switched currmeta from 203 -> 204 force true 2023-10-21T04:09:30.344+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.344+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.347+05:30 [Info] switched currmeta from 204 -> 205 force true 2023-10-21T04:09:30.389+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.389+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.394+05:30 [Info] switched currmeta from 205 -> 206 force true 2023-10-21T04:09:30.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9097810086492508, throttleDelayMs (new, old, change): (6, 10, -4) 2023-10-21T04:09:30.501+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2921784364458449578:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:30.501+05:30 [Info] Indexer::handleMergePartition Source 4611974930531028413 Target 2921784364458449578 2023-10-21T04:09:30.501+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2921784364458449578 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00c466120 2023-10-21T04:09:30.501+05:30 [Info] MergePartition: sent async request to update index instance 2921784364458449578 rstate moved to ACTIVE 2023-10-21T04:09:30.507+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 4611974930531028413, realInst: 2921784364458449578 2023-10-21T04:09:30.518+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.518+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.507+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.536678ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:30.518+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 13856704970496601958, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:30.518+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 13856704970496601958, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:30.527+05:30 [Info] switched currmeta from 206 -> 207 force true 2023-10-21T04:09:30.564+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8880461853743568138,realInstId: 0, indexDefn DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:30.570+05:30 [Info] Indexer::handleRecoverIndex InstId: 8880461853743568138 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:30.570+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 708.224µs 2023-10-21T04:09:30.570+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 911.499µs 2023-10-21T04:09:30.571+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8880461853743568138 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:30.571+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8880461853743568138 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:30.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:30.572+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:30.571+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8880461853743568138, indexDefn DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:30.573+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 55.094009ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:30.575+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.575+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.579+05:30 [Info] switched currmeta from 207 -> 208 force true 2023-10-21T04:09:30.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:30.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:30.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:30.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:30.590+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:30.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex started 2023-10-21T04:09:30.590+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8880461853743568138 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:30.590+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8880461853743568138 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:30.590+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:30.590+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.590+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.590+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:30.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex] 2023-10-21T04:09:30.591+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:30.591+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex] 2023-10-21T04:09:30.591+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex] 2023-10-21T04:09:30.591+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:30.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex stopped 2023-10-21T04:09:30.591+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:30.591+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:30.591+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:30.591+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:30.591+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:30.595+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:30.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex] 2023-10-21T04:09:30.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex closed 2023-10-21T04:09:30.596+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:30.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex ... 2023-10-21T04:09:30.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex ... 2023-10-21T04:09:30.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index destroyed 2023-10-21T04:09:30.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:30.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:30.608+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:30.608+05:30 [Info] bucket_%2/#primary_defer/Mainstore#8880461853743568138:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:30.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 81 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:30.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:30.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:30.614+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:30.614+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex 2023-10-21T04:09:30.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index/mainIndex started 2023-10-21T04:09:30.615+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8880461853743568138 PartitionId 0 Warmup took 42.891151ms 2023-10-21T04:09:30.615+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8880461853743568138 partitionId 0 WriterThreads 4 2023-10-21T04:09:30.615+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8880461853743568138 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_8880461853743568138_0.index Index: 8880461853743568138 Partition: 0 2023-10-21T04:09:30.615+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8880461853743568138, shardIdMap: map[0:[6611724767191156837]] err: 2023-10-21T04:09:30.615+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8880461853743568138, partnShardIdMap: map[0:[6611724767191156837]], err: 2023-10-21T04:09:30.615+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8880461853743568138 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:30.615+05:30 [Info] bucket_%2/#primary_defer/Mainstore#8880461853743568138:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:30.615+05:30 [Info] bucket_%2/#primary_defer/Mainstore#8880461853743568138:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:30.616+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8880461853743568138 2023-10-21T04:09:30.616+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8880461853743568138 Name: #primary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:30.616+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:30.616+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:30.616+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8880461853743568138 Name: #primary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:30.617+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8880461853743568138 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:30.617+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8880461853743568138 Partitions [0] 2023-10-21T04:09:30.617+05:30 [Info] StorageMgr::openSnapshot IndexInst:8880461853743568138 Partition:0 No Snapshot Found. 2023-10-21T04:09:30.617+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8880461853743568138 Adding Nil Snapshot. 2023-10-21T04:09:30.628+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.931993ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:30.628+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 77.555µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:30.644+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.644+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.651+05:30 [Info] switched currmeta from 208 -> 209 force true 2023-10-21T04:09:30.898+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:30.898+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:30.901+05:30 [Info] switched currmeta from 209 -> 210 force true 2023-10-21T04:09:31.117+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4435591218553988516:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:31.118+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4435591218553988516 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00ac7d020 2023-10-21T04:09:31.118+05:30 [Info] handleUpdateIndexRState: Index instance 4435591218553988516 rstate moved to ACTIVE 2023-10-21T04:09:31.131+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.033998ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:31.135+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 5103684059183126577, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:31.135+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 5103684059183126577, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:31.156+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.156+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.160+05:30 [Info] switched currmeta from 210 -> 211 force true 2023-10-21T04:09:31.182+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11160011617777336020,realInstId: 0, indexDefn DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:31.182+05:30 [Info] Indexer::handleRecoverIndex InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:31.183+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 152.833µs 2023-10-21T04:09:31.183+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 199.868µs 2023-10-21T04:09:31.183+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11160011617777336020 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:31.183+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11160011617777336020, indexDefn DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:31.183+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 47.741933ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:31.183+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11160011617777336020 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:31.184+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:31.184+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1054714 memSz 772 2023-10-21T04:09:31.184+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 1000 metaSz: 2492 rp [1] sn: 2 count: 1000 metaSz: 2557 2023-10-21T04:09:31.185+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 54942 recovery UsedSpace 22335 2023-10-21T04:09:31.190+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.190+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.193+05:30 [Info] switched currmeta from 211 -> 212 force true 2023-10-21T04:09:31.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex 2023-10-21T04:09:31.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:31.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:31.207+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex 2023-10-21T04:09:31.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index/mainIndex started 2023-10-21T04:09:31.208+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:31.209+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11160011617777336020 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:31.209+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:31.263+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11160011617777336020 PartitionId 0 Warmup took 79.507346ms 2023-10-21T04:09:31.266+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11160011617777336020 partitionId 0 WriterThreads 4 2023-10-21T04:09:31.266+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11160011617777336020 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:31.266+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11160011617777336020 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_11160011617777336020_0.index Index: 11160011617777336020 Partition: 0 2023-10-21T04:09:31.266+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11160011617777336020, shardIdMap: map[0:[10563635432419104314]] err: 2023-10-21T04:09:31.266+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11160011617777336020, partnShardIdMap: map[0:[10563635432419104314]], err: 2023-10-21T04:09:31.266+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:31.267+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11160011617777336020 2023-10-21T04:09:31.268+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:31.268+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:31.268+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:31.268+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:31.268+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:31.269+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11160011617777336020 Partitions [0] 2023-10-21T04:09:31.269+05:30 [Info] StorageMgr::openSnapshot IndexInst:11160011617777336020 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:31.291+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.563146ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:31.291+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 74.864µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:31.301+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.301+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.304+05:30 [Info] switched currmeta from 212 -> 213 force true 2023-10-21T04:09:31.425+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.425+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.428+05:30 [Info] switched currmeta from 213 -> 214 force true 2023-10-21T04:09:31.452+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.79939378754759, throttleDelayMs (new, old, change): (1, 6, -5) 2023-10-21T04:09:31.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.456+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.470+05:30 [Info] switched currmeta from 214 -> 215 force true 2023-10-21T04:09:31.500+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.500+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.508+05:30 [Info] switched currmeta from 215 -> 216 force true 2023-10-21T04:09:31.580+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8880461853743568138:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:31.580+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8880461853743568138 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007785ec0 2023-10-21T04:09:31.580+05:30 [Info] handleUpdateIndexRState: Index instance 8880461853743568138 rstate moved to ACTIVE 2023-10-21T04:09:31.587+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.104328ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:31.588+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 8857269605176042330, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:31.588+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 8857269605176042330, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:31.602+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.602+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.612+05:30 [Info] switched currmeta from 216 -> 217 force true 2023-10-21T04:09:31.634+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17729593977321451597,realInstId: 0, indexDefn DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:31.656+05:30 [Info] Indexer::handleRecoverIndex InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:31.661+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 5.155158ms 2023-10-21T04:09:31.662+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.662+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.662+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17729593977321451597 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:31.662+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17729593977321451597, indexDefn DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:31.664+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 76.011136ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:31.663+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 6.53443ms 2023-10-21T04:09:31.663+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17729593977321451597 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:31.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:31.665+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1055104 memSz 596 2023-10-21T04:09:31.666+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 14 count: 1000 metaSz: 2552 rp [1] sn: 15 count: 1000 metaSz: 2551 2023-10-21T04:09:31.667+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 55671 recovery UsedSpace 38516 2023-10-21T04:09:31.671+05:30 [Info] switched currmeta from 217 -> 218 force true 2023-10-21T04:09:31.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex 2023-10-21T04:09:31.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:31.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:31.674+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex 2023-10-21T04:09:31.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index/mainIndex started 2023-10-21T04:09:31.674+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17729593977321451597 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:31.674+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:31.674+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:31.689+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17729593977321451597 PartitionId 0 Warmup took 24.684272ms 2023-10-21T04:09:31.690+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17729593977321451597 partitionId 0 WriterThreads 4 2023-10-21T04:09:31.690+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17729593977321451597 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17729593977321451597_0.index Index: 17729593977321451597 Partition: 0 2023-10-21T04:09:31.690+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17729593977321451597, shardIdMap: map[0:[6611724767191156837]] err: 2023-10-21T04:09:31.690+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17729593977321451597, partnShardIdMap: map[0:[6611724767191156837]], err: 2023-10-21T04:09:31.690+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:31.690+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17729593977321451597 2023-10-21T04:09:31.690+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:31.690+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:31.690+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:31.691+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:31.691+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:31.691+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17729593977321451597 Partitions [0] 2023-10-21T04:09:31.692+05:30 [Info] StorageMgr::openSnapshot IndexInst:17729593977321451597 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:31.703+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.41976ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:31.706+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 363.989µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:31.723+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.723+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.729+05:30 [Info] switched currmeta from 218 -> 219 force true 2023-10-21T04:09:31.986+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:31.986+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:31.998+05:30 [Info] switched currmeta from 219 -> 222 force true 2023-10-21T04:09:32.013+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.013+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.015+05:30 [Info] switched currmeta from 222 -> 222 force true 2023-10-21T04:09:32.191+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2170849387178940761:INDEX_STATE_RECOVERED 4004220452657849790:INDEX_STATE_RECOVERED 11160011617777336020:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:32.191+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [12140458060295645238 14026476026078401783 5103684059183126577] belonging to collectionId: d, ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4. Initiating build 2023-10-21T04:09:32.219+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.220+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.257+05:30 [Info] switched currmeta from 222 -> 224 force true 2023-10-21T04:09:32.258+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [4004220452657849790 2170849387178940761 11160011617777336020] &{2} 2023-10-21T04:09:32.271+05:30 [Info] Indexer::handleBuildRecoveredIndexes [4004220452657849790 2170849387178940761 11160011617777336020] 2023-10-21T04:09:32.274+05:30 [Info] bulkUpdateRState: Index instance 4004220452657849790 rstate moved to PENDING 2023-10-21T04:09:32.271+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.277+05:30 [Info] bulkUpdateRState: Index instance 2170849387178940761 rstate moved to PENDING 2023-10-21T04:09:32.295+05:30 [Info] bulkUpdateRState: Index instance 11160011617777336020 rstate moved to PENDING 2023-10-21T04:09:32.289+05:30 [Info] switched currmeta from 224 -> 225 force true 2023-10-21T04:09:32.297+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [4004220452657849790 2170849387178940761 11160011617777336020] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:32.301+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:32.302+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:32.302+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:32.323+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:09:32.323+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.323+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:32.323+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:32.324+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:09:32.324+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:09:32.326+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:32.326+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:32.326+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:09:32.326+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.326+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:32.326+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c2% reset bucket state to NIL 2023-10-21T04:09:32.326+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:09:32.326+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% SessionId 85997 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:32.326+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.326+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:32.327+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c2% Added for Stream INIT_STREAM 2023-10-21T04:09:32.327+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_1:_default:c2%:1]] 2023-10-21T04:09:32.327+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4004220452657849790 stream INIT_STREAM keyspaceId bucket_1:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:32.327+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_1:_default:c2%:2]] 2023-10-21T04:09:32.328+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 11160011617777336020 stream INIT_STREAM keyspaceId bucket_1:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:32.328+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_1:_default:c2%:3]] 2023-10-21T04:09:32.328+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 2170849387178940761 stream INIT_STREAM keyspaceId bucket_1:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:32.328+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:09:32.328+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 85997 CollectionId: d CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:09:32.330+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:09:32.331+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:09:32.334+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:32.336+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 65.417902ms 2023-10-21T04:09:32.337+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 65.872665ms 2023-10-21T04:09:32.338+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c2% Check pending stream merge. 2023-10-21T04:09:32.338+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [4004220452657849790 2170849387178940761 11160011617777336020] 2023-10-21T04:09:32.338+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 146.539525ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:32.341+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[2170849387178940761:true 4004220452657849790:true 11160011617777336020:true] 2023-10-21T04:09:32.343+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c2% Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:09:32.350+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.265751ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:32.373+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.693449ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:32.388+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59002" doAuth() ... 2023-10-21T04:09:32.389+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59002" auth successful 2023-10-21T04:09:32.390+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.391117ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:32.391+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:59002" +1 2023-10-21T04:09:32.418+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.418+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.436+05:30 [Info] switched currmeta from 225 -> 229 force true 2023-10-21T04:09:32.459+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.773924926814179, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:09:32.467+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.467+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.474+05:30 [Info] switched currmeta from 229 -> 229 force true 2023-10-21T04:09:32.552+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.552+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.569+05:30 [Info] switched currmeta from 229 -> 232 force true 2023-10-21T04:09:32.582+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.583+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.587+05:30 [Info] switched currmeta from 232 -> 232 force true 2023-10-21T04:09:32.689+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[80911002600697866:INDEX_STATE_RECOVERED 11400459767051699632:INDEX_STATE_RECOVERED 17729593977321451597:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:32.690+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [12027054384338714804 16162012140118277065 8857269605176042330] belonging to collectionId: 0, ttid: ShardToken80:53:e0:a8:fa:c4:c4:85. Initiating build 2023-10-21T04:09:32.690+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.691+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.694+05:30 [Info] switched currmeta from 232 -> 233 force true 2023-10-21T04:09:32.748+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.748+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.757+05:30 [Info] switched currmeta from 233 -> 235 force true 2023-10-21T04:09:32.807+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [11400459767051699632 80911002600697866 17729593977321451597] &{2} 2023-10-21T04:09:32.813+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.821+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.820+05:30 [Info] Indexer::handleBuildRecoveredIndexes [11400459767051699632 80911002600697866 17729593977321451597] 2023-10-21T04:09:32.824+05:30 [Info] bulkUpdateRState: Index instance 11400459767051699632 rstate moved to PENDING 2023-10-21T04:09:32.825+05:30 [Info] bulkUpdateRState: Index instance 80911002600697866 rstate moved to PENDING 2023-10-21T04:09:32.827+05:30 [Info] bulkUpdateRState: Index instance 17729593977321451597 rstate moved to PENDING 2023-10-21T04:09:32.825+05:30 [Info] switched currmeta from 235 -> 237 force true 2023-10-21T04:09:32.829+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [11400459767051699632 80911002600697866 17729593977321451597] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:32.833+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.833+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.836+05:30 [Info] switched currmeta from 237 -> 237 force true 2023-10-21T04:09:32.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:32.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:32.873+05:30 [Info] switched currmeta from 237 -> 238 force true 2023-10-21T04:09:32.886+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:32.886+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:32.886+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:32.942+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2 2023-10-21T04:09:32.943+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.944+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:32.945+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.946+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:32.947+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:32.949+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:09:32.950+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:09:32.950+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:32.951+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:32.951+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_default:c2%:{0xc004792a80 }] 2023-10-21T04:09:32.951+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.951+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:32.951+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.951+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:32.951+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:09:32.952+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2 SessionId 26790 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:32.952+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_%2. OSO not supported for default scope/collection. 2023-10-21T04:09:32.952+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.952+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:32.952+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:32.952+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:32.952+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream INIT_STREAM 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_%2:1 bucket_1:_default:c2%:3]] 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 11400459767051699632 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_%2:2 bucket_1:_default:c2%:3]] 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17729593977321451597 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c2%:3]] 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 80911002600697866 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:32.952+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2 2023-10-21T04:09:32.953+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 26790 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:09:32.953+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0002c2240 } bucket_1:_default:c2%:{0xc004792a80 }] 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.958+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 4004220452657849790 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 2170849387178940761 RealInstId: 2170849387178940761 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 16726320877690809560 RealInstId: 16726320877690809560 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 11160011617777336020 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 17148287981272169168 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 11400459767051699632 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 80911002600697866 RealInstId: 80911002600697866 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9664682240473767939 RealInstId: 9664682240473767939 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 17729593977321451597 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 17128008689111072444 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 15710672812788661570 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:32.959+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:09:32.954+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:32.961+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 141.430297ms 2023-10-21T04:09:32.961+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 141.477367ms 2023-10-21T04:09:32.961+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [11400459767051699632 80911002600697866 17729593977321451597] 2023-10-21T04:09:32.961+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 265.995278ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:32.965+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:09:32.966+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2 Check pending stream merge. 2023-10-21T04:09:32.966+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[80911002600697866:true 11400459767051699632:true 17729593977321451597:true] 2023-10-21T04:09:32.981+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.717567ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:32.994+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.231358ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:33.025+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 31.224962ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:33.070+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:33.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:33.077+05:30 [Info] switched currmeta from 238 -> 242 force true 2023-10-21T04:09:33.091+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:33.091+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:33.099+05:30 [Info] switched currmeta from 242 -> 242 force true 2023-10-21T04:09:33.139+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:33.140+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:33.150+05:30 [Info] switched currmeta from 242 -> 243 force true 2023-10-21T04:09:33.445+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 0 186517626268899 55 85997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:33.445+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 1 178840478062605 54 85997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:33.448+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 3 215374859564337 35 85997 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:09:33.448+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 2 148075163102199 54 85997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:33.450+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 5 39552632604610 52 85997 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:33.450+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 4 28605411869539 58 85997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:33.453+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 7 191903564840858 63 85997 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:33.453+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 6 242327631330054 62 85997 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:33.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 8 50961953886801 45 85997 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:33.457+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 10 274410200699189 54 85997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:33.457+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 9 225280996447309 53 85997 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:33.458+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 11 66881023717063 51 85997 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:33.461+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 13 183148377680315 46 85997 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:33.461+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 12 4429827396609 42 85997 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:09:33.464+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 14 226814108379280 72 85997 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:09:33.466+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 15 127185710837163 50 85997 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:33.470+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 17 55290066231400 51 85997 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:33.471+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 16 257031483891250 55 85997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:33.472+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 18 242578104468179 63 85997 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:33.476+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 19 210306088158507 48 85997 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:09:33.480+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 20 230239359684242 50 85997 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:33.483+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 21 209771427122473 47 85997 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:33.497+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 22 260036488774725 51 85997 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:33.504+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 23 216462437971699 57 85997 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:33.515+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 27 184056281482149 49 85997 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:33.553+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 25 280874278672777 71 85997 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:09:33.556+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 26 219132834127088 52 85997 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:33.557+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 31 217381278987664 67 85997 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:33.557+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 30 99433130332778 58 85997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:33.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 29 45957426777107 57 85997 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:33.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 34 28417669987133 58 85997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:33.559+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 33 136958029391999 65 85997 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:09:33.560+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 43 146267938201125 55 85997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:33.561+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 38 188107826286720 60 85997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:33.561+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 41 259381390638966 58 85997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:33.563+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 42 252931432434966 46 85997 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:33.563+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 35 70213628416850 47 85997 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:33.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 39 222555662236724 67 85997 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:33.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 46 242994859770444 54 85997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:33.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 45 12301831552869 63 85997 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:33.570+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 51 200436537417141 54 85997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:33.570+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 53 204867593700407 56 85997 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:33.572+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 47 271074273345339 53 85997 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:33.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 49 215467668600630 34 85997 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:09:33.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 50 207367350302991 64 85997 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:33.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 55 170056388266420 70 85997 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:09:33.579+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 54 279362078165491 55 85997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:33.583+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 37 253494845579900 54 85997 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:33.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 59 125009072873642 53 85997 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:33.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 44 158545118983293 67 85997 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:33.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 57 129411666769864 55 85997 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:33.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 24 160961818008256 59 85997 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:33.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 32 82469662608686 60 85997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:33.604+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 40 5915864305121 60 85997 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:33.604+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 36 253845164956592 58 85997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:33.604+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 28 102578907139247 58 85997 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:33.604+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 58 16044440367455 53 85997 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:33.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 48 227378777358533 57 85997 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:33.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 52 188718367400378 56 85997 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:33.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 60 75566139929862 56 85997 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:33.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 56 264835102027079 52 85997 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:33.607+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 63 133748675726909 56 85997 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:33.617+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c2% InstanceIds [2170849387178940761 4004220452657849790 11160011617777336020] 2023-10-21T04:09:33.617+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% Cid d SessionId 85997 2023-10-21T04:09:33.617+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 85997 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 4004220452657849790 INDEX_STATE_CATCHUP 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 11160011617777336020 INDEX_STATE_CATCHUP 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 2170849387178940761 INDEX_STATE_CATCHUP 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 11160011617777336020 true 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 2170849387178940761 true 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 4004220452657849790 true 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 4004220452657849790 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% Session: 85997 BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c2%, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:09:33.617+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:09:33.618+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:09:33.618+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 85997 STREAM_ACTIVE 2023-10-21T04:09:33.618+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId: 85997 2023-10-21T04:09:33.618+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c2% 85997 2023-10-21T04:09:33.618+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:33.618+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:33.618+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:33.619+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:33.631+05:30 [Info] send progress stats to clients 2023-10-21T04:09:33.631+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c2% set minMergeTs 2023-10-21T04:09:33.632+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c2% StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:09:33.632+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c2%:3]] 2023-10-21T04:09:33.632+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:5] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c2%:3]] 2023-10-21T04:09:33.632+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:6] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c2%:3]] 2023-10-21T04:09:33.632+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 85997 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:09:33.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 62 10527297481171 57 85997 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:33.633+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [0=55,1=53,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=68,15=50,16=55,17=51,18=62,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=56,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=57,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=52,59=53,60=56,61=53,62=57,63=56] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:33.642+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.738496ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:33.660+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 61 106932309485550 53 85997 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:33.664+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:09:33.669+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:33.669+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:33.683+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 41.245078ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:33.699+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.619344ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:33.725+05:30 [Info] switched currmeta from 243 -> 249 force true 2023-10-21T04:09:33.749+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:33.751+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:33.772+05:30 [Info] switched currmeta from 249 -> 249 force true 2023-10-21T04:09:33.773+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 2023-10-21T04:09:33.773+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 85997 2023-10-21T04:09:33.774+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 85997 2023-10-21T04:09:33.774+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c2% set minMergeTs 2023-10-21T04:09:33.774+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:09:33.774+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 4004220452657849790 false 2023-10-21T04:09:33.774+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 11160011617777336020 false 2023-10-21T04:09:33.775+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 2170849387178940761 false 2023-10-21T04:09:33.775+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c2% 4004220452657849790 2023-10-21T04:09:33.775+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c2% 11160011617777336020 2023-10-21T04:09:33.775+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c2% 2170849387178940761 2023-10-21T04:09:33.777+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c2%. 2023-10-21T04:09:33.777+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 85997 2023-10-21T04:09:33.850+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:33.859+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:33.859+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:33.873+05:30 [Info] switched currmeta from 249 -> 249 force true 2023-10-21T04:09:34.050+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.050+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.062+05:30 [Info] switched currmeta from 249 -> 251 force true 2023-10-21T04:09:34.082+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.082+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.093+05:30 [Info] switched currmeta from 251 -> 251 force true 2023-10-21T04:09:34.133+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.133+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.139+05:30 [Info] switched currmeta from 251 -> 253 force true 2023-10-21T04:09:34.162+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.162+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.166+05:30 [Info] switched currmeta from 253 -> 253 force true 2023-10-21T04:09:34.243+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.243+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.246+05:30 [Info] switched currmeta from 253 -> 254 force true 2023-10-21T04:09:34.264+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.264+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.269+05:30 [Info] switched currmeta from 254 -> 255 force true 2023-10-21T04:09:34.300+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.300+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.318+05:30 [Info] switched currmeta from 255 -> 256 force true 2023-10-21T04:09:34.449+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:34.449+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 6] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 4] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:34.450+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 7] 2023-10-21T04:09:34.451+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 7] 2023-10-21T04:09:34.452+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9742586549820542, throttleDelayMs (new, old, change): (2, 0, 2) 2023-10-21T04:09:34.556+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 0 263268055970459 52 26790 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:34.560+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 2 35403626585217 50 26790 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:34.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 4 159113185399719 58 26790 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:34.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 1 176639880977228 58 26790 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:34.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 3 215749515705641 45 26790 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:34.567+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 6 261493429196201 60 26790 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:34.567+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 5 3874519919278 54 26790 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:34.572+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 8 264975169139761 56 26790 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:34.572+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 7 170288557775498 52 26790 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:34.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 10 206533515052028 50 26790 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:34.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 11 234758907181738 56 26790 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:34.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 9 236398693961229 62 26790 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:34.579+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 12 116104661063478 55 26790 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:34.581+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 14 140236896721046 60 26790 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:34.581+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 13 102350060311343 53 26790 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:34.582+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 15 162158720181435 67 26790 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:34.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 16 200752780087280 51 26790 172.31.5.112:9000. HWT [49-51,51]. 2023-10-21T04:09:34.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 17 98820809009123 44 26790 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:09:34.589+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 19 208836052671833 45 26790 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:34.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 20 92956357968546 50 26790 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:34.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 18 207134287529256 46 26790 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:34.594+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 22 192215088742243 55 26790 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:34.594+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 21 27979933591532 54 26790 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:34.596+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 24 271667630317209 52 26790 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:34.597+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 23 154934968850942 51 26790 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:34.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 25 168740244208559 59 26790 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:34.600+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 26 120989102157727 59 26790 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:34.602+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 27 5715487545729 54 26790 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:34.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 28 236953356346174 54 26790 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:34.606+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 29 130231725986856 49 26790 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:34.609+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 30 60545336618717 46 26790 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:34.611+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 31 73432771436458 60 26790 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:34.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 32 67362620867705 67 26790 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:34.618+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 33 278426329668882 54 26790 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:34.618+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c2% StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:09:34.619+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c2% 2023-10-21T04:09:34.619+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 4004220452657849790 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% SessionId: 85997, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:34.619+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 4004220452657849790 INDEX_STATE_ACTIVE 2023-10-21T04:09:34.619+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 11160011617777336020 INDEX_STATE_ACTIVE 2023-10-21T04:09:34.619+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 2170849387178940761 INDEX_STATE_ACTIVE 2023-10-21T04:09:34.619+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% State Changed to INACTIVE 2023-10-21T04:09:34.619+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:09:34.619+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:09:34.620+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:34.620+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId: 85997 2023-10-21T04:09:34.623+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:34.624+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:34.624+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2170849387178940761 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4004220452657849790 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 11160011617777336020 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:34.624+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 85997 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:34.625+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4004220452657849790 from stream INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:09:34.625+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 11160011617777336020 from stream INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:09:34.625+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 2170849387178940761 from stream INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:09:34.625+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:09:34.626+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:09:34.626+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:34.626+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 85997 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:34.627+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0002c2240 }] 2023-10-21T04:09:34.644+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:34.644+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:34.649+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:34.649+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:34.650+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:34.650+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:34.650+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:34.652+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:34.652+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:09:34.653+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:34.654+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId 85997 2023-10-21T04:09:34.654+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 34 51776324008509 60 26790 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:34.655+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c2%] 2023-10-21T04:09:34.656+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 35 24368176313355 50 26790 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:34.656+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 37 83647976044922 59 26790 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:34.656+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 36 43594649874772 52 26790 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:34.658+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 38 181267443431019 53 26790 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:34.658+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 39 100659545510465 49 26790 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:09:34.659+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 40 153478150002150 68 26790 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:09:34.660+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 41 116785907755771 55 26790 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:34.660+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 42 45479338375128 55 26790 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:34.660+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.592287ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:34.666+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 43 249919763471169 64 26790 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:34.671+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.742621ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:34.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 45 49796766757346 58 26790 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:34.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 44 166675095997741 57 26790 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:34.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 46 124194574848641 54 26790 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:34.687+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.687+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.694+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.208278ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:34.716+05:30 [Info] switched currmeta from 256 -> 259 force true 2023-10-21T04:09:34.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 54 24856128127173 60 26790 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:34.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 47 149022976193937 55 26790 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:34.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 48 19274425379955 55 26790 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:34.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 53 138935694816848 61 26790 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:34.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 50 199728167534433 66 26790 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:09:34.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 49 184487117398647 55 26790 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:34.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 52 158076890855634 54 26790 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:34.740+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 58 103336286678362 46 26790 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:34.749+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 56 29071415952852 61 26790 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:34.749+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 60 56626109228904 46 26790 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:34.810+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2 InstanceIds [2170849387178940761 4004220452657849790 11160011617777336020 11400459767051699632 17729593977321451597 80911002600697866] 2023-10-21T04:09:34.810+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2 Cid 0 SessionId 26790 2023-10-21T04:09:34.812+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c2%] 2023-10-21T04:09:34.812+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 85997 2023-10-21T04:09:34.810+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 26790 2023-10-21T04:09:34.827+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:09:34.827+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:09:34.828+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 11400459767051699632 INDEX_STATE_CATCHUP 2023-10-21T04:09:34.828+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 17729593977321451597 INDEX_STATE_CATCHUP 2023-10-21T04:09:34.828+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 80911002600697866 INDEX_STATE_CATCHUP 2023-10-21T04:09:34.828+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 11400459767051699632 true 2023-10-21T04:09:34.828+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 17729593977321451597 true 2023-10-21T04:09:34.830+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 80911002600697866 true 2023-10-21T04:09:34.830+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 17729593977321451597 Stream: INIT_STREAM KeyspaceId: bucket_%2 Session: 26790 BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:09:34.831+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:09:34.831+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:09:34.831+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 26790 STREAM_ACTIVE 2023-10-21T04:09:34.831+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 51 196777743741606 51 26790 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:34.832+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 62 177105814270799 58 26790 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:34.832+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 61 119871837460216 39 26790 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:09:34.832+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 57 191719959781498 63 26790 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:34.832+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 85997 2023-10-21T04:09:34.833+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:09:34.833+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 8 50961953886801 45 2023-10-21T04:09:34.833+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.833+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 0 186517626268899 55 2023-10-21T04:09:34.833+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.833+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 1 178840478062605 54 2023-10-21T04:09:34.840+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.842+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 2 148075163102199 54 2023-10-21T04:09:34.842+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.842+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 4 28605411869539 58 2023-10-21T04:09:34.842+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.842+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 5 39552632604610 52 2023-10-21T04:09:34.843+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.843+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 59 140942236111856 63 26790 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:34.843+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 55 61798873988101 67 26790 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:34.843+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 3 215374859564337 35 2023-10-21T04:09:34.843+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.843+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 6 242327631330054 62 2023-10-21T04:09:34.843+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.843+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 7 191903564840858 63 2023-10-21T04:09:34.844+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.844+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [0=19,1=19,2=19,3=18,4=19,5=19,6=25,7=19,8=25,9=26,10=19,11=23,12=22,13=22,14=27,15=26,16=20,17=12,18=21,19=17,20=20,21=19,22=23,23=19,24=15,25=22,26=22,27=29,28=18,29=19,30=20,31=22,32=26,33=22,34=26,35=18,36=21,37=20,38=25,39=19,40=27,41=25,42=26,43=26,44=18,45=18,46=25,47=20,48=21,49=27,50=25,51=20,52=19,53=25,54=24,55=28,56=21,57=24,58=20,59=26,60=19,61=14,62=23,63=21] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:34.844+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 36 253845164956592 58 2023-10-21T04:09:34.848+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.849+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 35 70213628416850 47 2023-10-21T04:09:34.849+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 39 222555662236724 67 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 47 271074273345339 53 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 15 127185710837163 50 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 50 207367350302991 64 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 18 242578104468179 63 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 21 209771427122473 47 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 53 204867593700407 56 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.850+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 30 99433130332778 58 2023-10-21T04:09:34.850+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.851+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 62 10527297481171 57 2023-10-21T04:09:34.851+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.851+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 51 200436537417141 54 2023-10-21T04:09:34.851+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.851+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 19 210306088158507 48 2023-10-21T04:09:34.851+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.851+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 54 279362078165491 55 2023-10-21T04:09:34.852+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.852+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 22 260036488774725 51 2023-10-21T04:09:34.853+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.853+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 26 219132834127088 52 2023-10-21T04:09:34.854+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.854+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 58 16044440367455 53 2023-10-21T04:09:34.854+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.855+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 28 102578907139247 58 2023-10-21T04:09:34.855+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.855+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 60 75566139929862 56 2023-10-21T04:09:34.856+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.856+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 27 184056281482149 49 2023-10-21T04:09:34.861+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.858+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 59 125009072873642 53 2023-10-21T04:09:34.864+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.864+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 16 257031483891250 55 2023-10-21T04:09:34.864+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.865+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 48 227378777358533 57 2023-10-21T04:09:34.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.866+05:30 [Info] switched currmeta from 259 -> 261 force true 2023-10-21T04:09:34.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 57 129411666769864 55 2023-10-21T04:09:34.868+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.868+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 25 280874278672777 71 2023-10-21T04:09:34.869+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.869+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 12 4429827396609 42 2023-10-21T04:09:34.872+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.889+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 44 158545118983293 67 2023-10-21T04:09:34.889+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.889+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 42 252931432434966 46 2023-10-21T04:09:34.890+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.890+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 10 274410200699189 54 2023-10-21T04:09:34.891+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.891+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 49 215467668600630 34 2023-10-21T04:09:34.892+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.892+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 17 55290066231400 51 2023-10-21T04:09:34.893+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.893+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 43 146267938201125 55 2023-10-21T04:09:34.894+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.894+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 11 66881023717063 51 2023-10-21T04:09:34.895+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.895+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 33 136958029391999 65 2023-10-21T04:09:34.895+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.896+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 40 5915864305121 60 2023-10-21T04:09:34.896+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.898+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 41 259381390638966 58 2023-10-21T04:09:34.898+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.898+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 9 225280996447309 53 2023-10-21T04:09:34.899+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.899+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 63 133748675726909 56 2023-10-21T04:09:34.899+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.899+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 63 252537041863196 73 26790 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:09:34.897+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.901+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.905+05:30 [Info] switched currmeta from 261 -> 262 force true 2023-10-21T04:09:34.912+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.913+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.899+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 31 217381278987664 67 2023-10-21T04:09:34.914+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.918+05:30 [Info] switched currmeta from 262 -> 262 force true 2023-10-21T04:09:34.918+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 46 242994859770444 54 2023-10-21T04:09:34.918+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.918+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 14 226814108379280 72 2023-10-21T04:09:34.918+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.918+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 24 160961818008256 59 2023-10-21T04:09:34.918+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.919+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 56 264835102027079 52 2023-10-21T04:09:34.919+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.919+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 29 45957426777107 57 2023-10-21T04:09:34.920+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.921+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 61 106932309485550 53 2023-10-21T04:09:34.922+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.922+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 55 170056388266420 70 2023-10-21T04:09:34.923+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.923+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 23 216462437971699 57 2023-10-21T04:09:34.924+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.924+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 13 183148377680315 46 2023-10-21T04:09:34.925+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.926+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 45 12301831552869 63 2023-10-21T04:09:34.927+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.928+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 52 188718367400378 56 2023-10-21T04:09:34.930+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.934+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 20 230239359684242 50 2023-10-21T04:09:34.937+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.940+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 38 188107826286720 60 2023-10-21T04:09:34.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.940+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 37 253494845579900 54 2023-10-21T04:09:34.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.941+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.951+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.948+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 32 82469662608686 60 2023-10-21T04:09:34.968+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.967+05:30 [Info] switched currmeta from 262 -> 263 force true 2023-10-21T04:09:34.970+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c2% 34 28417669987133 58 2023-10-21T04:09:34.971+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:09:34.971+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2 Stream: INIT_STREAM SessionId: 26790 2023-10-21T04:09:34.972+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2 26790 2023-10-21T04:09:34.985+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:34.985+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:34.987+05:30 [Info] switched currmeta from 263 -> 265 force true 2023-10-21T04:09:34.993+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:34.994+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:34.994+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:34.995+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:34.996+05:30 [Info] send progress stats to clients 2023-10-21T04:09:34.996+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2 set minMergeTs 2023-10-21T04:09:34.996+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:09:34.996+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:6] INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:09:34.996+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:6] INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:09:34.997+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:6] INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:09:34.997+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 26790 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:09:35.013+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:09:35.022+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.023+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.028+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 2023-10-21T04:09:35.029+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2 SessionId 26790 2023-10-21T04:09:35.034+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 26790 2023-10-21T04:09:35.034+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2 set minMergeTs 2023-10-21T04:09:35.035+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:09:35.036+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 11400459767051699632 false 2023-10-21T04:09:35.035+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 34.799373ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.037+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 17729593977321451597 false 2023-10-21T04:09:35.039+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 80911002600697866 false 2023-10-21T04:09:35.039+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2 11400459767051699632 2023-10-21T04:09:35.040+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2 17729593977321451597 2023-10-21T04:09:35.041+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2 80911002600697866 2023-10-21T04:09:35.042+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2. 2023-10-21T04:09:35.045+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 26790 2023-10-21T04:09:35.065+05:30 [Info] switched currmeta from 265 -> 266 force true 2023-10-21T04:09:35.106+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 67.700724ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.134+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 28.132107ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.187+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.187+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.191+05:30 [Info] switched currmeta from 266 -> 269 force true 2023-10-21T04:09:35.253+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.253+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.274+05:30 [Info] switched currmeta from 269 -> 271 force true 2023-10-21T04:09:35.306+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.306+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.314+05:30 [Info] switched currmeta from 271 -> 272 force true 2023-10-21T04:09:35.320+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.320+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.323+05:30 [Info] switched currmeta from 272 -> 272 force true 2023-10-21T04:09:35.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.415+05:30 [Info] switched currmeta from 272 -> 273 force true 2023-10-21T04:09:35.417+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4004220452657849790 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006d6c5a0 2023-10-21T04:09:35.418+05:30 [Info] handleUpdateIndexRState: Index instance 4004220452657849790 rstate moved to ACTIVE 2023-10-21T04:09:35.424+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.821389ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.424+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11160011617777336020 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007e7ea80 2023-10-21T04:09:35.424+05:30 [Info] handleUpdateIndexRState: Index instance 11160011617777336020 rstate moved to ACTIVE 2023-10-21T04:09:35.424+05:30 [Info] Indexer::handleMergePartition Source 579097042483677314 Target 2170849387178940761 2023-10-21T04:09:35.424+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2170849387178940761 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00aa649c0 2023-10-21T04:09:35.424+05:30 [Info] MergePartition: sent async request to update index instance 2170849387178940761 rstate moved to ACTIVE 2023-10-21T04:09:35.433+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.04468ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.436+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.438+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.439+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 13258659765240018889. Retrying... 2023-10-21T04:09:35.439+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 7597064108789640455. Retrying... 2023-10-21T04:09:35.439+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[2170849387178940761:true 4004220452657849790:true 11160011617777336020:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:35.442+05:30 [Info] switched currmeta from 273 -> 275 force true 2023-10-21T04:09:35.447+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.293258ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.447+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 579097042483677314, realInst: 2170849387178940761 2023-10-21T04:09:35.451+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 14292583894719865678, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:35.451+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 14292583894719865678, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:35.451+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.451+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.452+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9740092291326963, throttleDelayMs (new, old, change): (4, 2, 2) 2023-10-21T04:09:35.456+05:30 [Info] switched currmeta from 275 -> 276 force true 2023-10-21T04:09:35.463+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.463+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.529+05:30 [Info] switched currmeta from 276 -> 276 force true 2023-10-21T04:09:35.538+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3292093078969249751,realInstId: 0, indexDefn DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:35.539+05:30 [Info] Indexer::handleRecoverIndex InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:35.539+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 474.383µs 2023-10-21T04:09:35.539+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 637.65µs 2023-10-21T04:09:35.540+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3292093078969249751 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:35.540+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3292093078969249751 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:35.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:35.541+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1056200 memSz 2074 2023-10-21T04:09:35.541+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 16 count: 1000 metaSz: 2516 rp [1] sn: 17 count: 1000 metaSz: 2582 rp [2] sn: 18 count: 1000 metaSz: 2580 2023-10-21T04:09:35.541+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 62430 recovery UsedSpace 33070 2023-10-21T04:09:35.542+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3292093078969249751, indexDefn DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:35.542+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 91.725007ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:35.543+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1055980 memSz 900 2023-10-21T04:09:35.543+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 17 count: 1000 metaSz: 2582 rp [1] sn: 18 count: 1000 metaSz: 2580 2023-10-21T04:09:35.544+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 64190 recovery UsedSpace 39097 2023-10-21T04:09:35.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex 2023-10-21T04:09:35.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:35.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:35.554+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex 2023-10-21T04:09:35.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/docIndex started 2023-10-21T04:09:35.554+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:35.554+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:35.555+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.555+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.562+05:30 [Info] switched currmeta from 276 -> 277 force true 2023-10-21T04:09:35.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex 2023-10-21T04:09:35.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:35.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:35.570+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex 2023-10-21T04:09:35.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index/mainIndex started 2023-10-21T04:09:35.570+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3292093078969249751 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:35.570+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:35.571+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:35.600+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3292093078969249751 PartitionId 0 Warmup took 59.190872ms 2023-10-21T04:09:35.600+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3292093078969249751 partitionId 0 WriterThreads 4 2023-10-21T04:09:35.600+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3292093078969249751 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:35.601+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3292093078969249751 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3292093078969249751_0.index Index: 3292093078969249751 Partition: 0 2023-10-21T04:09:35.601+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3292093078969249751, shardIdMap: map[0:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:35.601+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3292093078969249751, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:35.601+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:35.609+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3292093078969249751 2023-10-21T04:09:35.610+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:35.610+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:35.610+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:35.610+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:35.610+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:35.611+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3292093078969249751 Partitions [0] 2023-10-21T04:09:35.618+05:30 [Info] StorageMgr::openSnapshot IndexInst:3292093078969249751 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:35.618+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:09:35.619+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.611394ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.619+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 149.96µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:35.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.639+05:30 [Info] switched currmeta from 277 -> 278 force true 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 8205593268191821683:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.720+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 8205593268191821683:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:35.832+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 80911002600697866 Stream: INIT_STREAM KeyspaceId: bucket_%2 SessionId: 26790, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 11400459767051699632 INDEX_STATE_ACTIVE 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 17729593977321451597 INDEX_STATE_ACTIVE 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 80911002600697866 INDEX_STATE_ACTIVE 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2 State Changed to INACTIVE 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:09:35.832+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:09:35.832+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 0 New 0 2023-10-21T04:09:35.832+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2 Stream: INIT_STREAM SessionId: 26790 2023-10-21T04:09:35.833+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:35.833+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:35.833+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 80911002600697866 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 11400459767051699632 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 17729593977321451597 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:35.834+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 26790 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:35.834+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 11400459767051699632 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:09:35.834+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17729593977321451597 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:09:35.834+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 80911002600697866 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:09:35.834+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:09:35.834+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:09:35.834+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:35.834+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 26790 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:35.834+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:09:35.834+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:35.834+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:35.835+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:09:35.835+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:59002: use of closed network connection 2023-10-21T04:09:35.835+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:59002" exit: read tcp4 127.0.0.1:9121->127.0.0.1:59002: use of closed network connection 2023-10-21T04:09:35.835+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59002" closed ! 2023-10-21T04:09:35.835+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:09:35.835+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:59002" already gone 2023-10-21T04:09:35.835+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:35.835+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:35.835+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:35.835+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:35.835+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:09:35.835+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:35.836+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2 Stream: INIT_STREAM SessionId 26790 2023-10-21T04:09:35.836+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2] 2023-10-21T04:09:35.848+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2]. Err genServer.closed 2023-10-21T04:09:35.848+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:09:35.848+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2] Treating genServer.closed As Success 2023-10-21T04:09:35.848+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2 SessionId 26790 2023-10-21T04:09:35.848+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 26790 2023-10-21T04:09:35.848+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:09:35.864+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 28.139539ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.901+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.433922ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.908+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.219802ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:35.929+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.930+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.941+05:30 [Info] switched currmeta from 278 -> 281 force true 2023-10-21T04:09:35.953+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:35.953+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:35.955+05:30 [Info] switched currmeta from 281 -> 281 force true 2023-10-21T04:09:36.076+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11400459767051699632 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005b721e0 2023-10-21T04:09:36.076+05:30 [Info] handleUpdateIndexRState: Index instance 11400459767051699632 rstate moved to ACTIVE 2023-10-21T04:09:36.082+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.270509ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:36.083+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17729593977321451597 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005b72300 2023-10-21T04:09:36.084+05:30 [Info] handleUpdateIndexRState: Index instance 17729593977321451597 rstate moved to ACTIVE 2023-10-21T04:09:36.088+05:30 [Info] Indexer::handleMergePartition Source 6377858590284099462 Target 80911002600697866 2023-10-21T04:09:36.092+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 80911002600697866 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006386a80 2023-10-21T04:09:36.100+05:30 [Info] MergePartition: sent async request to update index instance 80911002600697866 rstate moved to ACTIVE 2023-10-21T04:09:36.102+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.290003ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:36.106+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[80911002600697866:true 11400459767051699632:true 17729593977321451597:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:36.112+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.112+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.114+05:30 [Info] switched currmeta from 281 -> 283 force true 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7] 11160011617777336020:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7] 11160011617777336020:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7] 11160011617777336020:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.115+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2170849387178940761:[8 1 2 3 4 5 6 7] 4004220452657849790:[0] 11160011617777336020:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:36.123+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.355201ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:36.123+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 6377858590284099462, realInst: 80911002600697866 2023-10-21T04:09:36.126+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 3674078063353481907, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:36.126+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 3674078063353481907, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:36.131+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.131+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.133+05:30 [Info] switched currmeta from 283 -> 284 force true 2023-10-21T04:09:36.143+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.143+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.147+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:09:36.217+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.220+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.224+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14780363202971367008,realInstId: 0, indexDefn DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:36.225+05:30 [Info] Indexer::handleRecoverIndex InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:36.225+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 588.453µs 2023-10-21T04:09:36.225+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 637.213µs 2023-10-21T04:09:36.225+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14780363202971367008 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:36.225+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14780363202971367008 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:36.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:36.225+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 64 2023-10-21T04:09:36.225+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 10 count: 12 metaSz: 2547 rp [1] sn: 17 count: 21 metaSz: 2547 2023-10-21T04:09:36.226+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 30404 recovery UsedSpace 37596 2023-10-21T04:09:36.227+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14780363202971367008, indexDefn DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:36.227+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 101.414109ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:36.239+05:30 [Info] switched currmeta from 284 -> 287 force true 2023-10-21T04:09:36.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex 2023-10-21T04:09:36.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:36.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:36.252+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex 2023-10-21T04:09:36.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index/mainIndex started 2023-10-21T04:09:36.253+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14780363202971367008 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:36.253+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.253+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.271+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14780363202971367008 PartitionId 0 Warmup took 46.003513ms 2023-10-21T04:09:36.271+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14780363202971367008 partitionId 0 WriterThreads 4 2023-10-21T04:09:36.272+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14780363202971367008 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14780363202971367008_0.index Index: 14780363202971367008 Partition: 0 2023-10-21T04:09:36.272+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14780363202971367008, shardIdMap: map[0:[6611724767191156837]] err: 2023-10-21T04:09:36.272+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14780363202971367008, partnShardIdMap: map[0:[6611724767191156837]], err: 2023-10-21T04:09:36.273+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:36.274+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.274+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.275+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14780363202971367008 2023-10-21T04:09:36.275+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:36.276+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:36.276+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:36.277+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:36.277+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:36.277+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14780363202971367008 Partitions [0] 2023-10-21T04:09:36.278+05:30 [Info] StorageMgr::openSnapshot IndexInst:14780363202971367008 Partition:0 Attempting to open snapshot (SnapshotInfo: count:21 committed:false) 2023-10-21T04:09:36.278+05:30 [Info] switched currmeta from 287 -> 288 force true 2023-10-21T04:09:36.281+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.077915ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:36.281+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 69.939µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:36.317+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.317+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.323+05:30 [Info] switched currmeta from 288 -> 291 force true 2023-10-21T04:09:36.332+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.332+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.341+05:30 [Info] switched currmeta from 291 -> 291 force true 2023-10-21T04:09:36.381+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.381+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.387+05:30 [Info] switched currmeta from 291 -> 293 force true 2023-10-21T04:09:36.410+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.410+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.414+05:30 [Info] switched currmeta from 293 -> 294 force true 2023-10-21T04:09:36.453+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9167445325647677, throttleDelayMs (new, old, change): (1, 4, -3) 2023-10-21T04:09:36.475+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.477+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.493+05:30 [Info] switched currmeta from 294 -> 296 force true 2023-10-21T04:09:36.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.501+05:30 [Info] switched currmeta from 296 -> 296 force true 2023-10-21T04:09:36.570+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3292093078969249751:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:36.578+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 17656144409930763141, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:36.578+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 17656144409930763141, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:36.602+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 13953156762317267443,realInstId: 0, indexDefn DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:36.602+05:30 [Info] Indexer::handleRecoverIndex InstId: 13953156762317267443 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:36.602+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 340.555µs 2023-10-21T04:09:36.603+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 385.129µs 2023-10-21T04:09:36.603+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:36.603+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 6 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:36.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.603+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.604+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:36.604+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 13953156762317267443, indexDefn DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:36.604+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 26.697232ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:36.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.605+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.605+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:36.616+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.616+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:36.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.621+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:36.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex started 2023-10-21T04:09:36.621+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.621+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.624+05:30 [Info] switched currmeta from 296 -> 298 force true 2023-10-21T04:09:36.638+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:36.638+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.646+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex started 2023-10-21T04:09:36.646+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:36.646+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:36.646+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.646+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.646+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex] 2023-10-21T04:09:36.646+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:36.646+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex] 2023-10-21T04:09:36.646+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex] 2023-10-21T04:09:36.646+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex stopped 2023-10-21T04:09:36.646+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.646+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.646+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.646+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.647+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.647+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.651+05:30 [Info] switched currmeta from 298 -> 298 force true 2023-10-21T04:09:36.652+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex] 2023-10-21T04:09:36.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex closed 2023-10-21T04:09:36.652+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:36.652+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.653+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.653+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex] 2023-10-21T04:09:36.653+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:36.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex] 2023-10-21T04:09:36.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex] 2023-10-21T04:09:36.653+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:36.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex stopped 2023-10-21T04:09:36.653+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.653+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.653+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.653+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.657+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex] 2023-10-21T04:09:36.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex closed 2023-10-21T04:09:36.658+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:36.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex ... 2023-10-21T04:09:36.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex ... 2023-10-21T04:09:36.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex sucessfully destroyed 2023-10-21T04:09:36.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex ... 2023-10-21T04:09:36.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex ... 2023-10-21T04:09:36.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index destroyed 2023-10-21T04:09:36.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.667+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.668+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.677+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.677+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 82 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:36.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.689+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:36.689+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex 2023-10-21T04:09:36.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/mainIndex started 2023-10-21T04:09:36.689+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.689+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 73 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:36.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.692+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:36.692+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex 2023-10-21T04:09:36.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index/docIndex started 2023-10-21T04:09:36.692+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 6 Warmup took 89.529849ms 2023-10-21T04:09:36.693+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.693+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.694+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 6 WriterThreads 1 2023-10-21T04:09:36.694+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 6 Stopped Billing WUs 2023-10-21T04:09:36.694+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_6.index Index: 13953156762317267443 Partition: 6 2023-10-21T04:09:36.694+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:36.695+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 7 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:36.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.696+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.696+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:36.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.698+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.699+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:36.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:36.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.705+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:36.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex started 2023-10-21T04:09:36.705+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.705+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:36.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.711+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:36.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex started 2023-10-21T04:09:36.711+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:36.711+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:36.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.711+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.711+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex] 2023-10-21T04:09:36.712+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:36.712+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex] 2023-10-21T04:09:36.712+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex] 2023-10-21T04:09:36.712+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:36.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex stopped 2023-10-21T04:09:36.712+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.712+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.712+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.712+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9010726441942672253:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9010726441942672253:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9010726441942672253:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9010726441942672253:[4 5 6 7 8 1 2 3] 11675990026580235111:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:36.723+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex] 2023-10-21T04:09:36.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex closed 2023-10-21T04:09:36.724+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:36.724+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.724+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.725+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex] 2023-10-21T04:09:36.725+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:36.725+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex] 2023-10-21T04:09:36.726+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex] 2023-10-21T04:09:36.726+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:36.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex stopped 2023-10-21T04:09:36.726+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.727+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.727+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.727+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.740+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex] 2023-10-21T04:09:36.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex closed 2023-10-21T04:09:36.741+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:36.741+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex ... 2023-10-21T04:09:36.741+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex ... 2023-10-21T04:09:36.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex sucessfully destroyed 2023-10-21T04:09:36.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex ... 2023-10-21T04:09:36.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex ... 2023-10-21T04:09:36.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index destroyed 2023-10-21T04:09:36.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.755+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.758+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.759+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 74 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:36.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.766+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:36.767+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex 2023-10-21T04:09:36.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 83 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:36.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/docIndex started 2023-10-21T04:09:36.768+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.768+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:36.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex 2023-10-21T04:09:36.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index/mainIndex started 2023-10-21T04:09:36.769+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 7 Warmup took 74.121742ms 2023-10-21T04:09:36.770+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.770+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.770+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 7 WriterThreads 1 2023-10-21T04:09:36.771+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 7 Stopped Billing WUs 2023-10-21T04:09:36.771+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_7.index Index: 13953156762317267443 Partition: 7 2023-10-21T04:09:36.771+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:36.771+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 8 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:36.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.772+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.773+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:36.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.774+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.775+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:36.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:36.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.778+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:36.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex started 2023-10-21T04:09:36.778+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.778+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:36.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.781+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:36.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex started 2023-10-21T04:09:36.782+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:36.782+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:36.782+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.782+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.783+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex] 2023-10-21T04:09:36.783+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:36.783+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex] 2023-10-21T04:09:36.783+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex] 2023-10-21T04:09:36.783+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:36.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex stopped 2023-10-21T04:09:36.784+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.784+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.784+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.784+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.788+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.788+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex] 2023-10-21T04:09:36.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex closed 2023-10-21T04:09:36.789+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:36.790+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.790+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.790+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex] 2023-10-21T04:09:36.790+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:36.790+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex] 2023-10-21T04:09:36.791+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex] 2023-10-21T04:09:36.791+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:36.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex stopped 2023-10-21T04:09:36.791+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.791+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.791+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.791+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.796+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex] 2023-10-21T04:09:36.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex closed 2023-10-21T04:09:36.797+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:36.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex ... 2023-10-21T04:09:36.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex ... 2023-10-21T04:09:36.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex sucessfully destroyed 2023-10-21T04:09:36.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex ... 2023-10-21T04:09:36.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex ... 2023-10-21T04:09:36.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index destroyed 2023-10-21T04:09:36.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.808+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.808+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.811+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.811+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 75 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:36.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.818+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:36.818+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex 2023-10-21T04:09:36.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/docIndex started 2023-10-21T04:09:36.818+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.818+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 84 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:36.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.827+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:36.827+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex 2023-10-21T04:09:36.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index/mainIndex started 2023-10-21T04:09:36.828+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 8 Warmup took 56.034826ms 2023-10-21T04:09:36.828+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 8 WriterThreads 1 2023-10-21T04:09:36.828+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 8 Stopped Billing WUs 2023-10-21T04:09:36.828+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_8.index Index: 13953156762317267443 Partition: 8 2023-10-21T04:09:36.828+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:36.828+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 1 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:36.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.828+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.829+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.829+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.829+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.829+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:36.830+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:36.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:36.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.837+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:36.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex started 2023-10-21T04:09:36.837+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.837+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:36.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.838+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:36.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex started 2023-10-21T04:09:36.838+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:36.838+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex] 2023-10-21T04:09:36.838+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:36.838+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex] 2023-10-21T04:09:36.838+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex] 2023-10-21T04:09:36.838+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:36.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex stopped 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.838+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex] 2023-10-21T04:09:36.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex closed 2023-10-21T04:09:36.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:36.847+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.847+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.847+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex] 2023-10-21T04:09:36.847+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:36.847+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex] 2023-10-21T04:09:36.847+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex] 2023-10-21T04:09:36.847+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:36.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex stopped 2023-10-21T04:09:36.848+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.848+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.848+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.848+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.859+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex] 2023-10-21T04:09:36.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex closed 2023-10-21T04:09:36.860+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:36.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex ... 2023-10-21T04:09:36.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex ... 2023-10-21T04:09:36.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex sucessfully destroyed 2023-10-21T04:09:36.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex ... 2023-10-21T04:09:36.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex ... 2023-10-21T04:09:36.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index destroyed 2023-10-21T04:09:36.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.874+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.874+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.878+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.878+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 76 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:36.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.883+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:36.883+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex 2023-10-21T04:09:36.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/docIndex started 2023-10-21T04:09:36.884+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.884+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 85 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:36.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.887+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:36.888+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex 2023-10-21T04:09:36.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index/mainIndex started 2023-10-21T04:09:36.888+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 1 Warmup took 59.460849ms 2023-10-21T04:09:36.888+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 1 WriterThreads 1 2023-10-21T04:09:36.888+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 1 Stopped Billing WUs 2023-10-21T04:09:36.888+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_1.index Index: 13953156762317267443 Partition: 1 2023-10-21T04:09:36.889+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:36.889+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 2 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:36.888+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.888+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.889+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.890+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.890+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:36.890+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:36.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:36.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.897+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:36.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex started 2023-10-21T04:09:36.897+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.897+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:36.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.898+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:36.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex started 2023-10-21T04:09:36.898+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:36.898+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex] 2023-10-21T04:09:36.898+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:36.898+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex] 2023-10-21T04:09:36.898+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex] 2023-10-21T04:09:36.898+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:36.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex stopped 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.898+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.902+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex] 2023-10-21T04:09:36.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex closed 2023-10-21T04:09:36.903+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:36.903+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.903+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex] 2023-10-21T04:09:36.903+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:36.903+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex] 2023-10-21T04:09:36.903+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex] 2023-10-21T04:09:36.903+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.903+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:36.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex stopped 2023-10-21T04:09:36.904+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.904+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.904+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.904+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.910+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex] 2023-10-21T04:09:36.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex closed 2023-10-21T04:09:36.910+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:36.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex ... 2023-10-21T04:09:36.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex ... 2023-10-21T04:09:36.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex sucessfully destroyed 2023-10-21T04:09:36.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex ... 2023-10-21T04:09:36.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex ... 2023-10-21T04:09:36.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index destroyed 2023-10-21T04:09:36.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.918+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.918+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.921+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.922+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 77 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:36.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.927+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:36.927+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex 2023-10-21T04:09:36.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/docIndex started 2023-10-21T04:09:36.927+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.927+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 86 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:36.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.928+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:36.928+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex 2023-10-21T04:09:36.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index/mainIndex started 2023-10-21T04:09:36.928+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 2 Warmup took 38.741619ms 2023-10-21T04:09:36.928+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 2 WriterThreads 1 2023-10-21T04:09:36.928+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 2 Stopped Billing WUs 2023-10-21T04:09:36.928+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_2.index Index: 13953156762317267443 Partition: 2 2023-10-21T04:09:36.928+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:36.928+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 3 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:36.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.929+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.929+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:36.929+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.929+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.930+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:36.930+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:36.937+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex started 2023-10-21T04:09:36.937+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.937+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:36.937+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex started 2023-10-21T04:09:36.937+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:36.937+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:36.937+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.937+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:36.937+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:36.937+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex] 2023-10-21T04:09:36.938+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:36.938+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex] 2023-10-21T04:09:36.938+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex] 2023-10-21T04:09:36.938+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:36.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex stopped 2023-10-21T04:09:36.938+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.938+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.938+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.938+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.938+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.949+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex] 2023-10-21T04:09:36.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex closed 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:36.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex] 2023-10-21T04:09:36.950+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:36.950+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex] 2023-10-21T04:09:36.950+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex] 2023-10-21T04:09:36.950+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:36.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex stopped 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:36.950+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:36.960+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:36.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex] 2023-10-21T04:09:36.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex closed 2023-10-21T04:09:36.961+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:36.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex ... 2023-10-21T04:09:36.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex ... 2023-10-21T04:09:36.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex sucessfully destroyed 2023-10-21T04:09:36.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex ... 2023-10-21T04:09:36.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex ... 2023-10-21T04:09:36.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index destroyed 2023-10-21T04:09:36.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:36.992+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.992+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:36.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:36.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:36.996+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:36.996+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:37.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 87 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:37.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:37.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:37.004+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:37.004+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex 2023-10-21T04:09:37.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/mainIndex started 2023-10-21T04:09:37.004+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.005+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 15710672812788661570 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 3841645022501780362 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 4004220452657849790 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 78 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:37.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:37.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:37.006+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:37.006+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex 2023-10-21T04:09:37.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index/docIndex started 2023-10-21T04:09:37.006+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 3 Warmup took 77.233719ms 2023-10-21T04:09:37.006+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 3 WriterThreads 1 2023-10-21T04:09:37.006+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 3 Stopped Billing WUs 2023-10-21T04:09:37.006+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_3.index Index: 13953156762317267443 Partition: 3 2023-10-21T04:09:37.006+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:37.006+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 4 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:37.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:37.006+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:37.007+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:37.005+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 3292093078969249751 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 2170849387178940761 RealInstId: 2170849387178940761 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 16726320877690809560 RealInstId: 16726320877690809560 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 11160011617777336020 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 17148287981272169168 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 11400459767051699632 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 80911002600697866 RealInstId: 80911002600697866 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9664682240473767939 RealInstId: 9664682240473767939 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 14780363202971367008 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 17729593977321451597 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 17128008689111072444 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:37.007+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:09:37.008+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.008+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:37.008+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:37.008+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:37.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:37.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:37.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:37.020+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:37.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex started 2023-10-21T04:09:37.020+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.020+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:37.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:37.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:37.021+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:37.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex started 2023-10-21T04:09:37.021+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.021+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.021+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:37.022+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:37.022+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:37.022+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:37.025+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:37.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex] 2023-10-21T04:09:37.025+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:37.025+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex] 2023-10-21T04:09:37.025+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex] 2023-10-21T04:09:37.025+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:37.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex stopped 2023-10-21T04:09:37.025+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:37.025+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:37.025+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:37.025+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:37.035+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:37.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex] 2023-10-21T04:09:37.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex closed 2023-10-21T04:09:37.035+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:37.035+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:37.035+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:37.035+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:37.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex] 2023-10-21T04:09:37.035+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:37.035+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex] 2023-10-21T04:09:37.035+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex] 2023-10-21T04:09:37.036+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:37.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex stopped 2023-10-21T04:09:37.036+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:37.036+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:37.036+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:37.036+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:37.041+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:37.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex] 2023-10-21T04:09:37.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex closed 2023-10-21T04:09:37.042+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:37.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex ... 2023-10-21T04:09:37.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex ... 2023-10-21T04:09:37.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex sucessfully destroyed 2023-10-21T04:09:37.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex ... 2023-10-21T04:09:37.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex ... 2023-10-21T04:09:37.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index destroyed 2023-10-21T04:09:37.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:37.053+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:37.053+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:37.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:37.057+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:37.057+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:37.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 79 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:37.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:37.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:37.060+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:37.060+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex 2023-10-21T04:09:37.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/docIndex started 2023-10-21T04:09:37.060+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.060+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 88 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:37.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:37.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:37.062+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:37.062+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex 2023-10-21T04:09:37.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index/mainIndex started 2023-10-21T04:09:37.063+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 4 Warmup took 56.487841ms 2023-10-21T04:09:37.063+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.063+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.063+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 4 WriterThreads 1 2023-10-21T04:09:37.063+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 4 Stopped Billing WUs 2023-10-21T04:09:37.064+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_4.index Index: 13953156762317267443 Partition: 4 2023-10-21T04:09:37.064+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13953156762317267443 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:37.065+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13953156762317267443 PartitionId: 5 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:37.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:37.065+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:37.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:37.065+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:37.066+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 21005 recovery UsedSpace 8236 2023-10-21T04:09:37.066+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:37.071+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex started 2023-10-21T04:09:37.071+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:37.071+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:37.071+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex started 2023-10-21T04:09:37.071+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:37.071+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:37.071+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:37.071+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.071+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.071+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:37.072+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:37.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex] 2023-10-21T04:09:37.072+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:37.072+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex] 2023-10-21T04:09:37.072+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex] 2023-10-21T04:09:37.072+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:37.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex stopped 2023-10-21T04:09:37.072+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:37.072+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:37.072+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:37.072+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:37.085+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:37.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex] 2023-10-21T04:09:37.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex closed 2023-10-21T04:09:37.086+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:37.086+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:37.087+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:37.087+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:37.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex] 2023-10-21T04:09:37.087+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:37.087+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex] 2023-10-21T04:09:37.087+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex] 2023-10-21T04:09:37.087+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:37.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex stopped 2023-10-21T04:09:37.087+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:37.087+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:37.087+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:37.087+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:37.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11400459767051699632:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[80911002600697866:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[80911002600697866:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[80911002600697866:[4 5 6 7 8 1 2 3] 11400459767051699632:[0] 17729593977321451597:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:37.107+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:37.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex] 2023-10-21T04:09:37.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex closed 2023-10-21T04:09:37.108+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:37.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex ... 2023-10-21T04:09:37.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex ... 2023-10-21T04:09:37.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex sucessfully destroyed 2023-10-21T04:09:37.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex ... 2023-10-21T04:09:37.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex ... 2023-10-21T04:09:37.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index destroyed 2023-10-21T04:09:37.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:37.129+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:37.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:37.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:37.139+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:37.139+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:37.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 89 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:37.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:37.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:37.147+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:37.147+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex 2023-10-21T04:09:37.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/mainIndex started 2023-10-21T04:09:37.147+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.147+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#13953156762317267443:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 80 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:37.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:37.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:37.154+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:37.154+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex 2023-10-21T04:09:37.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index/docIndex started 2023-10-21T04:09:37.154+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13953156762317267443 PartitionId 5 Warmup took 89.334125ms 2023-10-21T04:09:37.154+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13953156762317267443 partitionId 5 WriterThreads 1 2023-10-21T04:09:37.154+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13953156762317267443 PartitionId 5 Stopped Billing WUs 2023-10-21T04:09:37.154+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13953156762317267443 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_13953156762317267443_5.index Index: 13953156762317267443 Partition: 5 2023-10-21T04:09:37.154+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 13953156762317267443, shardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:37.154+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 13953156762317267443, partnShardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:37.155+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13953156762317267443 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:37.155+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.155+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#13953156762317267443:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.156+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13953156762317267443 2023-10-21T04:09:37.157+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13953156762317267443 Name: idx_partitioned_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:37.157+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:37.158+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:37.159+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13953156762317267443 Name: idx_partitioned_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:37.160+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 13953156762317267443 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:37.160+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13953156762317267443 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:09:37.160+05:30 [Info] StorageMgr::openSnapshot IndexInst:13953156762317267443 Partition:6 No Snapshot Found. 2023-10-21T04:09:37.160+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13953156762317267443 Adding Nil Snapshot. 2023-10-21T04:09:37.169+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.089443ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:37.169+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 125.847µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:37.219+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.219+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.224+05:30 [Info] switched currmeta from 298 -> 299 force true 2023-10-21T04:09:37.241+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14780363202971367008:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:37.241+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [3674078063353481907] belonging to collectionId: 9, ttid: ShardToken80:53:e0:a8:fa:c4:c4:85. Initiating build 2023-10-21T04:09:37.254+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [14780363202971367008] &{2} 2023-10-21T04:09:37.268+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.268+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.275+05:30 [Info] switched currmeta from 299 -> 300 force true 2023-10-21T04:09:37.267+05:30 [Info] Indexer::handleBuildRecoveredIndexes [14780363202971367008] 2023-10-21T04:09:37.279+05:30 [Info] bulkUpdateRState: Index instance 14780363202971367008 rstate moved to PENDING 2023-10-21T04:09:37.280+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [14780363202971367008] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:37.282+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:37.283+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:37.284+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:37.285+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:09:37.285+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:37.285+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:37.285+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:37.286+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:09:37.287+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:09:37.287+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:37.287+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:37.287+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:09:37.287+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:37.287+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:37.288+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_system:_query reset bucket state to NIL 2023-10-21T04:09:37.288+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:09:37.288+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId 52545 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:37.288+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:37.288+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:37.288+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:09:37.288+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:6] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:09:37.288+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 14780363202971367008 stream INIT_STREAM keyspaceId bucket_%2:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:37.288+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:09:37.288+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 52545 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:09:37.289+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:09:37.289+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:09:37.290+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:37.290+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 23.395877ms 2023-10-21T04:09:37.290+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 23.446956ms 2023-10-21T04:09:37.290+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [14780363202971367008] 2023-10-21T04:09:37.291+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[14780363202971367008:true] 2023-10-21T04:09:37.291+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 47.937839ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:37.298+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_system:_query Check pending stream merge. 2023-10-21T04:09:37.300+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.783875ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:37.311+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.312+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.319+05:30 [Info] switched currmeta from 300 -> 301 force true 2023-10-21T04:09:37.446+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.446+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.465+05:30 [Info] switched currmeta from 301 -> 302 force true 2023-10-21T04:09:37.469+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8645864487128554, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:09:37.539+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.539+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.545+05:30 [Info] switched currmeta from 302 -> 303 force true 2023-10-21T04:09:37.581+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.581+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.593+05:30 [Info] switched currmeta from 303 -> 304 force true 2023-10-21T04:09:37.611+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[13953156762317267443:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:37.611+05:30 [Info] Indexer::handleMergePartition Source 8934648905917366073 Target 13953156762317267443 2023-10-21T04:09:37.611+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13953156762317267443 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc01022cba0 2023-10-21T04:09:37.612+05:30 [Info] MergePartition: sent async request to update index instance 13953156762317267443 rstate moved to ACTIVE 2023-10-21T04:09:37.624+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.296828ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:37.624+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 8934648905917366073, realInst: 13953156762317267443 2023-10-21T04:09:37.626+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 15945117906390072636, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:37.626+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 15945117906390072636, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:37.645+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.646+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.654+05:30 [Info] switched currmeta from 304 -> 305 force true 2023-10-21T04:09:37.664+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11739645294255241349,realInstId: 0, indexDefn DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:37.666+05:30 [Info] Indexer::handleRecoverIndex InstId: 11739645294255241349 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:37.682+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 16.038484ms 2023-10-21T04:09:37.683+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 16.208756ms 2023-10-21T04:09:37.683+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11739645294255241349 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:37.683+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11739645294255241349 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:37.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:37.684+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:37.685+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11739645294255241349, indexDefn DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:37.686+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 59.224974ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:37.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:37.690+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:37.694+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 21005 recovery UsedSpace 8236 2023-10-21T04:09:37.695+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:37.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:37.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:37.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:37.703+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:37.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex started 2023-10-21T04:09:37.703+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.703+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:37.709+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.710+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:37.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:37.710+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:37.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex started 2023-10-21T04:09:37.710+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11739645294255241349 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:37.710+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11739645294255241349 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:37.710+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:37.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex] 2023-10-21T04:09:37.711+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:37.711+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex] 2023-10-21T04:09:37.711+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex] 2023-10-21T04:09:37.711+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:37.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex stopped 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.711+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.716+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:37.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex] 2023-10-21T04:09:37.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex closed 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:37.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex] 2023-10-21T04:09:37.717+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:37.717+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex] 2023-10-21T04:09:37.717+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex] 2023-10-21T04:09:37.717+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:37.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex stopped 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:37.717+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:37.729+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:37.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex] 2023-10-21T04:09:37.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex closed 2023-10-21T04:09:37.730+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:37.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex ... 2023-10-21T04:09:37.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex ... 2023-10-21T04:09:37.738+05:30 [Info] switched currmeta from 305 -> 306 force true 2023-10-21T04:09:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex sucessfully destroyed 2023-10-21T04:09:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex ... 2023-10-21T04:09:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex ... 2023-10-21T04:09:37.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index destroyed 2023-10-21T04:09:37.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:37.759+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:37.759+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#11739645294255241349:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:37.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:37.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:37.767+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:37.767+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#11739645294255241349:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:37.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 81 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:37.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:37.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:37.784+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:37.784+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex 2023-10-21T04:09:37.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/docIndex started 2023-10-21T04:09:37.784+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#11739645294255241349:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.784+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#11739645294255241349:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 90 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:37.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:37.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:37.785+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:37.785+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex 2023-10-21T04:09:37.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index/mainIndex started 2023-10-21T04:09:37.785+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#11739645294255241349:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:37.785+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#11739645294255241349:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:37.785+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11739645294255241349 PartitionId 0 Warmup took 101.223137ms 2023-10-21T04:09:37.785+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11739645294255241349 partitionId 0 WriterThreads 4 2023-10-21T04:09:37.785+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11739645294255241349 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:37.785+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11739645294255241349 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_11739645294255241349_0.index Index: 11739645294255241349 Partition: 0 2023-10-21T04:09:37.785+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11739645294255241349, shardIdMap: map[0:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:37.785+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11739645294255241349, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:37.785+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11739645294255241349 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:37.786+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11739645294255241349 2023-10-21T04:09:37.786+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11739645294255241349 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:37.786+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:37.786+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:37.787+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11739645294255241349 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:37.787+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11739645294255241349 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:37.787+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11739645294255241349 Partitions [0] 2023-10-21T04:09:37.787+05:30 [Info] StorageMgr::openSnapshot IndexInst:11739645294255241349 Partition:0 No Snapshot Found. 2023-10-21T04:09:37.787+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11739645294255241349 Adding Nil Snapshot. 2023-10-21T04:09:37.804+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.374144ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:37.804+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 76.701µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:37.838+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.838+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.843+05:30 [Info] switched currmeta from 306 -> 307 force true 2023-10-21T04:09:37.966+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:37.966+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:37.969+05:30 [Info] switched currmeta from 307 -> 308 force true 2023-10-21T04:09:38.030+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:38.030+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:38.035+05:30 [Info] switched currmeta from 308 -> 309 force true 2023-10-21T04:09:38.292+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:09:38.302+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59356" doAuth() ... 2023-10-21T04:09:38.302+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59356" auth successful 2023-10-21T04:09:38.303+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:59356" +1 2023-10-21T04:09:38.518+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:38.518+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:38.526+05:30 [Info] switched currmeta from 309 -> 310 force true 2023-10-21T04:09:38.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 0 263268055970459 52 52545 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:38.596+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 1 176639880977228 58 52545 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:38.598+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 3 215749515705641 45 52545 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:38.598+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 2 35403626585217 50 52545 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:38.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 4 159113185399719 58 52545 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:38.608+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 6 261493429196201 60 52545 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:38.609+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:38.615+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:38.615+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 5 3874519919278 54 52545 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:38.622+05:30 [Info] switched currmeta from 310 -> 311 force true 2023-10-21T04:09:38.638+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 7 170288557775498 52 52545 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 17 98820809009123 44 52545 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 15 162158720181435 67 52545 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 9 236398693961229 62 52545 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 10 206533515052028 50 52545 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 21 27979933591532 54 52545 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 13 102350060311343 53 52545 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 14 140236896721046 60 52545 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 18 207134287529256 46 52545 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 11 234758907181738 56 52545 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 8 264975169139761 56 52545 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 12 116104661063478 55 52545 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:38.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 16 200752780087280 51 52545 172.31.5.112:9000. HWT [49-51,51]. 2023-10-21T04:09:38.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 22 192215088742243 55 52545 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:38.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 19 208836052671833 45 52545 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:38.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 20 92956357968546 50 52545 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:38.643+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 23 154934968850942 51 52545 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:38.655+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 26 120989102157727 59 52545 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:38.655+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 24 271667630317209 52 52545 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:38.655+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 25 168740244208559 59 52545 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:38.655+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 29 130231725986856 49 52545 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:38.657+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 27 5715487545729 54 52545 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:38.657+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 30 60545336618717 46 52545 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:38.657+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 28 236953356346174 54 52545 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:38.659+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 31 73432771436458 60 52545 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:38.660+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 32 67362620867705 67 52545 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:38.663+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 33 278426329668882 54 52545 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:38.665+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 34 51776324008509 60 52545 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:38.669+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 36 43594649874772 52 52545 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:38.669+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 35 24368176313355 50 52545 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:38.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 38 181267443431019 53 52545 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:38.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 37 83647976044922 59 52545 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:38.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 40 153478150002150 68 52545 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:09:38.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 39 100659545510465 49 52545 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:09:38.678+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 42 45479338375128 55 52545 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:38.678+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 41 116785907755771 55 52545 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:38.682+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 44 166675095997741 57 52545 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:38.682+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 43 249919763471169 64 52545 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:38.683+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 45 49796766757346 58 52545 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:38.687+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 46 124194574848641 54 52545 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:38.692+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 47 149022976193937 55 52545 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:38.700+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:38.700+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:38.704+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11739645294255241349:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:38.704+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11739645294255241349 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0070ed080 2023-10-21T04:09:38.704+05:30 [Info] handleUpdateIndexRState: Index instance 11739645294255241349 rstate moved to ACTIVE 2023-10-21T04:09:38.710+05:30 [Info] switched currmeta from 311 -> 312 force true 2023-10-21T04:09:38.710+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 48 19274425379955 55 52545 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:38.710+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 49 184487117398647 55 52545 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:38.732+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 51 196777743741606 51 52545 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:38.732+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 50 199728167534433 66 52545 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:09:38.735+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 30.27802ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:38.740+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 573521267164877540, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:38.740+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 573521267164877540, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:38.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:38.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:38.756+05:30 [Info] switched currmeta from 312 -> 313 force true 2023-10-21T04:09:38.777+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 53 138935694816848 61 52545 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:38.786+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 54 24856128127173 60 52545 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:38.787+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 1532172649864318584,realInstId: 0, indexDefn DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:38.789+05:30 [Info] Indexer::handleRecoverIndex InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:38.814+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 24.391412ms 2023-10-21T04:09:38.817+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 57 191719959781498 63 52545 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:38.814+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:38.818+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 3 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:38.818+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 52 158076890855634 54 52545 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:38.818+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 55 61798873988101 67 52545 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:38.814+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 1532172649864318584, indexDefn DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:38.819+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 78.455066ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:38.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:38.819+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:38.819+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 15 count: 132 metaSz: 2575 rp [1] sn: 16 count: 132 metaSz: 2574 2023-10-21T04:09:38.820+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 38161 recovery UsedSpace 37480 2023-10-21T04:09:38.814+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 25.699448ms 2023-10-21T04:09:38.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 56 29071415952852 61 52545 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:38.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 58 103336286678362 46 52545 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:38.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 59 140942236111856 63 52545 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:38.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:38.828+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:09:38.828+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 15 count: 132 metaSz: 2575 rp [1] sn: 16 count: 132 metaSz: 2574 2023-10-21T04:09:38.829+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 38055 recovery UsedSpace 37802 2023-10-21T04:09:38.829+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:38.830+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:38.833+05:30 [Info] switched currmeta from 313 -> 314 force true 2023-10-21T04:09:38.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex 2023-10-21T04:09:38.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:38.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:38.836+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex 2023-10-21T04:09:38.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/mainIndex started 2023-10-21T04:09:38.836+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:38.837+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:38.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex 2023-10-21T04:09:38.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:38.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:38.847+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex 2023-10-21T04:09:38.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index/docIndex started 2023-10-21T04:09:38.847+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:38.849+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:38.849+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:38.857+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_system:_query InstanceIds [14780363202971367008] 2023-10-21T04:09:38.857+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query Cid 9 SessionId 52545 2023-10-21T04:09:38.857+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 52545 2023-10-21T04:09:38.857+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:09:38.857+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:09:38.858+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 14780363202971367008 INDEX_STATE_CATCHUP 2023-10-21T04:09:38.858+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 14780363202971367008 true 2023-10-21T04:09:38.858+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 14780363202971367008 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query Session: 52545 BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:09:38.858+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_system:_query, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:09:38.858+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:09:38.858+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:09:38.858+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 52545 STREAM_ACTIVE 2023-10-21T04:09:38.858+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 52545 2023-10-21T04:09:38.858+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_system:_query 52545 2023-10-21T04:09:38.858+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:09:38.859+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:38.859+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:38.859+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:38.859+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:38.859+05:30 [Info] send progress stats to clients 2023-10-21T04:09:38.859+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:09:38.859+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:09:38.859+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:6] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:09:38.859+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 52545 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:09:38.870+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 62 177105814270799 58 52545 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:38.870+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 61 119871837460216 39 52545 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:09:38.870+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 63 252537041863196 73 52545 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:09:38.870+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 60 56626109228904 46 52545 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:38.871+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:09:38.871+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=4,1=58,2=4,3=45,4=4,5=4,6=48,7=4,8=4,9=4,10=4,11=4,12=4,13=4,14=60,15=4,16=40,17=4,18=4,19=18,20=4,21=4,22=4,23=22,24=16,25=4,26=4,27=4,28=4,29=4,30=4,31=4,32=4,33=25,34=4,35=4,36=34,37=4,38=26,39=4,40=68,41=4,42=4,43=42,44=57,45=37,46=4,47=4,48=22,49=55,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=4,59=4,60=4,61=15,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:38.880+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.562091ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:38.881+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 2023-10-21T04:09:38.881+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 52545 2023-10-21T04:09:38.881+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 52545 2023-10-21T04:09:38.881+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:09:38.881+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:09:38.881+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 14780363202971367008 false 2023-10-21T04:09:38.881+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_system:_query 14780363202971367008 2023-10-21T04:09:38.881+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:09:38.881+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 52545 2023-10-21T04:09:38.889+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:38.889+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:38.893+05:30 [Info] switched currmeta from 314 -> 315 force true 2023-10-21T04:09:38.894+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 3 Warmup took 76.286885ms 2023-10-21T04:09:38.895+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 3 WriterThreads 1 2023-10-21T04:09:38.895+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_3.index Index: 1532172649864318584 Partition: 3 2023-10-21T04:09:38.895+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:38.896+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 4 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:38.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:38.896+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:38.896+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 14 count: 124 metaSz: 2510 rp [1] sn: 15 count: 124 metaSz: 2575 rp [2] sn: 16 count: 124 metaSz: 2574 2023-10-21T04:09:38.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:38.897+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:38.897+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 15 count: 124 metaSz: 2575 rp [1] sn: 16 count: 124 metaSz: 2574 2023-10-21T04:09:38.900+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 38794 recovery UsedSpace 37480 2023-10-21T04:09:38.901+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 36775 recovery UsedSpace 29266 2023-10-21T04:09:38.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex 2023-10-21T04:09:38.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:38.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:38.917+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex 2023-10-21T04:09:38.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/docIndex started 2023-10-21T04:09:38.917+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:38.918+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:38.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex 2023-10-21T04:09:38.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:38.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:38.931+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex 2023-10-21T04:09:38.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index/mainIndex started 2023-10-21T04:09:38.931+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:38.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:38.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:38.972+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 4 Warmup took 75.76155ms 2023-10-21T04:09:38.977+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 4 WriterThreads 1 2023-10-21T04:09:38.979+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_4.index Index: 1532172649864318584 Partition: 4 2023-10-21T04:09:38.980+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:38.982+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 5 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:38.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:38.984+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:38.984+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 15 count: 111 metaSz: 2574 rp [1] sn: 16 count: 111 metaSz: 2574 2023-10-21T04:09:38.985+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 38176 recovery UsedSpace 37480 2023-10-21T04:09:38.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:38.986+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:09:38.986+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 15 count: 111 metaSz: 2574 rp [1] sn: 16 count: 111 metaSz: 2574 2023-10-21T04:09:38.987+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 37274 recovery UsedSpace 37480 2023-10-21T04:09:39.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex 2023-10-21T04:09:39.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:39.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:39.004+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex 2023-10-21T04:09:39.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/mainIndex started 2023-10-21T04:09:39.004+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.005+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex 2023-10-21T04:09:39.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:39.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:39.006+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex 2023-10-21T04:09:39.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index/docIndex started 2023-10-21T04:09:39.007+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:39.007+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.007+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.066+05:30 [Error] ScanCoordinator: Unable to compute index items_count for bucket_1/idx_partitioned/1532172649864318584 state 0 (Index not found) 2023-10-21T04:09:39.073+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 5 Warmup took 90.244619ms 2023-10-21T04:09:39.073+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 5 WriterThreads 1 2023-10-21T04:09:39.073+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_5.index Index: 1532172649864318584 Partition: 5 2023-10-21T04:09:39.073+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:39.074+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 6 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:39.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:39.074+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:39.074+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 16 count: 140 metaSz: 2575 rp [1] sn: 17 count: 140 metaSz: 2574 2023-10-21T04:09:39.075+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 39710 recovery UsedSpace 37480 2023-10-21T04:09:39.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:39.075+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:09:39.075+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 15 count: 140 metaSz: 2510 rp [1] sn: 16 count: 140 metaSz: 2575 rp [2] sn: 17 count: 140 metaSz: 2574 2023-10-21T04:09:39.075+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 39625 recovery UsedSpace 29588 2023-10-21T04:09:39.078+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:39.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex 2023-10-21T04:09:39.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:39.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:39.092+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex 2023-10-21T04:09:39.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/docIndex started 2023-10-21T04:09:39.092+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.092+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.092+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.092+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.095+05:30 [Info] switched currmeta from 315 -> 315 force true 2023-10-21T04:09:39.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex 2023-10-21T04:09:39.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:39.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:39.097+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex 2023-10-21T04:09:39.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index/mainIndex started 2023-10-21T04:09:39.097+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:39.098+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.098+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.116+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 6 Warmup took 42.416417ms 2023-10-21T04:09:39.117+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 6 WriterThreads 1 2023-10-21T04:09:39.117+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_6.index Index: 1532172649864318584 Partition: 6 2023-10-21T04:09:39.117+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:39.117+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 7 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:39.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:39.117+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:09:39.117+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 15 count: 125 metaSz: 2510 rp [1] sn: 16 count: 125 metaSz: 2574 rp [2] sn: 17 count: 125 metaSz: 2574 2023-10-21T04:09:39.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:39.118+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:39.118+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 16 count: 125 metaSz: 2574 rp [1] sn: 17 count: 125 metaSz: 2574 2023-10-21T04:09:39.118+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 39165 recovery UsedSpace 29396 2023-10-21T04:09:39.118+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 39935 recovery UsedSpace 37480 2023-10-21T04:09:39.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex 2023-10-21T04:09:39.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:39.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:39.149+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex 2023-10-21T04:09:39.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/mainIndex started 2023-10-21T04:09:39.149+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.149+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex 2023-10-21T04:09:39.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:39.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:39.150+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex 2023-10-21T04:09:39.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index/docIndex started 2023-10-21T04:09:39.150+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:39.150+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.150+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.187+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 7 Warmup took 70.218565ms 2023-10-21T04:09:39.188+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 7 WriterThreads 1 2023-10-21T04:09:39.188+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_7.index Index: 1532172649864318584 Partition: 7 2023-10-21T04:09:39.188+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:39.191+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 8 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:39.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:39.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:39.192+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 14 count: 121 metaSz: 2510 rp [1] sn: 15 count: 121 metaSz: 2574 rp [2] sn: 16 count: 121 metaSz: 2574 2023-10-21T04:09:39.193+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 38560 recovery UsedSpace 29266 2023-10-21T04:09:39.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:39.193+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:39.193+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 15 count: 121 metaSz: 2574 rp [1] sn: 16 count: 121 metaSz: 2574 2023-10-21T04:09:39.194+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 39556 recovery UsedSpace 37480 2023-10-21T04:09:39.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex 2023-10-21T04:09:39.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:39.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:39.211+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex 2023-10-21T04:09:39.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/docIndex started 2023-10-21T04:09:39.211+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.211+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex 2023-10-21T04:09:39.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:39.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:39.212+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex 2023-10-21T04:09:39.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index/mainIndex started 2023-10-21T04:09:39.212+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:39.212+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.212+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.246+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 8 Warmup took 53.712569ms 2023-10-21T04:09:39.246+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 8 WriterThreads 1 2023-10-21T04:09:39.246+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_8.index Index: 1532172649864318584 Partition: 8 2023-10-21T04:09:39.246+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:39.246+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 1 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:39.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:39.247+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:09:39.247+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 13 count: 134 metaSz: 2510 rp [1] sn: 14 count: 134 metaSz: 2575 rp [2] sn: 15 count: 134 metaSz: 2574 2023-10-21T04:09:39.247+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 41248 recovery UsedSpace 29588 2023-10-21T04:09:39.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:39.248+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:39.248+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 14 count: 134 metaSz: 2575 rp [1] sn: 15 count: 134 metaSz: 2574 2023-10-21T04:09:39.248+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 40267 recovery UsedSpace 37480 2023-10-21T04:09:39.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex 2023-10-21T04:09:39.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:39.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:39.262+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex 2023-10-21T04:09:39.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/mainIndex started 2023-10-21T04:09:39.262+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.262+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex 2023-10-21T04:09:39.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:39.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:39.265+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex 2023-10-21T04:09:39.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index/docIndex started 2023-10-21T04:09:39.265+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:39.265+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.265+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.324+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 1 Warmup took 77.407972ms 2023-10-21T04:09:39.324+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 1 WriterThreads 1 2023-10-21T04:09:39.324+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_1.index Index: 1532172649864318584 Partition: 1 2023-10-21T04:09:39.324+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1532172649864318584 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:39.325+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1532172649864318584 PartitionId: 2 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:39.325+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:09:39.325+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:09:39.325+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 14 count: 113 metaSz: 2510 rp [1] sn: 15 count: 113 metaSz: 2574 rp [2] sn: 16 count: 113 metaSz: 2574 2023-10-21T04:09:39.326+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 39418 recovery UsedSpace 29266 2023-10-21T04:09:39.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:39.329+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:39.331+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 15 count: 113 metaSz: 2574 rp [1] sn: 16 count: 113 metaSz: 2574 2023-10-21T04:09:39.331+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 40522 recovery UsedSpace 37480 2023-10-21T04:09:39.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex 2023-10-21T04:09:39.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:09:39.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:09:39.335+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex 2023-10-21T04:09:39.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/docIndex started 2023-10-21T04:09:39.335+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.335+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex 2023-10-21T04:09:39.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:39.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:39.342+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex 2023-10-21T04:09:39.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index/mainIndex started 2023-10-21T04:09:39.342+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:39.342+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.342+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.357+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1532172649864318584 PartitionId 2 Warmup took 31.782613ms 2023-10-21T04:09:39.357+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1532172649864318584 partitionId 2 WriterThreads 1 2023-10-21T04:09:39.357+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1532172649864318584 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_1532172649864318584_2.index Index: 1532172649864318584 Partition: 2 2023-10-21T04:09:39.357+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 1532172649864318584, shardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]] err: 2023-10-21T04:09:39.357+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 1532172649864318584, partnShardIdMap: map[1:[10563635432419104314 3268862397300335529] 2:[10563635432419104314 3268862397300335529] 3:[10563635432419104314 3268862397300335529] 4:[10563635432419104314 3268862397300335529] 5:[10563635432419104314 3268862397300335529] 6:[10563635432419104314 3268862397300335529] 7:[10563635432419104314 3268862397300335529] 8:[10563635432419104314 3268862397300335529]], err: 2023-10-21T04:09:39.357+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:39.358+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 1532172649864318584 2023-10-21T04:09:39.358+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:39.358+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:39.359+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:39.359+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:39.359+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:39.359+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 1532172649864318584 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:09:39.360+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:1 Attempting to open snapshot (SnapshotInfo: count:134 committed:false) 2023-10-21T04:09:39.363+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:2 Attempting to open snapshot (SnapshotInfo: count:113 committed:false) 2023-10-21T04:09:39.364+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:3 Attempting to open snapshot (SnapshotInfo: count:132 committed:false) 2023-10-21T04:09:39.365+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:4 Attempting to open snapshot (SnapshotInfo: count:124 committed:false) 2023-10-21T04:09:39.367+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:5 Attempting to open snapshot (SnapshotInfo: count:111 committed:false) 2023-10-21T04:09:39.368+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:6 Attempting to open snapshot (SnapshotInfo: count:140 committed:false) 2023-10-21T04:09:39.368+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.470954ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:39.368+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 129.186µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:39.369+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:7 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:09:39.374+05:30 [Info] StorageMgr::openSnapshot IndexInst:1532172649864318584 Partition:8 Attempting to open snapshot (SnapshotInfo: count:121 committed:false) 2023-10-21T04:09:39.381+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.381+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.385+05:30 [Info] switched currmeta from 315 -> 316 force true 2023-10-21T04:09:39.438+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.438+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.444+05:30 [Info] switched currmeta from 316 -> 317 force true 2023-10-21T04:09:39.447+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 7] call. taskList: &{Rev:[0 0 0 0 0 0 0 8] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 5] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:39.448+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 8] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:39.450+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 8] 2023-10-21T04:09:39.450+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 8] 2023-10-21T04:09:39.496+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.496+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.499+05:30 [Info] switched currmeta from 317 -> 318 force true 2023-10-21T04:09:39.529+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.529+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.534+05:30 [Info] switched currmeta from 318 -> 319 force true 2023-10-21T04:09:39.559+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.559+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.570+05:30 [Info] switched currmeta from 319 -> 320 force true 2023-10-21T04:09:39.656+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.656+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.659+05:30 [Info] switched currmeta from 320 -> 321 force true 2023-10-21T04:09:39.859+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[1532172649864318584:INDEX_STATE_RECOVERED 3292093078969249751:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:39.859+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:09:39.859+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_system:_query 2023-10-21T04:09:39.859+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 14780363202971367008 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId: 52545, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:39.859+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 14780363202971367008 INDEX_STATE_ACTIVE 2023-10-21T04:09:39.859+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query State Changed to INACTIVE 2023-10-21T04:09:39.859+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:09:39.859+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:09:39.859+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:39.859+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 52545 2023-10-21T04:09:39.860+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:39.860+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 12338036531800130039, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:39.860+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:39.860+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 12338036531800130039, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:39.860+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14780363202971367008 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:39.860+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 52545 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:39.861+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 14780363202971367008 from stream INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:09:39.861+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:09:39.861+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:09:39.861+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:39.861+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 52545 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:39.861+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:09:39.861+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:39.861+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:39.861+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:09:39.862+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:59356: use of closed network connection 2023-10-21T04:09:39.862+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:59356" exit: read tcp4 127.0.0.1:9121->127.0.0.1:59356: use of closed network connection 2023-10-21T04:09:39.862+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59356" closed ! 2023-10-21T04:09:39.862+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:09:39.862+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:59356" already gone 2023-10-21T04:09:39.862+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:39.862+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:39.862+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:39.862+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:39.862+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:09:39.862+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:39.862+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId 52545 2023-10-21T04:09:39.862+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:09:39.882+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_system:_query]. Err genServer.closed 2023-10-21T04:09:39.882+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:09:39.882+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Treating genServer.closed As Success 2023-10-21T04:09:39.882+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 52545 2023-10-21T04:09:39.882+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 52545 2023-10-21T04:09:39.882+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:09:39.923+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3810060733504350542,realInstId: 0, indexDefn DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:39.923+05:30 [Info] Indexer::handleRecoverIndex InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:39.923+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 149.698µs 2023-10-21T04:09:39.923+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 194.927µs 2023-10-21T04:09:39.923+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3810060733504350542 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:39.923+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3810060733504350542 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:39.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:39.924+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1054732 memSz 772 2023-10-21T04:09:39.924+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 20 count: 1000 metaSz: 2553 rp [1] sn: 21 count: 1000 metaSz: 2552 2023-10-21T04:09:39.924+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 66100 recovery UsedSpace 38908 2023-10-21T04:09:39.928+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3810060733504350542, indexDefn DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:39.928+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 68.484304ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:39.937+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:39.937+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:39.941+05:30 [Info] switched currmeta from 321 -> 322 force true 2023-10-21T04:09:39.961+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 32.80025ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:39.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex 2023-10-21T04:09:39.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:39.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:39.972+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex 2023-10-21T04:09:39.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index/mainIndex started 2023-10-21T04:09:39.972+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3810060733504350542 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:39.973+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:39.973+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:39.993+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3810060733504350542 PartitionId 0 Warmup took 69.335045ms 2023-10-21T04:09:39.993+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3810060733504350542 partitionId 0 WriterThreads 4 2023-10-21T04:09:39.993+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3810060733504350542 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3810060733504350542_0.index Index: 3810060733504350542 Partition: 0 2023-10-21T04:09:39.993+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3810060733504350542, shardIdMap: map[0:[10563635432419104314]] err: 2023-10-21T04:09:39.993+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3810060733504350542, partnShardIdMap: map[0:[10563635432419104314]], err: 2023-10-21T04:09:39.993+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:39.994+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3810060733504350542 2023-10-21T04:09:39.994+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:39.994+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:39.994+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:39.994+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:39.995+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:39.995+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3810060733504350542 Partitions [0] 2023-10-21T04:09:39.996+05:30 [Info] StorageMgr::openSnapshot IndexInst:3810060733504350542 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:40.007+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.007+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.008+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.408252ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:40.020+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 84.421µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:40.034+05:30 [Info] switched currmeta from 322 -> 324 force true 2023-10-21T04:09:40.065+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.065+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.068+05:30 [Info] switched currmeta from 324 -> 324 force true 2023-10-21T04:09:40.315+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14780363202971367008 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006f56780 2023-10-21T04:09:40.334+05:30 [Info] handleUpdateIndexRState: Index instance 14780363202971367008 rstate moved to ACTIVE 2023-10-21T04:09:40.321+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.334+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.339+05:30 [Info] switched currmeta from 324 -> 325 force true 2023-10-21T04:09:40.353+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.505756ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:40.353+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 15131823346681830059. Retrying... 2023-10-21T04:09:40.353+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[14780363202971367008:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:40.356+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 15649171661394925140, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:40.361+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 15649171661394925140, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:40.365+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.365+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.369+05:30 [Info] switched currmeta from 325 -> 326 force true 2023-10-21T04:09:40.419+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12043401624902403496,realInstId: 0, indexDefn DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:40.419+05:30 [Info] Indexer::handleRecoverIndex InstId: 12043401624902403496 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:40.420+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 367.429µs 2023-10-21T04:09:40.420+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 416.55µs 2023-10-21T04:09:40.420+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:40.420+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 5 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:40.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.421+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.421+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:40.423+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12043401624902403496, indexDefn DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:40.423+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 67.336031ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:40.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:40.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.425+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:40.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex started 2023-10-21T04:09:40.425+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.426+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.426+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:40.427+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.430+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.430+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:40.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.432+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:40.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex started 2023-10-21T04:09:40.432+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:40.432+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex] 2023-10-21T04:09:40.432+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex] 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex] 2023-10-21T04:09:40.432+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:40.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex stopped 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.432+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.437+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex] 2023-10-21T04:09:40.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex closed 2023-10-21T04:09:40.437+05:30 [Info] switched currmeta from 326 -> 327 force true 2023-10-21T04:09:40.437+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.437+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:40.438+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.438+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.438+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex] 2023-10-21T04:09:40.438+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:40.438+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex] 2023-10-21T04:09:40.438+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex] 2023-10-21T04:09:40.439+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:40.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex stopped 2023-10-21T04:09:40.439+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.439+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.439+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.439+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.446+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex] 2023-10-21T04:09:40.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex closed 2023-10-21T04:09:40.447+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:40.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex ... 2023-10-21T04:09:40.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex ... 2023-10-21T04:09:40.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex sucessfully destroyed 2023-10-21T04:09:40.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex ... 2023-10-21T04:09:40.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex ... 2023-10-21T04:09:40.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index destroyed 2023-10-21T04:09:40.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.465+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.465+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 73 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:40.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.469+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:40.469+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex 2023-10-21T04:09:40.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/docIndex started 2023-10-21T04:09:40.469+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.469+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.475+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.475+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 82 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:40.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.480+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:40.480+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex 2023-10-21T04:09:40.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index/mainIndex started 2023-10-21T04:09:40.480+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.480+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 5 Warmup took 59.901346ms 2023-10-21T04:09:40.480+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.480+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 5 WriterThreads 1 2023-10-21T04:09:40.480+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 5 Stopped Billing WUs 2023-10-21T04:09:40.481+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_5.index Index: 12043401624902403496 Partition: 5 2023-10-21T04:09:40.481+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:40.481+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 6 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:40.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.481+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.481+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.482+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:40.482+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 18618 recovery UsedSpace 8236 2023-10-21T04:09:40.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:40.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.486+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:40.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex started 2023-10-21T04:09:40.486+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.486+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:40.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.487+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:40.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex started 2023-10-21T04:09:40.487+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:40.487+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex] 2023-10-21T04:09:40.487+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex] 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex] 2023-10-21T04:09:40.487+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:40.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex stopped 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.492+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex] 2023-10-21T04:09:40.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex closed 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex] 2023-10-21T04:09:40.493+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex] 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex] 2023-10-21T04:09:40.493+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:40.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex stopped 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex] 2023-10-21T04:09:40.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex closed 2023-10-21T04:09:40.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:40.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex ... 2023-10-21T04:09:40.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex ... 2023-10-21T04:09:40.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex sucessfully destroyed 2023-10-21T04:09:40.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex ... 2023-10-21T04:09:40.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex ... 2023-10-21T04:09:40.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index destroyed 2023-10-21T04:09:40.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.514+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.514+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.525+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.525+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 74 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:40.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.543+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:40.543+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex 2023-10-21T04:09:40.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/docIndex started 2023-10-21T04:09:40.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 83 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:40.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.552+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:40.552+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex 2023-10-21T04:09:40.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index/mainIndex started 2023-10-21T04:09:40.552+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 6 Warmup took 71.587266ms 2023-10-21T04:09:40.553+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 6 WriterThreads 1 2023-10-21T04:09:40.553+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 6 Stopped Billing WUs 2023-10-21T04:09:40.553+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_6.index Index: 12043401624902403496 Partition: 6 2023-10-21T04:09:40.553+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:40.553+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 7 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:40.553+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.553+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.553+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.554+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.554+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:40.555+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:40.578+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.578+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.587+05:30 [Info] switched currmeta from 327 -> 328 force true 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14780363202971367008:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:40.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:40.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.595+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:40.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex started 2023-10-21T04:09:40.595+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.595+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:40.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.597+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:40.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex started 2023-10-21T04:09:40.597+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:40.597+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:40.597+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.597+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.597+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex] 2023-10-21T04:09:40.598+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex] 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex] 2023-10-21T04:09:40.598+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:40.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex stopped 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.598+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.618+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex] 2023-10-21T04:09:40.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex closed 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex] 2023-10-21T04:09:40.619+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex] 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex] 2023-10-21T04:09:40.619+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:40.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex stopped 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.619+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.627+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex] 2023-10-21T04:09:40.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex closed 2023-10-21T04:09:40.628+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:40.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex ... 2023-10-21T04:09:40.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex ... 2023-10-21T04:09:40.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex sucessfully destroyed 2023-10-21T04:09:40.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex ... 2023-10-21T04:09:40.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex ... 2023-10-21T04:09:40.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.658+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.668+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex sucessfully destroyed 2023-10-21T04:09:40.667+05:30 [Info] switched currmeta from 328 -> 329 force true All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index destroyed 2023-10-21T04:09:40.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.677+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.677+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 84 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:40.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.695+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:40.696+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex 2023-10-21T04:09:40.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/mainIndex started 2023-10-21T04:09:40.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.697+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.697+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.698+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.698+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 75 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:40.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.710+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:40.710+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex 2023-10-21T04:09:40.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index/docIndex started 2023-10-21T04:09:40.710+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 7 Warmup took 157.339239ms 2023-10-21T04:09:40.711+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 7 WriterThreads 1 2023-10-21T04:09:40.711+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 7 Stopped Billing WUs 2023-10-21T04:09:40.712+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_7.index Index: 12043401624902403496 Partition: 7 2023-10-21T04:09:40.711+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.711+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.712+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:40.713+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.713+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.713+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 8 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:40.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.713+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.714+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:40.715+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:40.718+05:30 [Info] switched currmeta from 329 -> 330 force true 2023-10-21T04:09:40.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:40.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.721+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:40.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex started 2023-10-21T04:09:40.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.722+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:40.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.727+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:40.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex started 2023-10-21T04:09:40.727+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:40.727+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex] 2023-10-21T04:09:40.727+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex] 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex] 2023-10-21T04:09:40.727+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:40.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex stopped 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.727+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.728+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.735+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex] 2023-10-21T04:09:40.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex closed 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex] 2023-10-21T04:09:40.736+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex] 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex] 2023-10-21T04:09:40.736+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:40.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex stopped 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.736+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.745+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex] 2023-10-21T04:09:40.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex closed 2023-10-21T04:09:40.745+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:40.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex ... 2023-10-21T04:09:40.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex ... 2023-10-21T04:09:40.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex sucessfully destroyed 2023-10-21T04:09:40.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex ... 2023-10-21T04:09:40.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex ... 2023-10-21T04:09:40.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index destroyed 2023-10-21T04:09:40.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.765+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.765+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.775+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 76 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:40.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.777+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:40.778+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex 2023-10-21T04:09:40.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/docIndex started 2023-10-21T04:09:40.780+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.781+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.783+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.790+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 85 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:40.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.792+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:40.792+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex 2023-10-21T04:09:40.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index/mainIndex started 2023-10-21T04:09:40.795+05:30 [Info] switched currmeta from 330 -> 331 force true 2023-10-21T04:09:40.801+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 8 Warmup took 88.380269ms 2023-10-21T04:09:40.802+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.802+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.802+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 8 WriterThreads 1 2023-10-21T04:09:40.802+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 8 Stopped Billing WUs 2023-10-21T04:09:40.802+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_8.index Index: 12043401624902403496 Partition: 8 2023-10-21T04:09:40.802+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:40.803+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 1 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:40.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.804+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.805+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.808+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19051 recovery UsedSpace 8236 2023-10-21T04:09:40.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.818+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex started 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.818+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.818+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:40.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex started 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.819+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:40.819+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex] 2023-10-21T04:09:40.819+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex] 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex] 2023-10-21T04:09:40.819+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:40.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex stopped 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.819+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.833+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex] 2023-10-21T04:09:40.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex closed 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex] 2023-10-21T04:09:40.834+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex] 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex] 2023-10-21T04:09:40.834+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:40.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex stopped 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex] 2023-10-21T04:09:40.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex closed 2023-10-21T04:09:40.855+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:40.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex ... 2023-10-21T04:09:40.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex ... 2023-10-21T04:09:40.858+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:09:40.863+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:40.863+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:40.869+05:30 [Info] switched currmeta from 331 -> 332 force true 2023-10-21T04:09:40.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex sucessfully destroyed 2023-10-21T04:09:40.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex ... 2023-10-21T04:09:40.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex ... 2023-10-21T04:09:40.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index destroyed 2023-10-21T04:09:40.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.888+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.888+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.898+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 86 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:40.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.903+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:40.903+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex 2023-10-21T04:09:40.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/mainIndex started 2023-10-21T04:09:40.903+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.903+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 77 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:40.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.909+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:40.909+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex 2023-10-21T04:09:40.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index/docIndex started 2023-10-21T04:09:40.909+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 1 Warmup took 106.27758ms 2023-10-21T04:09:40.909+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 1 WriterThreads 1 2023-10-21T04:09:40.910+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 1 Stopped Billing WUs 2023-10-21T04:09:40.910+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_1.index Index: 12043401624902403496 Partition: 1 2023-10-21T04:09:40.910+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:40.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.911+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 2 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:40.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.911+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.912+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.913+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:40.913+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:40.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:40.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.919+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:40.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex started 2023-10-21T04:09:40.919+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.919+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:40.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.920+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:40.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex started 2023-10-21T04:09:40.920+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:40.920+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:40.920+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.920+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.920+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex] 2023-10-21T04:09:40.921+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:40.921+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex] 2023-10-21T04:09:40.920+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.920+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.921+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex] 2023-10-21T04:09:40.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:40.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex stopped 2023-10-21T04:09:40.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.923+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.928+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex] 2023-10-21T04:09:40.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex closed 2023-10-21T04:09:40.928+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:40.928+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.928+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.928+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex] 2023-10-21T04:09:40.928+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:40.928+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex] 2023-10-21T04:09:40.928+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex] 2023-10-21T04:09:40.928+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:40.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex stopped 2023-10-21T04:09:40.929+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.929+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.929+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.929+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex] 2023-10-21T04:09:40.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex closed 2023-10-21T04:09:40.933+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:40.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex ... 2023-10-21T04:09:40.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex ... 2023-10-21T04:09:40.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex sucessfully destroyed 2023-10-21T04:09:40.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex ... 2023-10-21T04:09:40.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex ... 2023-10-21T04:09:40.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index destroyed 2023-10-21T04:09:40.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.945+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.945+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:40.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.950+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:40.950+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:40.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 87 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:40.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.952+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:40.952+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex 2023-10-21T04:09:40.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/mainIndex started 2023-10-21T04:09:40.952+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.952+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 78 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:40.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.957+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:40.957+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex 2023-10-21T04:09:40.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index/docIndex started 2023-10-21T04:09:40.957+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.957+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.957+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 2 Warmup took 46.291251ms 2023-10-21T04:09:40.958+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 2 WriterThreads 1 2023-10-21T04:09:40.958+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 2 Stopped Billing WUs 2023-10-21T04:09:40.959+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_2.index Index: 12043401624902403496 Partition: 2 2023-10-21T04:09:40.959+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:40.960+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 3 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:40.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:40.960+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.961+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 20480 recovery UsedSpace 8236 2023-10-21T04:09:40.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:40.962+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:40.962+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:40.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:40.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:40.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:40.966+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:40.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex started 2023-10-21T04:09:40.966+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.966+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.971+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[1532172649864318584:INDEX_STATE_RECOVERED 3292093078969249751:INDEX_STATE_RECOVERED 3810060733504350542:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:40.972+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 11619165464560280276, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:40.972+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 11619165464560280276, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:40.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:40.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:40.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:40.972+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:40.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex started 2023-10-21T04:09:40.973+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:40.973+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:40.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex] 2023-10-21T04:09:40.973+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex] 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex] 2023-10-21T04:09:40.973+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:40.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex stopped 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.973+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:40.987+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:40.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex] 2023-10-21T04:09:40.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex closed 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:40.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex] 2023-10-21T04:09:40.988+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex] 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex] 2023-10-21T04:09:40.988+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:40.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex stopped 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:40.988+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:41.012+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7597064108789640455,realInstId: 0, indexDefn DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:41.012+05:30 [Info] Indexer::handleRecoverIndex InstId: 7597064108789640455 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:41.014+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 2.061519ms 2023-10-21T04:09:41.014+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 2.108897ms 2023-10-21T04:09:41.014+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7597064108789640455 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:41.014+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7597064108789640455 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:41.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:41.014+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:41.015+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 22755 recovery UsedSpace 8236 2023-10-21T04:09:41.015+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:41.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex] 2023-10-21T04:09:41.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex closed 2023-10-21T04:09:41.016+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:41.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex ... 2023-10-21T04:09:41.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex ... 2023-10-21T04:09:41.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:41.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:41.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:41.029+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:41.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex started 2023-10-21T04:09:41.029+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7597064108789640455 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:41.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.022+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7597064108789640455, indexDefn DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:41.030+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 58.078157ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:41.029+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7597064108789640455 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:41.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:41.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:41.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex] 2023-10-21T04:09:41.030+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:41.030+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex] 2023-10-21T04:09:41.030+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex] 2023-10-21T04:09:41.030+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:41.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex stopped 2023-10-21T04:09:41.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:41.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:41.031+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:41.031+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:41.032+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:41.034+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.034+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.037+05:30 [Info] switched currmeta from 332 -> 333 force true 2023-10-21T04:09:41.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex sucessfully destroyed 2023-10-21T04:09:41.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex ... 2023-10-21T04:09:41.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex ... 2023-10-21T04:09:41.040+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:41.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex] 2023-10-21T04:09:41.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex closed 2023-10-21T04:09:41.041+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:41.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index destroyed 2023-10-21T04:09:41.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex ... 2023-10-21T04:09:41.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex ... 2023-10-21T04:09:41.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index destroyed 2023-10-21T04:09:41.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:41.052+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:41.052+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:41.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:41.057+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:41.057+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:41.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:41.058+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:41.058+05:30 [Info] bucket_1/#primary_defer/Mainstore#7597064108789640455:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:41.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 79 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:41.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:41.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:41.063+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:41.063+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex 2023-10-21T04:09:41.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/docIndex started 2023-10-21T04:09:41.063+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.064+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 88 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:41.066+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:41.066+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index/mainIndex started 2023-10-21T04:09:41.066+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 3 Warmup took 106.186319ms 2023-10-21T04:09:41.066+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 91 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:41.066+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:41.066+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 3 WriterThreads 1 2023-10-21T04:09:41.066+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex 2023-10-21T04:09:41.066+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 3 Stopped Billing WUs 2023-10-21T04:09:41.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index/mainIndex started 2023-10-21T04:09:41.066+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_3.index Index: 12043401624902403496 Partition: 3 2023-10-21T04:09:41.066+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7597064108789640455 PartitionId 0 Warmup took 52.273738ms 2023-10-21T04:09:41.066+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.067+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7597064108789640455 partitionId 0 WriterThreads 4 2023-10-21T04:09:41.067+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7597064108789640455 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:41.067+05:30 [Info] bucket_1/#primary_defer/Mainstore#7597064108789640455:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.067+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7597064108789640455 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_7597064108789640455_0.index Index: 7597064108789640455 Partition: 0 2023-10-21T04:09:41.067+05:30 [Info] bucket_1/#primary_defer/Mainstore#7597064108789640455:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.067+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7597064108789640455, shardIdMap: map[0:[10563635432419104314]] err: 2023-10-21T04:09:41.066+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12043401624902403496 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:41.067+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7597064108789640455, partnShardIdMap: map[0:[10563635432419104314]], err: 2023-10-21T04:09:41.067+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7597064108789640455 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:41.067+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12043401624902403496 PartitionId: 4 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:41.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:41.067+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:41.067+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7597064108789640455 2023-10-21T04:09:41.067+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7597064108789640455 Name: #primary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:41.068+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:41.068+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19504 recovery UsedSpace 8236 2023-10-21T04:09:41.068+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:41.068+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7597064108789640455 Name: #primary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:41.068+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7597064108789640455 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:41.069+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7597064108789640455 Partitions [0] 2023-10-21T04:09:41.069+05:30 [Info] StorageMgr::openSnapshot IndexInst:7597064108789640455 Partition:0 No Snapshot Found. 2023-10-21T04:09:41.069+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7597064108789640455 Adding Nil Snapshot. 2023-10-21T04:09:41.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:41.069+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:41.073+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 21005 recovery UsedSpace 8236 2023-10-21T04:09:41.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:41.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:41.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:41.076+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:41.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex started 2023-10-21T04:09:41.077+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.077+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.082+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.54961ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:41.082+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 221.62µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:41.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:41.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:41.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:41.086+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:41.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex started 2023-10-21T04:09:41.087+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.087+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.090+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:41.090+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:41.090+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:41.090+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:41.090+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:41.091+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex] 2023-10-21T04:09:41.091+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:41.091+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex] 2023-10-21T04:09:41.091+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex] 2023-10-21T04:09:41.091+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:41.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex stopped 2023-10-21T04:09:41.100+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:41.101+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:41.101+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:41.101+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:41.121+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:41.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex] 2023-10-21T04:09:41.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex closed 2023-10-21T04:09:41.122+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.122+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.123+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:41.129+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:41.129+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:41.130+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:41.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex] 2023-10-21T04:09:41.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:41.130+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex] 2023-10-21T04:09:41.130+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex] 2023-10-21T04:09:41.131+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:41.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex stopped 2023-10-21T04:09:41.131+05:30 [Info] switched currmeta from 333 -> 334 force true 2023-10-21T04:09:41.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:41.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:41.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:41.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:41.137+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:41.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex] 2023-10-21T04:09:41.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex closed 2023-10-21T04:09:41.138+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:41.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex ... 2023-10-21T04:09:41.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex ... 2023-10-21T04:09:41.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex sucessfully destroyed 2023-10-21T04:09:41.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex ... 2023-10-21T04:09:41.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex ... 2023-10-21T04:09:41.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index destroyed 2023-10-21T04:09:41.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:41.157+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:41.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:41.167+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:41.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:41.168+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:41.168+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:41.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 80 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:41.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:41.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:41.178+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:41.178+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex 2023-10-21T04:09:41.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/docIndex started 2023-10-21T04:09:41.178+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.178+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12043401624902403496:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 89 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:41.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:41.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:41.180+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:41.180+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex 2023-10-21T04:09:41.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index/mainIndex started 2023-10-21T04:09:41.180+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12043401624902403496 PartitionId 4 Warmup took 112.916856ms 2023-10-21T04:09:41.180+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.180+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12043401624902403496:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.180+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12043401624902403496 partitionId 4 WriterThreads 1 2023-10-21T04:09:41.180+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12043401624902403496 PartitionId 4 Stopped Billing WUs 2023-10-21T04:09:41.180+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12043401624902403496 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12043401624902403496_4.index Index: 12043401624902403496 Partition: 4 2023-10-21T04:09:41.180+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12043401624902403496, shardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:41.180+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12043401624902403496, partnShardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:41.180+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12043401624902403496 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:41.181+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12043401624902403496 2023-10-21T04:09:41.181+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12043401624902403496 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:41.181+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:41.181+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:41.182+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12043401624902403496 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:41.182+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12043401624902403496 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:41.182+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12043401624902403496 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:09:41.182+05:30 [Info] StorageMgr::openSnapshot IndexInst:12043401624902403496 Partition:4 No Snapshot Found. 2023-10-21T04:09:41.182+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12043401624902403496 Adding Nil Snapshot. 2023-10-21T04:09:41.199+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.198177ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:41.199+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 117.884µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:41.208+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.208+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.211+05:30 [Info] switched currmeta from 334 -> 335 force true 2023-10-21T04:09:41.431+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[12043401624902403496:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:41.432+05:30 [Info] Indexer::handleMergePartition Source 7543997995673100641 Target 12043401624902403496 2023-10-21T04:09:41.432+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12043401624902403496 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0057bf920 2023-10-21T04:09:41.432+05:30 [Info] MergePartition: sent async request to update index instance 12043401624902403496 rstate moved to ACTIVE 2023-10-21T04:09:41.442+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.744752ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:41.442+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 7543997995673100641, realInst: 12043401624902403496 2023-10-21T04:09:41.442+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 4860549750398470299, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:41.443+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 4860549750398470299, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:41.474+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4030261110898422634,realInstId: 0, indexDefn DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:41.475+05:30 [Info] Indexer::handleRecoverIndex InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:41.476+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 243.916µs 2023-10-21T04:09:41.476+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 303.932µs 2023-10-21T04:09:41.476+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4030261110898422634 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:41.476+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4030261110898422634, indexDefn DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:41.476+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4030261110898422634 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:41.476+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 33.530632ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:41.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:41.476+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1056210 memSz 2252 2023-10-21T04:09:41.477+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 1000 metaSz: 2514 rp [1] sn: 4 count: 1000 metaSz: 2579 rp [2] sn: 5 count: 1000 metaSz: 2578 2023-10-21T04:09:41.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:41.477+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1088 2023-10-21T04:09:41.477+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 1000 metaSz: 2579 rp [1] sn: 5 count: 1000 metaSz: 2578 2023-10-21T04:09:41.477+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 64804 recovery UsedSpace 33366 2023-10-21T04:09:41.478+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 65894 recovery UsedSpace 39461 2023-10-21T04:09:41.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex 2023-10-21T04:09:41.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:41.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:41.483+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex 2023-10-21T04:09:41.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex started 2023-10-21T04:09:41.483+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.484+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex 2023-10-21T04:09:41.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:41.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:41.488+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex 2023-10-21T04:09:41.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex started 2023-10-21T04:09:41.488+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4030261110898422634 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:41.488+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:41.488+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:41.520+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4030261110898422634 PartitionId 0 Warmup took 44.03367ms 2023-10-21T04:09:41.520+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4030261110898422634 partitionId 0 WriterThreads 4 2023-10-21T04:09:41.520+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4030261110898422634 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:41.520+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4030261110898422634 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index Index: 4030261110898422634 Partition: 0 2023-10-21T04:09:41.520+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4030261110898422634, shardIdMap: map[0:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:41.520+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4030261110898422634, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:41.520+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:41.521+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4030261110898422634 2023-10-21T04:09:41.521+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:41.522+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:41.522+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:41.523+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:41.523+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:41.524+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4030261110898422634 Partitions [0] 2023-10-21T04:09:41.525+05:30 [Info] StorageMgr::openSnapshot IndexInst:4030261110898422634 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:41.538+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.146102ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:41.538+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 157.263µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:41.566+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 11400459767051699632 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.566+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 80911002600697866 RealInstId: 80911002600697866 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.567+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.567+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9664682240473767939 RealInstId: 9664682240473767939 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.567+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 14780363202971367008 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.567+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 17729593977321451597 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.567+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 17128008689111072444 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.567+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.568+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.568+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 15710672812788661570 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.568+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 18051244886420988269 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.568+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.568+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.569+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.569+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.569+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.569+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.569+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.569+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 3841645022501780362 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.569+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 5299824419646640495 RealInstId: 5299824419646640495 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 6796752657857722550 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 4004220452657849790 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 3292093078969249751 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 2170849387178940761 RealInstId: 2170849387178940761 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 16726320877690809560 RealInstId: 16726320877690809560 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 1532172649864318584 RealInstId: 1532172649864318584 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 3810060733504350542 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 11160011617777336020 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 17148287981272169168 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:41.570+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:09:41.645+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.645+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.697+05:30 [Info] switched currmeta from 335 -> 342 force true 2023-10-21T04:09:41.717+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.717+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.722+05:30 [Info] switched currmeta from 342 -> 342 force true 2023-10-21T04:09:41.839+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.840+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.858+05:30 [Info] switched currmeta from 342 -> 344 force true 2023-10-21T04:09:41.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.944+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.955+05:30 [Info] switched currmeta from 344 -> 347 force true 2023-10-21T04:09:41.972+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:41.974+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:41.987+05:30 [Info] switched currmeta from 347 -> 347 force true 2023-10-21T04:09:42.043+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[7597064108789640455:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:42.047+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7597064108789640455 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009d87800 2023-10-21T04:09:42.054+05:30 [Info] handleUpdateIndexRState: Index instance 7597064108789640455 rstate moved to ACTIVE 2023-10-21T04:09:42.056+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.056+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.065+05:30 [Info] switched currmeta from 347 -> 348 force true 2023-10-21T04:09:42.075+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.000819ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:42.077+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [14292583894719865678 573521267164877540 12338036531800130039] belonging to collectionId: 0, ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4. Initiating build 2023-10-21T04:09:42.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.125+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.143+05:30 [Info] switched currmeta from 348 -> 350 force true 2023-10-21T04:09:42.185+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.185+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.189+05:30 [Info] switched currmeta from 350 -> 351 force true 2023-10-21T04:09:42.195+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [3292093078969249751 1532172649864318584 3810060733504350542] &{2} 2023-10-21T04:09:42.196+05:30 [Info] Indexer::handleBuildRecoveredIndexes [3292093078969249751 1532172649864318584 3810060733504350542] 2023-10-21T04:09:42.196+05:30 [Info] bulkUpdateRState: Index instance 3292093078969249751 rstate moved to PENDING 2023-10-21T04:09:42.196+05:30 [Info] bulkUpdateRState: Index instance 1532172649864318584 rstate moved to PENDING 2023-10-21T04:09:42.196+05:30 [Info] bulkUpdateRState: Index instance 3810060733504350542 rstate moved to PENDING 2023-10-21T04:09:42.196+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [3292093078969249751 1532172649864318584 3810060733504350542] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:42.197+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:42.198+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:42.199+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:42.214+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.216+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.225+05:30 [Info] switched currmeta from 351 -> 352 force true 2023-10-21T04:09:42.232+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.233+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.236+05:30 [Info] switched currmeta from 352 -> 352 force true 2023-10-21T04:09:42.243+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1 2023-10-21T04:09:42.244+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:42.245+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:42.246+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:42.247+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:09:42.248+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:42.249+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:42.251+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:42.252+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:09:42.253+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:42.254+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:42.255+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:09:42.256+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:09:42.257+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1 SessionId 64671 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:42.258+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_1. OSO not supported for default scope/collection. 2023-10-21T04:09:42.258+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:42.259+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:42.260+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream INIT_STREAM 2023-10-21T04:09:42.261+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:6] INIT_STREAM:map[bucket_1:1]] 2023-10-21T04:09:42.261+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 1532172649864318584 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:42.265+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:6] INIT_STREAM:map[bucket_1:2]] 2023-10-21T04:09:42.266+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3292093078969249751 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:42.267+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:6] INIT_STREAM:map[bucket_1:3]] 2023-10-21T04:09:42.267+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3810060733504350542 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:42.268+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1 2023-10-21T04:09:42.269+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 64671 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:09:42.271+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:09:42.272+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:09:42.273+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:42.277+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 81.15349ms 2023-10-21T04:09:42.275+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:09:42.277+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [3292093078969249751 1532172649864318584 3810060733504350542] 2023-10-21T04:09:42.277+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 81.389528ms 2023-10-21T04:09:42.279+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1 Check pending stream merge. 2023-10-21T04:09:42.280+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 192.622418ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:42.284+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[1532172649864318584:true 3292093078969249751:true 3810060733504350542:true] 2023-10-21T04:09:42.297+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.330954ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:42.322+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.328+05:30 [Info] switched currmeta from 352 -> 353 force true 2023-10-21T04:09:42.339+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 42.710458ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:42.356+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.356+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.359+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.776064ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:42.375+05:30 [Info] switched currmeta from 353 -> 354 force true 2023-10-21T04:09:42.378+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59576" doAuth() ... 2023-10-21T04:09:42.379+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59576" auth successful 2023-10-21T04:09:42.379+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:59576" +1 2023-10-21T04:09:42.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.392+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.399+05:30 [Info] switched currmeta from 354 -> 355 force true 2023-10-21T04:09:42.463+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:09:42.515+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4030261110898422634:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:42.515+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 15968368593314043432, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:42.515+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 15968368593314043432, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:42.534+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 325705934323625324,realInstId: 0, indexDefn DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:42.534+05:30 [Info] Indexer::handleRecoverIndex InstId: 325705934323625324 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:42.535+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 311.811µs 2023-10-21T04:09:42.535+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 356.392µs 2023-10-21T04:09:42.535+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 325705934323625324 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:42.535+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 325705934323625324 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:42.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:42.535+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:42.536+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 19980 recovery UsedSpace 8236 2023-10-21T04:09:42.536+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 325705934323625324, indexDefn DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:42.536+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 21.365721ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:42.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:42.536+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:42.537+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 21005 recovery UsedSpace 8236 2023-10-21T04:09:42.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:42.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:42.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:42.541+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:42.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex started 2023-10-21T04:09:42.541+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:42.541+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:42.550+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.550+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.555+05:30 [Info] switched currmeta from 355 -> 356 force true 2023-10-21T04:09:42.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:42.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:42.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:42.575+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:42.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex started 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:42.579+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 325705934323625324 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:42.579+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 325705934323625324 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:42.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex] 2023-10-21T04:09:42.579+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex] 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex] 2023-10-21T04:09:42.579+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:42.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex stopped 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:42.579+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:42.587+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:42.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex] 2023-10-21T04:09:42.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex closed 2023-10-21T04:09:42.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:09:42.588+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:42.588+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:42.588+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:42.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex] 2023-10-21T04:09:42.588+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:42.588+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex] 2023-10-21T04:09:42.589+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex] 2023-10-21T04:09:42.589+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:42.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex stopped 2023-10-21T04:09:42.589+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:42.589+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:42.589+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:42.589+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:42.595+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:42.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex] 2023-10-21T04:09:42.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex closed 2023-10-21T04:09:42.596+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:42.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex ... 2023-10-21T04:09:42.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex ... 2023-10-21T04:09:42.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:42.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex sucessfully destroyed 2023-10-21T04:09:42.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex ... 2023-10-21T04:09:42.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex ... 2023-10-21T04:09:42.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:42.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index destroyed 2023-10-21T04:09:42.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:09:42.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:42.613+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:42.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#325705934323625324:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:42.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:42.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:42.616+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:42.616+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#325705934323625324:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:42.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 81 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:42.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:42.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:42.636+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:42.636+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex 2023-10-21T04:09:42.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/docIndex started 2023-10-21T04:09:42.636+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#325705934323625324:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:42.636+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#325705934323625324:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:42.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 90 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:42.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:42.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:42.638+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:42.638+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex 2023-10-21T04:09:42.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index/mainIndex started 2023-10-21T04:09:42.638+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 325705934323625324 PartitionId 0 Warmup took 102.554653ms 2023-10-21T04:09:42.638+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 325705934323625324 partitionId 0 WriterThreads 4 2023-10-21T04:09:42.638+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 325705934323625324 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:42.638+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 325705934323625324 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_325705934323625324_0.index Index: 325705934323625324 Partition: 0 2023-10-21T04:09:42.638+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 325705934323625324, shardIdMap: map[0:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:42.638+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 325705934323625324, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:42.638+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 325705934323625324 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:42.639+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#325705934323625324:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:42.639+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#325705934323625324:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:42.640+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 325705934323625324 2023-10-21T04:09:42.640+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 325705934323625324 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:42.640+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:42.641+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:42.641+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 325705934323625324 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:42.641+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 325705934323625324 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:42.641+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 325705934323625324 Partitions [0] 2023-10-21T04:09:42.642+05:30 [Info] StorageMgr::openSnapshot IndexInst:325705934323625324 Partition:0 No Snapshot Found. 2023-10-21T04:09:42.642+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 325705934323625324 Adding Nil Snapshot. 2023-10-21T04:09:42.661+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.013334ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:42.661+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 91.424µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:42.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.685+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.690+05:30 [Info] switched currmeta from 356 -> 357 force true 2023-10-21T04:09:42.910+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 0 186517626268899 55 64671 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:42.911+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 1 178840478062605 54 64671 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:42.916+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 3 215374859564337 35 64671 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:09:42.916+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 2 148075163102199 54 64671 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:42.925+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 4 28605411869539 58 64671 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:42.928+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 5 39552632604610 52 64671 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:42.928+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 6 242327631330054 62 64671 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:42.928+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 7 191903564840858 63 64671 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:42.929+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 8 50961953886801 45 64671 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:42.929+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 9 225280996447309 53 64671 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:42.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 10 274410200699189 54 64671 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:42.933+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 12 4429827396609 42 64671 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:09:42.937+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 11 66881023717063 51 64671 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:42.941+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 14 226814108379280 72 64671 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:09:42.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 17 55290066231400 51 64671 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:42.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 13 183148377680315 46 64671 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:42.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 18 242578104468179 63 64671 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:42.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 15 127185710837163 50 64671 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:42.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 16 257031483891250 55 64671 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:42.951+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 20 230239359684242 50 64671 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:42.951+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 19 210306088158507 48 64671 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:09:42.955+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 22 260036488774725 51 64671 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:42.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 21 209771427122473 47 64671 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:42.960+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 23 216462437971699 57 64671 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:42.960+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 24 160961818008256 59 64671 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:42.973+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:42.973+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:42.987+05:30 [Info] switched currmeta from 357 -> 358 force true 2023-10-21T04:09:43.014+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.017+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.025+05:30 [Info] switched currmeta from 358 -> 359 force true 2023-10-21T04:09:43.033+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 25 280874278672777 71 64671 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:09:43.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 26 219132834127088 52 64671 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:43.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 29 45957426777107 57 64671 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:43.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 27 184056281482149 49 64671 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:43.055+05:30 [Info] switched currmeta from 359 -> 360 force true 2023-10-21T04:09:43.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 30 99433130332778 58 64671 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:43.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 32 82469662608686 60 64671 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:43.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 36 253845164956592 58 64671 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:43.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 28 102578907139247 58 64671 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:43.065+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 39 222555662236724 67 64671 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:43.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 34 28417669987133 58 64671 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:43.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 37 253494845579900 54 64671 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:43.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 33 136958029391999 65 64671 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:09:43.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 31 217381278987664 67 64671 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:43.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 35 70213628416850 47 64671 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:43.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 38 188107826286720 60 64671 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:43.068+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 40 5915864305121 60 64671 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:43.083+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 42 252931432434966 46 64671 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:43.083+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 41 259381390638966 58 64671 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:43.084+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.084+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.091+05:30 [Info] switched currmeta from 360 -> 362 force true 2023-10-21T04:09:43.116+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 43 146267938201125 55 64671 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:43.116+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 44 158545118983293 67 64671 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:43.118+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 45 12301831552869 63 64671 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:43.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 49 215467668600630 34 64671 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:09:43.122+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 46 242994859770444 54 64671 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:43.122+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 47 271074273345339 53 64671 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:43.122+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 48 227378777358533 57 64671 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:43.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 51 200436537417141 54 64671 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:43.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 50 207367350302991 64 64671 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:43.124+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 52 188718367400378 56 64671 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:43.125+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 54 279362078165491 55 64671 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:43.125+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 53 204867593700407 56 64671 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:43.132+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 55 170056388266420 70 64671 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:09:43.132+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 57 129411666769864 55 64671 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:43.133+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.134+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.138+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 63 133748675726909 56 64671 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:43.138+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 60 75566139929862 56 64671 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:43.138+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 58 16044440367455 53 64671 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:43.138+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 56 264835102027079 52 64671 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:43.139+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 61 106932309485550 53 64671 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:43.139+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 59 125009072873642 53 64671 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:43.140+05:30 [Info] switched currmeta from 362 -> 363 force true 2023-10-21T04:09:43.141+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 62 10527297481171 57 64671 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:43.178+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1 InstanceIds [1532172649864318584 3292093078969249751 3810060733504350542] 2023-10-21T04:09:43.178+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1 Cid 0 SessionId 64671 2023-10-21T04:09:43.178+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 64671 2023-10-21T04:09:43.179+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:09:43.180+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:09:43.181+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 1532172649864318584 INDEX_STATE_CATCHUP 2023-10-21T04:09:43.182+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 3292093078969249751 INDEX_STATE_CATCHUP 2023-10-21T04:09:43.182+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 3810060733504350542 INDEX_STATE_CATCHUP 2023-10-21T04:09:43.183+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 1532172649864318584 true 2023-10-21T04:09:43.184+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 3292093078969249751 true 2023-10-21T04:09:43.184+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 3810060733504350542 true 2023-10-21T04:09:43.184+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 1532172649864318584 Stream: INIT_STREAM KeyspaceId: bucket_1 Session: 64671 BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:09:43.185+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:09:43.186+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1 2023-10-21T04:09:43.187+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1 2023-10-21T04:09:43.187+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:09:43.187+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 64671 STREAM_ACTIVE 2023-10-21T04:09:43.188+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [0=17,1=19,2=20,3=13,4=18,5=28,6=27,7=27,8=24,9=19,10=24,11=21,12=19,13=15,14=26,15=23,16=24,17=21,18=26,19=20,20=19,21=23,22=17,23=17,24=24,25=26,26=17,27=18,28=18,29=19,30=29,31=24,32=23,33=21,34=23,35=21,36=22,37=25,38=21,39=20,40=20,41=20,42=22,43=20,44=21,45=24,46=23,47=22,48=23,49=13,50=24,51=19,52=22,53=24,54=21,55=23,56=28,57=24,58=18,59=27,60=24,61=20,62=21,63=23] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:43.188+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1 Stream: INIT_STREAM SessionId: 64671 2023-10-21T04:09:43.188+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1 64671 2023-10-21T04:09:43.189+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:43.189+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:43.189+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:43.189+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:43.190+05:30 [Info] send progress stats to clients 2023-10-21T04:09:43.190+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1 set minMergeTs 2023-10-21T04:09:43.190+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:09:43.190+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:7] INIT_STREAM:map[bucket_1:3]] 2023-10-21T04:09:43.190+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:8] INIT_STREAM:map[bucket_1:3]] 2023-10-21T04:09:43.190+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:9] INIT_STREAM:map[bucket_1:3]] 2023-10-21T04:09:43.190+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 64671 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:09:43.196+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:09:43.198+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:09:43.200+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.725832ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:43.208+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 2023-10-21T04:09:43.208+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1 SessionId 64671 2023-10-21T04:09:43.208+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 64671 2023-10-21T04:09:43.208+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1 set minMergeTs 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 1532172649864318584 false 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 3292093078969249751 false 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 3810060733504350542 false 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1 1532172649864318584 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1 3292093078969249751 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1 3810060733504350542 2023-10-21T04:09:43.208+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1. 2023-10-21T04:09:43.208+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 64671 2023-10-21T04:09:43.209+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.216+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.912839ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:43.217+05:30 [Info] switched currmeta from 363 -> 364 force true 2023-10-21T04:09:43.240+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.240+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.244+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.893334ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:43.249+05:30 [Info] switched currmeta from 364 -> 366 force true 2023-10-21T04:09:43.275+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.275+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.282+05:30 [Info] switched currmeta from 366 -> 366 force true 2023-10-21T04:09:43.415+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.415+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.444+05:30 [Info] switched currmeta from 366 -> 367 force true 2023-10-21T04:09:43.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9826627125923982, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:09:43.465+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.472+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.475+05:30 [Info] switched currmeta from 367 -> 369 force true 2023-10-21T04:09:43.552+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[325705934323625324:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:43.552+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 325705934323625324 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00a0873e0 2023-10-21T04:09:43.553+05:30 [Info] handleUpdateIndexRState: Index instance 325705934323625324 rstate moved to ACTIVE 2023-10-21T04:09:43.565+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.937283ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:43.569+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 14340086159258068923, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:43.569+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 14340086159258068923, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:43.583+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.583+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.594+05:30 [Info] switched currmeta from 369 -> 370 force true 2023-10-21T04:09:43.601+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7410154703733445404,realInstId: 0, indexDefn DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:43.601+05:30 [Info] Indexer::handleRecoverIndex InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:09:43.601+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 426.886µs 2023-10-21T04:09:43.602+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 471.384µs 2023-10-21T04:09:43.602+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.602+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 8 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.602+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:09:43.602+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 148 metaSz: 2510 rp [1] sn: 3 count: 148 metaSz: 2575 rp [2] sn: 4 count: 148 metaSz: 2574 2023-10-21T04:09:43.602+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7410154703733445404, indexDefn DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:43.603+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 33.43357ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:43.603+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 38567 recovery UsedSpace 29584 2023-10-21T04:09:43.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.603+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:43.603+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 148 metaSz: 2575 rp [1] sn: 4 count: 148 metaSz: 2574 2023-10-21T04:09:43.604+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 39607 recovery UsedSpace 37480 2023-10-21T04:09:43.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex 2023-10-21T04:09:43.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.607+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex 2023-10-21T04:09:43.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/docIndex started 2023-10-21T04:09:43.607+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.607+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex 2023-10-21T04:09:43.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.622+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex 2023-10-21T04:09:43.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index/mainIndex started 2023-10-21T04:09:43.623+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:09:43.625+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.626+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.632+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.632+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.647+05:30 [Info] switched currmeta from 370 -> 371 force true 2023-10-21T04:09:43.651+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 8 Warmup took 49.069783ms 2023-10-21T04:09:43.651+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 8 WriterThreads 1 2023-10-21T04:09:43.651+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 8 Stopped Billing WUs 2023-10-21T04:09:43.651+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_8.index Index: 7410154703733445404 Partition: 8 2023-10-21T04:09:43.651+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.652+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 1 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.652+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:43.652+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 112 metaSz: 2510 rp [1] sn: 3 count: 112 metaSz: 2575 rp [2] sn: 4 count: 112 metaSz: 2574 2023-10-21T04:09:43.653+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 36140 recovery UsedSpace 29266 2023-10-21T04:09:43.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:43.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 112 metaSz: 2575 rp [1] sn: 4 count: 112 metaSz: 2574 2023-10-21T04:09:43.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 38026 recovery UsedSpace 37480 2023-10-21T04:09:43.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex 2023-10-21T04:09:43.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.660+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex 2023-10-21T04:09:43.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/mainIndex started 2023-10-21T04:09:43.660+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.660+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex 2023-10-21T04:09:43.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.676+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex 2023-10-21T04:09:43.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index/docIndex started 2023-10-21T04:09:43.676+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:09:43.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.691+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 1 Warmup took 39.236459ms 2023-10-21T04:09:43.691+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 1 WriterThreads 1 2023-10-21T04:09:43.691+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 1 Stopped Billing WUs 2023-10-21T04:09:43.691+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_1.index Index: 7410154703733445404 Partition: 1 2023-10-21T04:09:43.691+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.692+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 2 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.692+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:43.692+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 123 metaSz: 2575 rp [1] sn: 4 count: 123 metaSz: 2574 2023-10-21T04:09:43.692+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 38523 recovery UsedSpace 37480 2023-10-21T04:09:43.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.693+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:43.693+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 123 metaSz: 2510 rp [1] sn: 3 count: 123 metaSz: 2575 rp [2] sn: 4 count: 123 metaSz: 2574 2023-10-21T04:09:43.693+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 36572 recovery UsedSpace 29266 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.712+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/docIndex started 2023-10-21T04:09:43.712+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.712+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.712+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex 2023-10-21T04:09:43.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index/mainIndex started 2023-10-21T04:09:43.712+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:09:43.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.736+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 2 Warmup took 44.876363ms 2023-10-21T04:09:43.737+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 2 WriterThreads 1 2023-10-21T04:09:43.737+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 2 Stopped Billing WUs 2023-10-21T04:09:43.737+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_2.index Index: 7410154703733445404 Partition: 2 2023-10-21T04:09:43.737+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.737+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 3 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.737+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:43.737+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 123 metaSz: 2510 rp [1] sn: 3 count: 123 metaSz: 2575 rp [2] sn: 4 count: 123 metaSz: 2574 2023-10-21T04:09:43.738+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 37584 recovery UsedSpace 29266 2023-10-21T04:09:43.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.739+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:43.739+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 123 metaSz: 2575 rp [1] sn: 4 count: 123 metaSz: 2574 2023-10-21T04:09:43.739+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 39643 recovery UsedSpace 37480 2023-10-21T04:09:43.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex 2023-10-21T04:09:43.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.754+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex 2023-10-21T04:09:43.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/mainIndex started 2023-10-21T04:09:43.754+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.754+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex 2023-10-21T04:09:43.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.758+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex 2023-10-21T04:09:43.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index/docIndex started 2023-10-21T04:09:43.758+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:09:43.758+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.758+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.775+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 3 Warmup took 37.891512ms 2023-10-21T04:09:43.775+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 3 WriterThreads 1 2023-10-21T04:09:43.775+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 3 Stopped Billing WUs 2023-10-21T04:09:43.775+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_3.index Index: 7410154703733445404 Partition: 3 2023-10-21T04:09:43.775+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.776+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 4 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.776+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:09:43.776+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 138 metaSz: 2510 rp [1] sn: 3 count: 138 metaSz: 2575 rp [2] sn: 4 count: 138 metaSz: 2574 2023-10-21T04:09:43.777+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 39079 recovery UsedSpace 29584 2023-10-21T04:09:43.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:43.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 138 metaSz: 2575 rp [1] sn: 4 count: 138 metaSz: 2574 2023-10-21T04:09:43.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 40302 recovery UsedSpace 37480 2023-10-21T04:09:43.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex 2023-10-21T04:09:43.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.783+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex 2023-10-21T04:09:43.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/mainIndex started 2023-10-21T04:09:43.784+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.784+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex 2023-10-21T04:09:43.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.784+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex 2023-10-21T04:09:43.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index/docIndex started 2023-10-21T04:09:43.784+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:09:43.785+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.785+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.796+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 4 Warmup took 20.678539ms 2023-10-21T04:09:43.797+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 4 WriterThreads 1 2023-10-21T04:09:43.797+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 4 Stopped Billing WUs 2023-10-21T04:09:43.797+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_4.index Index: 7410154703733445404 Partition: 4 2023-10-21T04:09:43.797+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.797+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 5 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.797+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:09:43.797+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 117 metaSz: 2510 rp [1] sn: 3 count: 117 metaSz: 2575 rp [2] sn: 4 count: 117 metaSz: 2574 2023-10-21T04:09:43.798+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 38244 recovery UsedSpace 29266 2023-10-21T04:09:43.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.799+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:43.799+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 117 metaSz: 2575 rp [1] sn: 4 count: 117 metaSz: 2574 2023-10-21T04:09:43.799+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 39227 recovery UsedSpace 37480 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/mainIndex started 2023-10-21T04:09:43.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex 2023-10-21T04:09:43.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index/docIndex started 2023-10-21T04:09:43.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.808+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:09:43.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.808+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.808+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.843+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 5 Warmup took 46.3673ms 2023-10-21T04:09:43.844+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 5 WriterThreads 1 2023-10-21T04:09:43.844+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 5 Stopped Billing WUs 2023-10-21T04:09:43.844+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_5.index Index: 7410154703733445404 Partition: 5 2023-10-21T04:09:43.844+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.844+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 6 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.844+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:09:43.844+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 137 metaSz: 2510 rp [1] sn: 3 count: 137 metaSz: 2575 rp [2] sn: 4 count: 137 metaSz: 2574 2023-10-21T04:09:43.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.845+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:09:43.845+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 137 metaSz: 2575 rp [1] sn: 4 count: 137 metaSz: 2574 2023-10-21T04:09:43.845+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 40034 recovery UsedSpace 29584 2023-10-21T04:09:43.846+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 41408 recovery UsedSpace 37480 2023-10-21T04:09:43.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex 2023-10-21T04:09:43.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex 2023-10-21T04:09:43.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/docIndex started 2023-10-21T04:09:43.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex 2023-10-21T04:09:43.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.857+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex 2023-10-21T04:09:43.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index/mainIndex started 2023-10-21T04:09:43.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.857+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:09:43.873+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 6 Warmup took 28.65493ms 2023-10-21T04:09:43.873+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 6 WriterThreads 1 2023-10-21T04:09:43.873+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 6 Stopped Billing WUs 2023-10-21T04:09:43.873+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_6.index Index: 7410154703733445404 Partition: 6 2023-10-21T04:09:43.873+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7410154703733445404 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:43.873+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7410154703733445404 PartitionId: 7 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:43.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:09:43.874+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:09:43.874+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 2 count: 102 metaSz: 2509 rp [1] sn: 3 count: 102 metaSz: 2574 rp [2] sn: 4 count: 102 metaSz: 2573 2023-10-21T04:09:43.874+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 38551 recovery UsedSpace 29266 2023-10-21T04:09:43.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:43.875+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:09:43.875+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 102 metaSz: 2574 rp [1] sn: 4 count: 102 metaSz: 2573 2023-10-21T04:09:43.876+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 39591 recovery UsedSpace 37480 2023-10-21T04:09:43.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex 2023-10-21T04:09:43.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:43.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:43.888+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex 2023-10-21T04:09:43.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/mainIndex started 2023-10-21T04:09:43.888+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.888+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex 2023-10-21T04:09:43.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:09:43.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:09:43.889+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex 2023-10-21T04:09:43.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index/docIndex started 2023-10-21T04:09:43.890+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:09:43.890+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:43.890+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:43.911+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7410154703733445404 PartitionId 7 Warmup took 38.139424ms 2023-10-21T04:09:43.913+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7410154703733445404 partitionId 7 WriterThreads 1 2023-10-21T04:09:43.913+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7410154703733445404 PartitionId 7 Stopped Billing WUs 2023-10-21T04:09:43.914+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7410154703733445404 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_7410154703733445404_7.index Index: 7410154703733445404 Partition: 7 2023-10-21T04:09:43.914+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7410154703733445404, shardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]] err: 2023-10-21T04:09:43.915+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7410154703733445404, partnShardIdMap: map[1:[6611724767191156837 15964300471522801617] 2:[6611724767191156837 15964300471522801617] 3:[6611724767191156837 15964300471522801617] 4:[6611724767191156837 15964300471522801617] 5:[6611724767191156837 15964300471522801617] 6:[6611724767191156837 15964300471522801617] 7:[6611724767191156837 15964300471522801617] 8:[6611724767191156837 15964300471522801617]], err: 2023-10-21T04:09:43.915+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:43.916+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7410154703733445404 2023-10-21T04:09:43.916+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:43.917+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:09:43.917+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:43.918+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:43.918+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:43.918+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7410154703733445404 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:09:43.919+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:3 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:09:43.920+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:4 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:09:43.922+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:5 Attempting to open snapshot (SnapshotInfo: count:117 committed:false) 2023-10-21T04:09:43.923+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:6 Attempting to open snapshot (SnapshotInfo: count:137 committed:false) 2023-10-21T04:09:43.924+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:7 Attempting to open snapshot (SnapshotInfo: count:102 committed:false) 2023-10-21T04:09:43.926+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:8 Attempting to open snapshot (SnapshotInfo: count:148 committed:false) 2023-10-21T04:09:43.927+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:1 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:09:43.928+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.127282ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:43.931+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 111.049µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:43.935+05:30 [Info] StorageMgr::openSnapshot IndexInst:7410154703733445404 Partition:2 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:09:43.938+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:43.938+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:43.941+05:30 [Info] switched currmeta from 371 -> 372 force true 2023-10-21T04:09:44.057+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.057+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.076+05:30 [Info] switched currmeta from 372 -> 376 force true 2023-10-21T04:09:44.123+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.123+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.153+05:30 [Info] switched currmeta from 376 -> 378 force true 2023-10-21T04:09:44.176+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.178+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.190+05:30 [Info] switched currmeta from 378 -> 378 force true 2023-10-21T04:09:44.191+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:09:44.192+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1 2023-10-21T04:09:44.195+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 1532172649864318584 Stream: INIT_STREAM KeyspaceId: bucket_1 SessionId: 64671, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:44.197+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 1532172649864318584 INDEX_STATE_ACTIVE 2023-10-21T04:09:44.197+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 3292093078969249751 INDEX_STATE_ACTIVE 2023-10-21T04:09:44.197+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 3810060733504350542 INDEX_STATE_ACTIVE 2023-10-21T04:09:44.197+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1 State Changed to INACTIVE 2023-10-21T04:09:44.197+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:09:44.197+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1 Stream: INIT_STREAM SessionId: 64671 2023-10-21T04:09:44.198+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:44.198+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:44.199+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:44.199+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:44.199+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1532172649864318584 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 3292093078969249751 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 3810060733504350542 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:44.199+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 64671 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:44.199+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3292093078969249751 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:09:44.199+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3810060733504350542 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:09:44.199+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 1532172649864318584 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:09:44.199+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:09:44.199+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:44.199+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:44.199+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 64671 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:44.200+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:09:44.200+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:44.200+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:44.200+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:09:44.200+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:59576: use of closed network connection 2023-10-21T04:09:44.200+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:59576" exit: read tcp4 127.0.0.1:9121->127.0.0.1:59576: use of closed network connection 2023-10-21T04:09:44.200+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59576" closed ! 2023-10-21T04:09:44.200+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:09:44.200+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:59576" already gone 2023-10-21T04:09:44.201+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:44.201+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:44.201+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:44.201+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:44.201+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:09:44.201+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:44.203+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1 Stream: INIT_STREAM SessionId 64671 2023-10-21T04:09:44.204+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1] 2023-10-21T04:09:44.220+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.158948ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.252+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 31.236591ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.259+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1]. Err genServer.closed 2023-10-21T04:09:44.259+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:09:44.259+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Treating genServer.closed As Success 2023-10-21T04:09:44.259+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1 SessionId 64671 2023-10-21T04:09:44.259+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 64671 2023-10-21T04:09:44.259+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:09:44.271+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.016854ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.330+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3292093078969249751 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005b723c0 2023-10-21T04:09:44.330+05:30 [Info] handleUpdateIndexRState: Index instance 3292093078969249751 rstate moved to ACTIVE 2023-10-21T04:09:44.337+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.020083ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.338+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3810060733504350542 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005b724e0 2023-10-21T04:09:44.338+05:30 [Info] handleUpdateIndexRState: Index instance 3810060733504350542 rstate moved to ACTIVE 2023-10-21T04:09:44.338+05:30 [Info] Indexer::handleMergePartition Source 4164706166433630808 Target 1532172649864318584 2023-10-21T04:09:44.338+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1532172649864318584 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00f674420 2023-10-21T04:09:44.338+05:30 [Info] MergePartition: sent async request to update index instance 1532172649864318584 rstate moved to ACTIVE 2023-10-21T04:09:44.346+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.90304ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.348+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 13258659765240018889. Retrying... 2023-10-21T04:09:44.348+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[1532172649864318584:true 3292093078969249751:true 3810060733504350542:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:44.370+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.584707ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.370+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 4164706166433630808, realInst: 1532172649864318584 2023-10-21T04:09:44.385+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 12833670934079048728, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:44.385+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 12833670934079048728, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:09:44.448+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 8] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 6] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:44.448+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:44.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9738090428189041, throttleDelayMs (new, old, change): (5, 3, 2) 2023-10-21T04:09:44.454+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 13258659765240018889,realInstId: 0, indexDefn DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:44.454+05:30 [Info] Indexer::handleRecoverIndex InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:44.454+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 154.505µs 2023-10-21T04:09:44.454+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 200.913µs 2023-10-21T04:09:44.454+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13258659765240018889 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10563635432419104314 3268862397300335529]] 2023-10-21T04:09:44.454+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13258659765240018889 PartitionId: 0 using shardIds: [10563635432419104314 3268862397300335529], rebalance running: true 2023-10-21T04:09:44.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:09:44.454+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 64 2023-10-21T04:09:44.455+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 11 count: 14 metaSz: 2549 rp [1] sn: 15 count: 21 metaSz: 2549 2023-10-21T04:09:44.455+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: doInit: data UsedSpace 36442 recovery UsedSpace 37596 2023-10-21T04:09:44.458+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 13258659765240018889, indexDefn DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:44.459+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 74.085371ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.458+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 9] 2023-10-21T04:09:44.458+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:09:44.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex 2023-10-21T04:09:44.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:09:44.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:09:44.466+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex 2023-10-21T04:09:44.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index/mainIndex started 2023-10-21T04:09:44.466+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13258659765240018889 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:44.467+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:44.468+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:44.485+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13258659765240018889 PartitionId 0 Warmup took 30.95634ms 2023-10-21T04:09:44.485+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13258659765240018889 partitionId 0 WriterThreads 4 2023-10-21T04:09:44.485+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13258659765240018889 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_13258659765240018889_0.index Index: 13258659765240018889 Partition: 0 2023-10-21T04:09:44.485+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 13258659765240018889, shardIdMap: map[0:[10563635432419104314]] err: 2023-10-21T04:09:44.485+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 13258659765240018889, partnShardIdMap: map[0:[10563635432419104314]], err: 2023-10-21T04:09:44.485+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:44.486+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13258659765240018889 2023-10-21T04:09:44.486+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:44.486+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:44.487+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:44.487+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:44.487+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:44.487+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13258659765240018889 Partitions [0] 2023-10-21T04:09:44.488+05:30 [Info] StorageMgr::openSnapshot IndexInst:13258659765240018889 Partition:0 Attempting to open snapshot (SnapshotInfo: count:21 committed:false) 2023-10-21T04:09:44.499+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.216854ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.500+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 77.773µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:44.596+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.597+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.610+05:30 [Info] switched currmeta from 378 -> 386 force true 2023-10-21T04:09:44.616+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.616+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.619+05:30 [Info] switched currmeta from 386 -> 386 force true 2023-10-21T04:09:44.646+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4030261110898422634:INDEX_STATE_RECOVERED 7410154703733445404:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:44.647+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 5197198800322775808, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:44.647+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 5197198800322775808, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:44.695+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14607307599259773249,realInstId: 0, indexDefn DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:44.695+05:30 [Info] Indexer::handleRecoverIndex InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:44.695+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 164.09µs 2023-10-21T04:09:44.695+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 211.698µs 2023-10-21T04:09:44.695+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14607307599259773249 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:44.696+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14607307599259773249 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:44.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:44.696+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1054712 memSz 772 2023-10-21T04:09:44.696+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 3 count: 1000 metaSz: 2555 rp [1] sn: 4 count: 1000 metaSz: 2554 2023-10-21T04:09:44.697+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 64088 recovery UsedSpace 38907 2023-10-21T04:09:44.697+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14607307599259773249, indexDefn DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:44.697+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 50.683352ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex 2023-10-21T04:09:44.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:44.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:44.720+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex 2023-10-21T04:09:44.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index/mainIndex started 2023-10-21T04:09:44.720+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14607307599259773249 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:44.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:44.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:44.754+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.754+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.770+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:44.774+05:30 [Info] switched currmeta from 386 -> 387 force true 2023-10-21T04:09:44.789+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.789+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.794+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14607307599259773249 PartitionId 0 Warmup took 97.841604ms 2023-10-21T04:09:44.794+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14607307599259773249 partitionId 0 WriterThreads 4 2023-10-21T04:09:44.794+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14607307599259773249 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:44.794+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14607307599259773249 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_14607307599259773249_0.index Index: 14607307599259773249 Partition: 0 2023-10-21T04:09:44.794+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14607307599259773249, shardIdMap: map[0:[6611724767191156837]] err: 2023-10-21T04:09:44.794+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14607307599259773249, partnShardIdMap: map[0:[6611724767191156837]], err: 2023-10-21T04:09:44.794+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:44.795+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14607307599259773249 2023-10-21T04:09:44.795+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:44.795+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:44.795+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:44.795+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:44.799+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:44.799+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14607307599259773249 Partitions [0] 2023-10-21T04:09:44.800+05:30 [Info] StorageMgr::openSnapshot IndexInst:14607307599259773249 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:09:44.803+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 4.314502ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:44.803+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 74.132µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:44.829+05:30 [Info] switched currmeta from 387 -> 388 force true 2023-10-21T04:09:44.838+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.839+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.844+05:30 [Info] switched currmeta from 388 -> 388 force true 2023-10-21T04:09:44.914+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.914+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.922+05:30 [Info] switched currmeta from 388 -> 392 force true 2023-10-21T04:09:44.926+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.926+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.930+05:30 [Info] switched currmeta from 392 -> 392 force true 2023-10-21T04:09:44.956+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:44.958+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:44.966+05:30 [Info] switched currmeta from 392 -> 393 force true 2023-10-21T04:09:45.103+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:45.103+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:45.108+05:30 [Info] switched currmeta from 393 -> 396 force true 2023-10-21T04:09:45.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:45.125+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:45.130+05:30 [Info] switched currmeta from 396 -> 396 force true 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6] 6796752657857722550:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6] 6796752657857722550:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.220+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3841645022501780362:[0] 5299824419646640495:[7 8 1 2 3 4 5 6] 6796752657857722550:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:45.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9243080231554994, throttleDelayMs (new, old, change): (2, 5, -3) 2023-10-21T04:09:45.471+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[13258659765240018889:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:45.471+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [12833670934079048728] belonging to collectionId: 9, ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4. Initiating build 2023-10-21T04:09:45.479+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [13258659765240018889] &{2} 2023-10-21T04:09:45.479+05:30 [Info] Indexer::handleBuildRecoveredIndexes [13258659765240018889] 2023-10-21T04:09:45.479+05:30 [Info] bulkUpdateRState: Index instance 13258659765240018889 rstate moved to PENDING 2023-10-21T04:09:45.479+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [13258659765240018889] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:45.480+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:45.480+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:45.480+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:45.481+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:09:45.482+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:45.482+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:45.482+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:45.482+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:09:45.482+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:09:45.482+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:45.482+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:45.482+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:09:45.482+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:45.482+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:45.482+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:09:45.482+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:09:45.482+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 36636 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:45.482+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:45.482+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:45.482+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:09:45.482+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:9] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:09:45.482+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 13258659765240018889 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:45.482+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:09:45.482+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 36636 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:09:45.483+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:09:45.483+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:09:45.484+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:45.484+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 4.4701ms 2023-10-21T04:09:45.484+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 4.507017ms 2023-10-21T04:09:45.484+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [13258659765240018889] 2023-10-21T04:09:45.484+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 12.140326ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:45.486+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[13258659765240018889:true] 2023-10-21T04:09:45.494+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:09:45.495+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.431868ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:45.497+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:45.513+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:45.521+05:30 [Info] switched currmeta from 396 -> 398 force true 2023-10-21T04:09:45.533+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:45.533+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:45.537+05:30 [Info] switched currmeta from 398 -> 398 force true 2023-10-21T04:09:45.592+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.592+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.592+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.593+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.593+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.593+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.593+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.593+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.594+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.594+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[2 3 4 5 6 7 8 1] 3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.594+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[2 3 4 5 6 7 8 1] 3292093078969249751:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.594+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[2 3 4 5 6 7 8 1] 3292093078969249751:[0] 3810060733504350542:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.594+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[2 3 4 5 6 7 8 1] 3292093078969249751:[0] 3810060733504350542:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.594+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[2 3 4 5 6 7 8 1] 3292093078969249751:[0] 3810060733504350542:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.594+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1532172649864318584:[2 3 4 5 6 7 8 1] 3292093078969249751:[0] 3810060733504350542:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:45.706+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4030261110898422634:INDEX_STATE_RECOVERED 7410154703733445404:INDEX_STATE_RECOVERED 14607307599259773249:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:09:45.707+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 17878371269610391239, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:45.707+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 17878371269610391239, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:09:45.723+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15131823346681830059,realInstId: 0, indexDefn DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:09:45.725+05:30 [Info] Indexer::handleRecoverIndex InstId: 15131823346681830059 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:09:45.725+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 186.124µs 2023-10-21T04:09:45.725+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15131823346681830059 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[6611724767191156837 15964300471522801617]] 2023-10-21T04:09:45.725+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15131823346681830059 PartitionId: 0 using shardIds: [6611724767191156837 15964300471522801617], rebalance running: true 2023-10-21T04:09:45.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:45.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:09:45.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: doInit: data UsedSpace 23405 recovery UsedSpace 8236 2023-10-21T04:09:45.727+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15131823346681830059, indexDefn DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:09:45.727+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 20.801014ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:45.728+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 230.267µs 2023-10-21T04:09:45.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:45.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:45.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:45.734+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:45.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex started 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:45.734+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15131823346681830059 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:09:45.734+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15131823346681830059 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:09:45.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex] 2023-10-21T04:09:45.734+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex] 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex] 2023-10-21T04:09:45.734+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:45.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex stopped 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:09:45.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:09:45.740+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:09:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex] 2023-10-21T04:09:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex closed 2023-10-21T04:09:45.740+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:09:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex ... 2023-10-21T04:09:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex ... 2023-10-21T04:09:45.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:45.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index destroyed 2023-10-21T04:09:45.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:09:45.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:09:45.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:09:45.757+05:30 [Info] bucket_%2/#primary_defer/Mainstore#15131823346681830059:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:09:45.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 91 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:45.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:09:45.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:09:45.769+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:45.770+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex 2023-10-21T04:09:45.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index/mainIndex started 2023-10-21T04:09:45.770+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15131823346681830059 PartitionId 0 Warmup took 44.570756ms 2023-10-21T04:09:45.770+05:30 [Info] bucket_%2/#primary_defer/Mainstore#15131823346681830059:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:09:45.770+05:30 [Info] bucket_%2/#primary_defer/Mainstore#15131823346681830059:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:09:45.770+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15131823346681830059 partitionId 0 WriterThreads 4 2023-10-21T04:09:45.771+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15131823346681830059 PartitionId 0 Stopped Billing WUs 2023-10-21T04:09:45.771+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15131823346681830059 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_15131823346681830059_0.index Index: 15131823346681830059 Partition: 0 2023-10-21T04:09:45.771+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15131823346681830059, shardIdMap: map[0:[6611724767191156837]] err: 2023-10-21T04:09:45.771+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15131823346681830059, partnShardIdMap: map[0:[6611724767191156837]], err: 2023-10-21T04:09:45.771+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15131823346681830059 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:45.773+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15131823346681830059 2023-10-21T04:09:45.773+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15131823346681830059 Name: #primary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:45.774+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:09:45.774+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:45.775+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15131823346681830059 Name: #primary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:45.775+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15131823346681830059 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:45.776+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15131823346681830059 Partitions [0] 2023-10-21T04:09:45.776+05:30 [Info] StorageMgr::openSnapshot IndexInst:15131823346681830059 Partition:0 No Snapshot Found. 2023-10-21T04:09:45.798+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15131823346681830059 Adding Nil Snapshot. 2023-10-21T04:09:45.777+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 4.803553ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:45.779+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:45.799+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:45.799+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 129.112µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:45.808+05:30 [Info] switched currmeta from 398 -> 400 force true 2023-10-21T04:09:45.812+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:45.812+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:45.815+05:30 [Info] switched currmeta from 400 -> 400 force true 2023-10-21T04:09:45.959+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:45.959+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:45.965+05:30 [Info] switched currmeta from 400 -> 401 force true 2023-10-21T04:09:46.007+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:46.007+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:46.014+05:30 [Info] switched currmeta from 401 -> 402 force true 2023-10-21T04:09:46.072+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.072+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.072+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 3841645022501780362 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 5299824419646640495 RealInstId: 5299824419646640495 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 6796752657857722550 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 4894899676975150934 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 18201836893981255880 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 4004220452657849790 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 3292093078969249751 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 2170849387178940761 RealInstId: 2170849387178940761 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 16726320877690809560 RealInstId: 16726320877690809560 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 1532172649864318584 RealInstId: 1532172649864318584 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 3810060733504350542 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 11160011617777336020 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:#primary Progress: 100 InstId: 17148287981272169168 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 13258659765240018889 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.073+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 11400459767051699632 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 80911002600697866 RealInstId: 80911002600697866 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 4030261110898422634 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 2483169159760864439 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 7410154703733445404 RealInstId: 7410154703733445404 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9664682240473767939 RealInstId: 9664682240473767939 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 14607307599259773249 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 14780363202971367008 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.074+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 17729593977321451597 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.075+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 17128008689111072444 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:09:46.075+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.075+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.075+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 15710672812788661570 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.075+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 18051244886420988269 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.075+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.075+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.082+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.082+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.082+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 16398781184179534607 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:09:46.082+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:09:46.458+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:46.458+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:46.463+05:30 [Info] switched currmeta from 402 -> 403 force true 2023-10-21T04:09:46.470+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7009010741701147, throttleDelayMs (new, old, change): (0, 2, -2) 2023-10-21T04:09:46.494+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:09:46.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:46.527+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:46.531+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59760" doAuth() ... 2023-10-21T04:09:46.535+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59760" auth successful 2023-10-21T04:09:46.535+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:59760" +1 2023-10-21T04:09:46.540+05:30 [Info] switched currmeta from 403 -> 406 force true 2023-10-21T04:09:46.551+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:46.551+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:46.557+05:30 [Info] switched currmeta from 406 -> 407 force true 2023-10-21T04:09:46.723+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.728+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.729+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.731+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.736+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.738+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.738+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.740+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.740+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.740+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.742+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.742+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.742+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.742+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.743+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.743+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.743+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:46.772+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[15131823346681830059:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:09:46.772+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15131823346681830059 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc000182300 2023-10-21T04:09:46.773+05:30 [Info] handleUpdateIndexRState: Index instance 15131823346681830059 rstate moved to ACTIVE 2023-10-21T04:09:46.785+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.634295ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:46.785+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [4860549750398470299 14340086159258068923 5197198800322775808] belonging to collectionId: c, ttid: ShardToken80:53:e0:a8:fa:c4:c4:85. Initiating build 2023-10-21T04:09:46.843+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [4030261110898422634 7410154703733445404 14607307599259773249] &{2} 2023-10-21T04:09:46.844+05:30 [Info] Indexer::handleBuildRecoveredIndexes [4030261110898422634 7410154703733445404 14607307599259773249] 2023-10-21T04:09:46.844+05:30 [Info] bulkUpdateRState: Index instance 4030261110898422634 rstate moved to PENDING 2023-10-21T04:09:46.844+05:30 [Info] bulkUpdateRState: Index instance 7410154703733445404 rstate moved to PENDING 2023-10-21T04:09:46.844+05:30 [Info] bulkUpdateRState: Index instance 14607307599259773249 rstate moved to PENDING 2023-10-21T04:09:46.845+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [4030261110898422634 7410154703733445404 14607307599259773249] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:09:46.847+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:46.848+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:46.849+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:46.885+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:46.885+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:46.907+05:30 [Info] switched currmeta from 407 -> 411 force true 2023-10-21T04:09:46.921+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:46.921+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:46.925+05:30 [Info] switched currmeta from 411 -> 411 force true 2023-10-21T04:09:46.944+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:09:46.957+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:46.961+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:46.964+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:46.971+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:46.975+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:46.976+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:09:46.977+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:46.977+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:46.977+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:46.978+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_system:_query:{0xc004792840 }] 2023-10-21T04:09:46.978+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:46.979+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:46.980+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:46.981+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:46.983+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:09:46.984+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:09:46.986+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId 11732 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:46.986+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:46.986+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:46.986+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:46.986+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:46.987+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:09:46.987+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:9] INIT_STREAM:map[bucket_%2:_default:c1:1 bucket_1:_system:_query:1]] 2023-10-21T04:09:46.988+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 7410154703733445404 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:46.988+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:9] INIT_STREAM:map[bucket_%2:_default:c1:2 bucket_1:_system:_query:1]] 2023-10-21T04:09:46.989+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4030261110898422634 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:46.989+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:9] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_system:_query:1]] 2023-10-21T04:09:46.989+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 14607307599259773249 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:09:46.990+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:09:46.990+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 11732 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:09:46.995+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc00d15a900 } bucket_1:_system:_query:{0xc004792840 }] 2023-10-21T04:09:46.998+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:09:46.996+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:47.000+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 155.911366ms 2023-10-21T04:09:47.001+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [4030261110898422634 7410154703733445404 14607307599259773249] 2023-10-21T04:09:47.001+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 212.670188ms len(expediates) 1 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:09:47.001+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 157.234312ms 2023-10-21T04:09:47.001+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:09:47.000+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 1 178840478062605 54 36636 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.003+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4030261110898422634:true 7410154703733445404:true 14607307599259773249:true] 2023-10-21T04:09:47.005+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 2 148075163102199 54 36636 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.007+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.005293ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:47.006+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 3 215374859564337 35 36636 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:09:47.012+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 5 39552632604610 52 36636 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:47.013+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 6 242327631330054 62 36636 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:47.014+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 7 191903564840858 63 36636 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:47.014+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 9 225280996447309 53 36636 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:47.015+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 11 66881023717063 51 36636 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:47.015+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 10 274410200699189 54 36636 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.016+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 4 28605411869539 58 36636 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.017+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 0 186517626268899 55 36636 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.018+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 8 50961953886801 45 36636 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:47.019+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 13 183148377680315 46 36636 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:47.020+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 14 226814108379280 72 36636 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:09:47.023+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 15 127185710837163 50 36636 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:47.023+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 17 55290066231400 51 36636 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:47.025+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 12 4429827396609 42 36636 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:09:47.026+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 18 242578104468179 63 36636 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:47.028+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 16 257031483891250 55 36636 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.029+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 19 210306088158507 48 36636 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:09:47.029+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 21 209771427122473 47 36636 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:47.029+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 20 230239359684242 50 36636 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:47.030+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 22 260036488774725 51 36636 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:47.031+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 23 216462437971699 57 36636 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:47.031+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 26 219132834127088 52 36636 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:47.033+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 24 160961818008256 59 36636 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:47.033+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 25 280874278672777 71 36636 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:09:47.034+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 27 184056281482149 49 36636 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:47.034+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 28 102578907139247 58 36636 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.035+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 28.094817ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:47.035+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 29 45957426777107 57 36636 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:47.035+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 31 217381278987664 67 36636 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:47.037+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 30 99433130332778 58 36636 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.037+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 33 136958029391999 65 36636 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 32 82469662608686 60 36636 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 34 28417669987133 58 36636 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 36 253845164956592 58 36636 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 35 70213628416850 47 36636 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 37 253494845579900 54 36636 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 39 222555662236724 67 36636 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 38 188107826286720 60 36636 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 40 5915864305121 60 36636 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 41 259381390638966 58 36636 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 43 146267938201125 55 36636 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 42 252931432434966 46 36636 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 44 158545118983293 67 36636 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 45 12301831552869 63 36636 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 46 242994859770444 54 36636 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 47 271074273345339 53 36636 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 48 227378777358533 57 36636 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 49 215467668600630 34 36636 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 50 207367350302991 64 36636 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 51 200436537417141 54 36636 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 52 188718367400378 56 36636 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:47.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 54 279362078165491 55 36636 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 53 204867593700407 56 36636 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 55 170056388266420 70 36636 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 57 129411666769864 55 36636 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 56 264835102027079 52 36636 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 60 75566139929862 56 36636 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 58 16044440367455 53 36636 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 59 125009072873642 53 36636 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 63 133748675726909 56 36636 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 61 106932309485550 53 36636 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:47.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 62 10527297481171 57 36636 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:47.038+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_system:_query InstanceIds [13258659765240018889] 2023-10-21T04:09:47.040+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 36636 2023-10-21T04:09:47.040+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 36636 2023-10-21T04:09:47.040+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:09:47.040+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:09:47.040+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 13258659765240018889 INDEX_STATE_CATCHUP 2023-10-21T04:09:47.040+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 13258659765240018889 true 2023-10-21T04:09:47.040+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 13258659765240018889 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query Session: 36636 BuildTS: [0=55,1=54,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=72,15=50,16=55,17=51,18=63,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:09:47.053+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_system:_query, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:09:47.053+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:09:47.053+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:09:47.053+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:09:47.053+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 36636 STREAM_ACTIVE 2023-10-21T04:09:47.053+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=4,1=54,2=4,3=4,4=4,5=29,6=4,7=4,8=4,9=4,10=4,11=4,12=4,13=4,14=72,15=4,16=36,17=4,18=63,19=4,20=4,21=33,22=4,23=20,24=37,25=4,26=4,27=4,28=4,29=57,30=4,31=4,32=4,33=25,34=4,35=4,36=4,37=4,38=4,39=4,40=60,41=4,42=4,43=35,44=4,45=45,46=24,47=4,48=24,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=53,59=4,60=4,61=21,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:47.053+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 36636 2023-10-21T04:09:47.053+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_system:_query 36636 2023-10-21T04:09:47.054+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:47.054+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:47.054+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:47.054+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:47.055+05:30 [Info] send progress stats to clients 2023-10-21T04:09:47.055+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:09:47.055+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:09:47.055+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:10] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_system:_query:1]] 2023-10-21T04:09:47.055+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 36636 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:09:47.071+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 35.74876ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:47.089+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:47.089+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:47.092+05:30 [Info] switched currmeta from 411 -> 414 force true 2023-10-21T04:09:47.105+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:09:47.115+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 2023-10-21T04:09:47.115+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 36636 2023-10-21T04:09:47.115+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 36636 2023-10-21T04:09:47.115+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:09:47.115+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:09:47.115+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 13258659765240018889 false 2023-10-21T04:09:47.115+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_system:_query 13258659765240018889 2023-10-21T04:09:47.115+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_system:_query. 2023-10-21T04:09:47.115+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 36636 2023-10-21T04:09:47.123+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:47.123+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:47.131+05:30 [Info] switched currmeta from 414 -> 414 force true 2023-10-21T04:09:47.137+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 66.075787ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:47.142+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:47.142+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:47.149+05:30 [Info] switched currmeta from 414 -> 415 force true 2023-10-21T04:09:47.576+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:47.576+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:47.583+05:30 [Info] switched currmeta from 415 -> 416 force true 2023-10-21T04:09:47.610+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:47.611+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:47.615+05:30 [Info] switched currmeta from 416 -> 417 force true 2023-10-21T04:09:47.686+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 263268055970459 52 11732 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:47.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 176639880977228 58 11732 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.690+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 35403626585217 50 11732 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:47.692+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 159113185399719 58 11732 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.692+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 215749515705641 45 11732 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:47.694+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 3874519919278 54 11732 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 261493429196201 60 11732 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 170288557775498 52 11732 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:47.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 236398693961229 62 11732 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:09:47.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 264975169139761 56 11732 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:47.702+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 206533515052028 50 11732 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:47.702+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 234758907181738 56 11732 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:09:47.707+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 116104661063478 55 11732 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.707+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 102350060311343 53 11732 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:47.710+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 162158720181435 67 11732 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:47.710+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 140236896721046 60 11732 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.713+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 200752780087280 51 11732 172.31.5.112:9000. HWT [49-51,51]. 2023-10-21T04:09:47.714+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 98820809009123 44 11732 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:09:47.722+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 208836052671833 45 11732 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:09:47.724+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 207134287529256 46 11732 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:47.724+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 92956357968546 50 11732 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:47.724+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 27979933591532 54 11732 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.727+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 192215088742243 55 11732 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.727+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 154934968850942 51 11732 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:47.731+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 271667630317209 52 11732 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:47.731+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 168740244208559 59 11732 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:47.733+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 120989102157727 59 11732 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:47.737+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5715487545729 54 11732 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.737+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 236953356346174 54 11732 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.742+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 130231725986856 49 11732 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:09:47.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 60545336618717 46 11732 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:47.746+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 73432771436458 60 11732 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.748+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 67362620867705 67 11732 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:47.750+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 278426329668882 54 11732 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.752+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 51776324008509 60 11732 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.754+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 24368176313355 50 11732 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:09:47.756+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 43594649874772 52 11732 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:09:47.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 83647976044922 59 11732 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:09:47.760+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 181267443431019 53 11732 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:09:47.762+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 100659545510465 49 11732 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:09:47.765+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 153478150002150 68 11732 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:09:47.765+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 116785907755771 55 11732 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.768+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 45479338375128 55 11732 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 249919763471169 64 11732 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:09:47.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 166675095997741 57 11732 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:09:47.774+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 49796766757346 58 11732 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.776+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 124194574848641 54 11732 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.779+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 19274425379955 55 11732 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.779+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 149022976193937 55 11732 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.782+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 184487117398647 55 11732 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:09:47.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 196777743741606 51 11732 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:09:47.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 199728167534433 66 11732 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:09:47.794+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 24856128127173 60 11732 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:09:47.795+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 158076890855634 54 11732 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:09:47.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 29071415952852 61 11732 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:47.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 61798873988101 67 11732 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:09:47.804+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 138935694816848 61 11732 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:09:47.804+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 191719959781498 63 11732 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:47.804+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 103336286678362 46 11732 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:47.805+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 140942236111856 63 11732 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:09:47.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 56626109228904 46 11732 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:09:47.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 119871837460216 39 11732 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:09:47.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 252537041863196 73 11732 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:09:47.813+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 177105814270799 58 11732 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:09:47.841+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c1 InstanceIds [13258659765240018889 7410154703733445404 4030261110898422634 14607307599259773249] 2023-10-21T04:09:47.841+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 11732 2023-10-21T04:09:47.841+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11732 2023-10-21T04:09:47.841+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:09:47.841+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:09:47.841+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 4030261110898422634 INDEX_STATE_CATCHUP 2023-10-21T04:09:47.841+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 14607307599259773249 INDEX_STATE_CATCHUP 2023-10-21T04:09:47.842+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 7410154703733445404 INDEX_STATE_CATCHUP 2023-10-21T04:09:47.842+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 7410154703733445404 true 2023-10-21T04:09:47.842+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 4030261110898422634 true 2023-10-21T04:09:47.842+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 14607307599259773249 true 2023-10-21T04:09:47.842+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 7410154703733445404 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 11732 BuildTS: [0=52,1=58,2=50,3=45,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=64,44=57,45=58,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:09:47.842+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:09:47.842+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:09:47.842+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11732 STREAM_ACTIVE 2023-10-21T04:09:47.842+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 11732 2023-10-21T04:09:47.842+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 11732 2023-10-21T04:09:47.842+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:47.843+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:47.843+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:47.843+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:47.844+05:30 [Info] send progress stats to clients 2023-10-21T04:09:47.844+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:09:47.844+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:09:47.844+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:8 bucket_1:10] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_system:_query:1]] 2023-10-21T04:09:47.844+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:10] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_system:_query:1]] 2023-10-21T04:09:47.844+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:10] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_system:_query:1]] 2023-10-21T04:09:47.844+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 11732 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:09:47.847+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:47.848+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:09:47.856+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.060037ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:47.872+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 2023-10-21T04:09:47.872+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11732 2023-10-21T04:09:47.872+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11732 2023-10-21T04:09:47.872+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 7410154703733445404 false 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 4030261110898422634 false 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 14607307599259773249 false 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 7410154703733445404 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 4030261110898422634 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 14607307599259773249 2023-10-21T04:09:47.872+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:09:47.872+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 11732 2023-10-21T04:09:47.892+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 36.605796ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:47.906+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.415087ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:47.925+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:47.925+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:47.931+05:30 [Info] switched currmeta from 417 -> 420 force true 2023-10-21T04:09:47.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:47.944+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:47.948+05:30 [Info] switched currmeta from 420 -> 420 force true 2023-10-21T04:09:48.053+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:09:48.053+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_system:_query 2023-10-21T04:09:48.053+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 13258659765240018889 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId: 36636, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 54 54 54} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 53 53 53} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 55 55 55} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 47 47 47} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 63 63 63} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 57 57 57} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 52 52 52} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:09:48.053+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 13258659765240018889 INDEX_STATE_ACTIVE 2023-10-21T04:09:48.053+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_system:_query State Changed to INACTIVE 2023-10-21T04:09:48.053+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:09:48.054+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:09:48.054+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 0 New 0 2023-10-21T04:09:48.054+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 36636 2023-10-21T04:09:48.054+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:48.054+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:48.055+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13258659765240018889 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:09:48.055+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 36636 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:48.055+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 13258659765240018889 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:09:48.055+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:09:48.055+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:09:48.055+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:48.055+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 36636 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:48.055+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc00d15a900 }] 2023-10-21T04:09:48.055+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:48.055+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:48.055+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:48.055+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:48.056+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:48.056+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:48.056+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:48.056+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:09:48.056+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:09:48.056+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:48.056+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId 36636 2023-10-21T04:09:48.056+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:09:48.062+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 0 186517626268899 55 2023-10-21T04:09:48.062+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.062+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 1 178840478062605 54 2023-10-21T04:09:48.062+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.062+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 2 148075163102199 54 2023-10-21T04:09:48.062+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 11 66881023717063 51 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 4 28605411869539 58 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 5 39552632604610 52 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 7 191903564840858 63 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 9 225280996447309 53 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 10 274410200699189 54 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 3 215374859564337 35 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.065+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 6 242327631330054 62 2023-10-21T04:09:48.065+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.066+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 8 50961953886801 45 2023-10-21T04:09:48.066+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.070+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.205519ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:48.072+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 12 4429827396609 42 2023-10-21T04:09:48.072+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.072+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 13 183148377680315 46 2023-10-21T04:09:48.072+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.072+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 14 226814108379280 72 2023-10-21T04:09:48.072+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.079+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 21 209771427122473 47 2023-10-21T04:09:48.079+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.079+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 17 55290066231400 51 2023-10-21T04:09:48.079+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.079+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 18 242578104468179 63 2023-10-21T04:09:48.079+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.079+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 19 210306088158507 48 2023-10-21T04:09:48.079+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.079+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 20 230239359684242 50 2023-10-21T04:09:48.080+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.080+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 22 260036488774725 51 2023-10-21T04:09:48.080+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.080+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 23 216462437971699 57 2023-10-21T04:09:48.080+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.080+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 15 127185710837163 50 2023-10-21T04:09:48.080+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.080+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 16 257031483891250 55 2023-10-21T04:09:48.080+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.083+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:09:48.083+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 36636 2023-10-21T04:09:48.083+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 36636 2023-10-21T04:09:48.083+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:09:48.083+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 26 219132834127088 52 2023-10-21T04:09:48.083+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.083+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 37 253494845579900 54 2023-10-21T04:09:48.083+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.083+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 45 12301831552869 63 2023-10-21T04:09:48.083+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.083+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 53 204867593700407 56 2023-10-21T04:09:48.083+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.083+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 29 45957426777107 57 2023-10-21T04:09:48.083+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.083+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 43 146267938201125 55 2023-10-21T04:09:48.083+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.083+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 47 271074273345339 53 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 51 200436537417141 54 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 24 160961818008256 59 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 28 102578907139247 58 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 34 28417669987133 58 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 36 253845164956592 58 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 39 222555662236724 67 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 42 252931432434966 46 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 50 207367350302991 64 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 40 5915864305121 60 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 52 188718367400378 56 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 25 280874278672777 71 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 30 99433130332778 58 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 38 188107826286720 60 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 44 158545118983293 67 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 35 70213628416850 47 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 48 227378777358533 57 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 49 215467668600630 34 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 54 279362078165491 55 2023-10-21T04:09:48.084+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.084+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 31 217381278987664 67 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 32 82469662608686 60 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 41 259381390638966 58 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 27 184056281482149 49 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 33 136958029391999 65 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 46 242994859770444 54 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 63 133748675726909 56 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 55 170056388266420 70 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 56 264835102027079 52 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 58 16044440367455 53 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 59 125009072873642 53 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 60 75566139929862 56 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 61 106932309485550 53 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 57 129411666769864 55 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.085+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 62 10527297481171 57 2023-10-21T04:09:48.085+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:09:48.088+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:48.088+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:48.101+05:30 [Info] switched currmeta from 420 -> 421 force true 2023-10-21T04:09:48.379+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:48.379+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:48.383+05:30 [Info] switched currmeta from 421 -> 422 force true 2023-10-21T04:09:48.530+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13258659765240018889 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009e6e0c0 2023-10-21T04:09:48.530+05:30 [Info] handleUpdateIndexRState: Index instance 13258659765240018889 rstate moved to ACTIVE 2023-10-21T04:09:48.541+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.883229ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:48.541+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[13258659765240018889:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:48.541+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4, elapsed(sec): 31.711443198 2023-10-21T04:09:48.554+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:48.554+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:48.554+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken26:d5:d0:e3:1f:91:4a:e4 2023-10-21T04:09:48.556+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:48.560+05:30 [Info] switched currmeta from 422 -> 423 force true 2023-10-21T04:09:48.596+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.597+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.598+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.599+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.600+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.601+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.602+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.603+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.604+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.604+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13258659765240018889:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:09:48.846+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 4030261110898422634 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 11732, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 58 58 58} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 60 60 60} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 51 49 51} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 59 59 59} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 64 64 64} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 58 58 58} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 7410154703733445404 INDEX_STATE_ACTIVE 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 4030261110898422634 INDEX_STATE_ACTIVE 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 14607307599259773249 INDEX_STATE_ACTIVE 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:09:48.846+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:48.846+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:48.846+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 11732 2023-10-21T04:09:48.847+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:48.847+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:09:48.847+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14607307599259773249 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 7410154703733445404 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:09:48.848+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11732 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:48.848+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 7410154703733445404 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:09:48.848+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4030261110898422634 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:09:48.848+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 14607307599259773249 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:09:48.848+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:09:48.848+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:09:48.848+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:09:48.848+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11732 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:09:48.848+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:09:48.848+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:48.848+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:09:48.849+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:09:48.849+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:59760: use of closed network connection 2023-10-21T04:09:48.849+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:59760" exit: read tcp4 127.0.0.1:9121->127.0.0.1:59760: use of closed network connection 2023-10-21T04:09:48.849+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:59760" closed ! 2023-10-21T04:09:48.850+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:09:48.850+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:59760" already gone 2023-10-21T04:09:48.850+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:48.850+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:48.850+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:09:48.850+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:09:48.851+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:09:48.851+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:09:48.852+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 11732 2023-10-21T04:09:48.853+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:09:48.863+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.279937ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:48.899+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 35.812326ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:48.919+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.939959ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:48.923+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:09:48.923+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:09:48.923+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:09:48.923+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11732 2023-10-21T04:09:48.923+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11732 2023-10-21T04:09:48.923+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:09:48.953+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:48.953+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:48.958+05:30 [Info] switched currmeta from 423 -> 426 force true 2023-10-21T04:09:48.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:48.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:48.974+05:30 [Info] switched currmeta from 426 -> 426 force true 2023-10-21T04:09:49.027+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:49.027+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:49.031+05:30 [Info] switched currmeta from 426 -> 427 force true 2023-10-21T04:09:49.045+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007bff260 2023-10-21T04:09:49.045+05:30 [Info] handleUpdateIndexRState: Index instance 4030261110898422634 rstate moved to ACTIVE 2023-10-21T04:09:49.053+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:09:49.061+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.036747ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:49.061+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14607307599259773249 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0051f3920 2023-10-21T04:09:49.061+05:30 [Info] handleUpdateIndexRState: Index instance 14607307599259773249 rstate moved to ACTIVE 2023-10-21T04:09:49.062+05:30 [Info] Indexer::handleMergePartition Source 10774491569505258841 Target 7410154703733445404 2023-10-21T04:09:49.062+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7410154703733445404 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0054862a0 2023-10-21T04:09:49.062+05:30 [Info] MergePartition: sent async request to update index instance 7410154703733445404 rstate moved to ACTIVE 2023-10-21T04:09:49.074+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.9814ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:49.075+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[4030261110898422634:true 7410154703733445404:true 14607307599259773249:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:09:49.079+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:49.079+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:49.087+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.247639ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:49.087+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 10774491569505258841, realInst: 7410154703733445404 2023-10-21T04:09:49.087+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken80:53:e0:a8:fa:c4:c4:85, elapsed(sec): 32.738654371 2023-10-21T04:09:49.096+05:30 [Info] switched currmeta from 427 -> 430 force true 2023-10-21T04:09:49.109+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:49.110+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:49.113+05:30 [Info] switched currmeta from 430 -> 430 force true 2023-10-21T04:09:49.138+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:49.203+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken80:53:e0:a8:fa:c4:c4:85 2023-10-21T04:09:49.408+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:49.409+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:49.414+05:30 [Info] switched currmeta from 430 -> 431 force true 2023-10-21T04:09:49.448+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 10] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:49.448+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. taskList: &{Rev:[0 0 0 0 0 0 0 10] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 7] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:49.450+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 10] 2023-10-21T04:09:49.451+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 10] 2023-10-21T04:09:49.456+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:49.464+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:49.464+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:49.466+05:30 [Info] switched currmeta from 431 -> 431 force true 2023-10-21T04:09:49.590+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.590+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.590+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.590+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4030261110898422634:[0] 14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4030261110898422634:[0] 14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4030261110898422634:[0] 14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4030261110898422634:[0] 7410154703733445404:[3 4 5 6 7 8 1 2] 14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4030261110898422634:[0] 7410154703733445404:[3 4 5 6 7 8 1 2] 14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4030261110898422634:[0] 7410154703733445404:[3 4 5 6 7 8 1 2] 14607307599259773249:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:09:49.660+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:49.660+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:49.669+05:30 [Info] switched currmeta from 431 -> 432 force true 2023-10-21T04:09:49.718+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:49.758+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbb:c4:4c:76:47:bc:ee:e4 MasterId: SourceId: DestId: RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:49.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11400459767051699632 PartitionId 0 Started Billing WUs 2023-10-21T04:09:49.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4030261110898422634 PartitionId 0 Started Billing WUs 2023-10-21T04:09:49.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 325705934323625324 PartitionId 0 Started Billing WUs 2023-10-21T04:09:49.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2483169159760864439 PartitionId 0 Started Billing WUs 2023-10-21T04:09:49.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14607307599259773249 PartitionId 0 Started Billing WUs 2023-10-21T04:09:49.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15131823346681830059 PartitionId 0 Started Billing WUs 2023-10-21T04:09:49.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17128008689111072444 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.052+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:50.053+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:50.059+05:30 [Info] switched currmeta from 432 -> 433 force true 2023-10-21T04:09:50.073+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:50.073+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:50.080+05:30 [Info] switched currmeta from 433 -> 434 force true 2023-10-21T04:09:50.127+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:50.134+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken41:50:80:87:56:22:5c:7c MasterId: SourceId: DestId: RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken53:c:96:ac:59:a4:5d:7b SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:50.134+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4004220452657849790 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.134+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3292093078969249751 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.134+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2288923528993975684 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.135+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11739645294255241349 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.135+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3985672602331234807 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.135+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11160011617777336020 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.135+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17148287981272169168 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.135+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7597064108789640455 PartitionId 0 Started Billing WUs 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.219+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:09:50.583+05:30 [Info] ShardRebalancer::computeProgress 1 2023-10-21T04:09:50.847+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:50.847+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:50.856+05:30 [Info] switched currmeta from 434 -> 436 force true 2023-10-21T04:09:50.874+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:50.874+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:50.885+05:30 [Info] switched currmeta from 436 -> 436 force true 2023-10-21T04:09:50.892+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:50.893+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:50.906+05:30 [Info] switched currmeta from 436 -> 437 force true 2023-10-21T04:09:50.987+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:50.987+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:50.993+05:30 [Info] switched currmeta from 437 -> 440 force true 2023-10-21T04:09:51.001+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.015+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.033+05:30 [Info] switched currmeta from 440 -> 441 force true 2023-10-21T04:09:51.061+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.062+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.073+05:30 [Info] switched currmeta from 441 -> 441 force true 2023-10-21T04:09:51.129+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.153+05:30 [Info] switched currmeta from 441 -> 442 force true 2023-10-21T04:09:51.214+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.214+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.251+05:30 [Info] switched currmeta from 442 -> 444 force true 2023-10-21T04:09:51.281+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.281+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.316+05:30 [Info] switched currmeta from 444 -> 444 force true 2023-10-21T04:09:51.354+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.354+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.357+05:30 [Info] switched currmeta from 444 -> 445 force true 2023-10-21T04:09:51.381+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.381+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.385+05:30 [Info] switched currmeta from 445 -> 445 force true 2023-10-21T04:09:51.458+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.460+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.510+05:30 [Info] switched currmeta from 445 -> 447 force true 2023-10-21T04:09:51.560+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.560+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.616+05:30 [Info] switched currmeta from 447 -> 450 force true 2023-10-21T04:09:51.646+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.646+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.650+05:30 [Info] switched currmeta from 450 -> 451 force true 2023-10-21T04:09:51.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.671+05:30 [Info] switched currmeta from 451 -> 451 force true 2023-10-21T04:09:51.702+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.702+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.709+05:30 [Info] switched currmeta from 451 -> 452 force true 2023-10-21T04:09:51.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.718+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.719+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:09:51.781+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.781+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.784+05:30 [Info] switched currmeta from 452 -> 452 force true 2023-10-21T04:09:51.795+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.798+05:30 [Info] switched currmeta from 452 -> 452 force true 2023-10-21T04:09:51.811+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.811+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.815+05:30 [Info] switched currmeta from 452 -> 453 force true 2023-10-21T04:09:51.853+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.853+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.884+05:30 [Info] switched currmeta from 453 -> 457 force true 2023-10-21T04:09:51.966+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.966+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.972+05:30 [Info] switched currmeta from 457 -> 459 force true 2023-10-21T04:09:51.978+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:51.979+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:51.999+05:30 [Info] switched currmeta from 459 -> 459 force true 2023-10-21T04:09:52.007+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.008+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.034+05:30 [Info] switched currmeta from 459 -> 460 force true 2023-10-21T04:09:52.081+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.082+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.087+05:30 [Info] switched currmeta from 460 -> 462 force true 2023-10-21T04:09:52.103+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.105+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.109+05:30 [Info] switched currmeta from 462 -> 462 force true 2023-10-21T04:09:52.169+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.171+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.177+05:30 [Info] switched currmeta from 462 -> 462 force true 2023-10-21T04:09:52.204+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.205+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.213+05:30 [Info] switched currmeta from 462 -> 463 force true 2023-10-21T04:09:52.308+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.308+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.316+05:30 [Info] switched currmeta from 463 -> 465 force true 2023-10-21T04:09:52.330+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.331+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.337+05:30 [Info] switched currmeta from 465 -> 465 force true 2023-10-21T04:09:52.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.456+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.460+05:30 [Info] switched currmeta from 465 -> 468 force true 2023-10-21T04:09:52.466+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.466+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.470+05:30 [Info] switched currmeta from 468 -> 468 force true 2023-10-21T04:09:52.489+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.490+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.494+05:30 [Info] switched currmeta from 468 -> 468 force true 2023-10-21T04:09:52.511+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.511+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.532+05:30 [Info] switched currmeta from 468 -> 470 force true 2023-10-21T04:09:52.541+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.541+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.545+05:30 [Info] switched currmeta from 470 -> 470 force true 2023-10-21T04:09:52.636+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.638+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.672+05:30 [Info] switched currmeta from 470 -> 474 force true 2023-10-21T04:09:52.712+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.712+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.725+05:30 [Info] switched currmeta from 474 -> 476 force true 2023-10-21T04:09:52.775+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.775+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.778+05:30 [Info] switched currmeta from 476 -> 476 force true 2023-10-21T04:09:52.805+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.805+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.810+05:30 [Info] switched currmeta from 476 -> 477 force true 2023-10-21T04:09:52.846+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:52.846+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:52.922+05:30 [Info] switched currmeta from 477 -> 480 force true 2023-10-21T04:09:53.010+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.010+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.020+05:30 [Info] switched currmeta from 480 -> 482 force true 2023-10-21T04:09:53.044+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.044+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.053+05:30 [Info] switched currmeta from 482 -> 482 force true 2023-10-21T04:09:53.106+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.106+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.121+05:30 [Info] switched currmeta from 482 -> 484 force true 2023-10-21T04:09:53.147+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.147+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.157+05:30 [Info] switched currmeta from 484 -> 484 force true 2023-10-21T04:09:53.185+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.185+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.204+05:30 [Info] switched currmeta from 484 -> 486 force true 2023-10-21T04:09:53.221+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.223+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.231+05:30 [Info] switched currmeta from 486 -> 486 force true 2023-10-21T04:09:53.401+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.401+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.438+05:30 [Info] switched currmeta from 486 -> 490 force true 2023-10-21T04:09:53.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 1, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:09:53.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.455+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.458+05:30 [Info] switched currmeta from 490 -> 490 force true 2023-10-21T04:09:53.473+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.474+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.480+05:30 [Info] switched currmeta from 490 -> 491 force true 2023-10-21T04:09:53.490+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.490+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.512+05:30 [Info] switched currmeta from 491 -> 492 force true 2023-10-21T04:09:53.569+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.569+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.599+05:30 [Info] switched currmeta from 492 -> 494 force true 2023-10-21T04:09:53.611+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.611+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.616+05:30 [Info] switched currmeta from 494 -> 494 force true 2023-10-21T04:09:53.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.636+05:30 [Info] switched currmeta from 494 -> 495 force true 2023-10-21T04:09:53.678+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.678+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.682+05:30 [Info] switched currmeta from 495 -> 496 force true 2023-10-21T04:09:53.696+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.696+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.702+05:30 [Info] switched currmeta from 496 -> 496 force true 2023-10-21T04:09:53.760+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.761+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.765+05:30 [Info] switched currmeta from 496 -> 496 force true 2023-10-21T04:09:53.777+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.777+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.781+05:30 [Info] switched currmeta from 496 -> 497 force true 2023-10-21T04:09:53.790+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.790+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.797+05:30 [Info] switched currmeta from 497 -> 497 force true 2023-10-21T04:09:53.852+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.852+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.857+05:30 [Info] switched currmeta from 497 -> 498 force true 2023-10-21T04:09:53.913+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.915+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.931+05:30 [Info] switched currmeta from 498 -> 500 force true 2023-10-21T04:09:53.981+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:53.981+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:53.985+05:30 [Info] switched currmeta from 500 -> 500 force true 2023-10-21T04:09:54.052+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.052+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.079+05:30 [Info] switched currmeta from 500 -> 501 force true 2023-10-21T04:09:54.105+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.105+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.118+05:30 [Info] switched currmeta from 501 -> 503 force true 2023-10-21T04:09:54.136+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.137+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.156+05:30 [Info] switched currmeta from 503 -> 504 force true 2023-10-21T04:09:54.202+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.203+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.258+05:30 [Info] switched currmeta from 504 -> 506 force true 2023-10-21T04:09:54.291+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.291+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.310+05:30 [Info] switched currmeta from 506 -> 506 force true 2023-10-21T04:09:54.352+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.352+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.366+05:30 [Info] switched currmeta from 506 -> 506 force true 2023-10-21T04:09:54.381+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.382+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.389+05:30 [Info] switched currmeta from 506 -> 506 force true 2023-10-21T04:09:54.416+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.416+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.420+05:30 [Info] switched currmeta from 506 -> 507 force true 2023-10-21T04:09:54.448+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 10] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 8] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.99 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:54.448+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:54.452+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9998842303458171, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:09:54.453+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 11] 2023-10-21T04:09:54.453+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 11] 2023-10-21T04:09:54.491+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.491+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.495+05:30 [Info] switched currmeta from 507 -> 510 force true 2023-10-21T04:09:54.506+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.506+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.513+05:30 [Info] switched currmeta from 510 -> 510 force true 2023-10-21T04:09:54.599+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.601+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.608+05:30 [Info] switched currmeta from 510 -> 510 force true 2023-10-21T04:09:54.696+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.698+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.702+05:30 [Info] switched currmeta from 510 -> 513 force true 2023-10-21T04:09:54.761+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.761+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.769+05:30 [Info] switched currmeta from 513 -> 514 force true 2023-10-21T04:09:54.813+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.815+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.852+05:30 [Info] switched currmeta from 514 -> 515 force true 2023-10-21T04:09:54.950+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:54.950+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:54.979+05:30 [Info] switched currmeta from 515 -> 516 force true 2023-10-21T04:09:55.034+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.034+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.053+05:30 [Info] switched currmeta from 516 -> 516 force true 2023-10-21T04:09:55.107+05:30 [Info] ShardRebalancer::computeProgress 1 2023-10-21T04:09:55.134+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:09:55.145+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.146+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.170+05:30 [Info] switched currmeta from 516 -> 516 force true 2023-10-21T04:09:55.248+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.248+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.253+05:30 [Info] switched currmeta from 516 -> 517 force true 2023-10-21T04:09:55.277+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.277+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.291+05:30 [Info] switched currmeta from 517 -> 519 force true 2023-10-21T04:09:55.303+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.305+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.313+05:30 [Info] switched currmeta from 519 -> 519 force true 2023-10-21T04:09:55.321+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.326+05:30 [Info] switched currmeta from 519 -> 520 force true 2023-10-21T04:09:55.332+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.332+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.336+05:30 [Info] switched currmeta from 520 -> 520 force true 2023-10-21T04:09:55.451+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:09:55.547+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.547+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.556+05:30 [Info] switched currmeta from 520 -> 521 force true 2023-10-21T04:09:55.677+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.677+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.681+05:30 [Info] switched currmeta from 521 -> 523 force true 2023-10-21T04:09:55.690+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.690+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.695+05:30 [Info] switched currmeta from 523 -> 524 force true 2023-10-21T04:09:55.714+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.715+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.724+05:30 [Info] switched currmeta from 524 -> 524 force true 2023-10-21T04:09:55.737+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.737+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.749+05:30 [Info] switched currmeta from 524 -> 525 force true 2023-10-21T04:09:55.801+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.801+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.835+05:30 [Info] switched currmeta from 525 -> 528 force true 2023-10-21T04:09:55.845+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.845+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.850+05:30 [Info] switched currmeta from 528 -> 529 force true 2023-10-21T04:09:55.905+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.905+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.908+05:30 [Info] switched currmeta from 529 -> 530 force true 2023-10-21T04:09:55.916+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.916+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.919+05:30 [Info] switched currmeta from 530 -> 530 force true 2023-10-21T04:09:55.956+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:55.956+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:55.959+05:30 [Info] switched currmeta from 530 -> 530 force true 2023-10-21T04:09:56.023+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.023+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.027+05:30 [Info] switched currmeta from 530 -> 531 force true 2023-10-21T04:09:56.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.125+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.131+05:30 [Info] switched currmeta from 531 -> 531 force true 2023-10-21T04:09:56.149+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.149+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.152+05:30 [Info] switched currmeta from 531 -> 532 force true 2023-10-21T04:09:56.239+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.239+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.243+05:30 [Info] switched currmeta from 532 -> 533 force true 2023-10-21T04:09:56.289+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.289+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.293+05:30 [Info] switched currmeta from 533 -> 533 force true 2023-10-21T04:09:56.302+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.302+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.307+05:30 [Info] switched currmeta from 533 -> 534 force true 2023-10-21T04:09:56.433+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.433+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.445+05:30 [Info] switched currmeta from 534 -> 535 force true 2023-10-21T04:09:56.451+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9969623169763157 2023-10-21T04:09:56.459+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.459+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.482+05:30 [Info] switched currmeta from 535 -> 536 force true 2023-10-21T04:09:56.493+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.493+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.505+05:30 [Info] switched currmeta from 536 -> 536 force true 2023-10-21T04:09:56.533+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.535+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.539+05:30 [Info] switched currmeta from 536 -> 538 force true 2023-10-21T04:09:56.546+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.546+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.550+05:30 [Info] switched currmeta from 538 -> 539 force true 2023-10-21T04:09:56.571+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.572+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.578+05:30 [Info] switched currmeta from 539 -> 539 force true 2023-10-21T04:09:56.626+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.626+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.637+05:30 [Info] switched currmeta from 539 -> 540 force true 2023-10-21T04:09:56.662+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.662+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.666+05:30 [Info] switched currmeta from 540 -> 540 force true 2023-10-21T04:09:56.683+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.683+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.690+05:30 [Info] switched currmeta from 540 -> 541 force true 2023-10-21T04:09:56.714+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.714+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.723+05:30 [Info] switched currmeta from 541 -> 542 force true 2023-10-21T04:09:56.745+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.745+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.749+05:30 [Info] switched currmeta from 542 -> 542 force true 2023-10-21T04:09:56.771+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.771+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.774+05:30 [Info] switched currmeta from 542 -> 543 force true 2023-10-21T04:09:56.798+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.798+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.804+05:30 [Info] switched currmeta from 543 -> 544 force true 2023-10-21T04:09:56.849+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.849+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.853+05:30 [Info] switched currmeta from 544 -> 545 force true 2023-10-21T04:09:56.860+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.861+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.864+05:30 [Info] switched currmeta from 545 -> 546 force true 2023-10-21T04:09:56.869+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.869+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.873+05:30 [Info] switched currmeta from 546 -> 546 force true 2023-10-21T04:09:56.883+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.883+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.891+05:30 [Info] switched currmeta from 546 -> 547 force true 2023-10-21T04:09:56.899+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:56.899+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:56.902+05:30 [Info] switched currmeta from 547 -> 547 force true 2023-10-21T04:09:57.144+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.144+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.150+05:30 [Info] switched currmeta from 547 -> 552 force true 2023-10-21T04:09:57.158+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.158+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.164+05:30 [Info] switched currmeta from 552 -> 553 force true 2023-10-21T04:09:57.176+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.178+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.183+05:30 [Info] switched currmeta from 553 -> 553 force true 2023-10-21T04:09:57.209+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.236+05:30 [Info] switched currmeta from 553 -> 555 force true 2023-10-21T04:09:57.244+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.244+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.248+05:30 [Info] switched currmeta from 555 -> 555 force true 2023-10-21T04:09:57.268+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.268+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.274+05:30 [Info] switched currmeta from 555 -> 557 force true 2023-10-21T04:09:57.290+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.291+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.299+05:30 [Info] switched currmeta from 557 -> 559 force true 2023-10-21T04:09:57.320+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.341+05:30 [Info] switched currmeta from 559 -> 560 force true 2023-10-21T04:09:57.399+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.399+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.405+05:30 [Info] switched currmeta from 560 -> 564 force true 2023-10-21T04:09:57.415+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.415+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.426+05:30 [Info] switched currmeta from 564 -> 566 force true 2023-10-21T04:09:57.450+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9960930190437022 2023-10-21T04:09:57.464+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.467+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.498+05:30 [Info] switched currmeta from 566 -> 570 force true 2023-10-21T04:09:57.567+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.568+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.600+05:30 [Info] switched currmeta from 570 -> 571 force true 2023-10-21T04:09:57.612+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.612+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.625+05:30 [Info] switched currmeta from 571 -> 571 force true 2023-10-21T04:09:57.661+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.662+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.676+05:30 [Info] switched currmeta from 571 -> 572 force true 2023-10-21T04:09:57.739+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.739+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.744+05:30 [Info] switched currmeta from 572 -> 574 force true 2023-10-21T04:09:57.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.754+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.816+05:30 [Info] switched currmeta from 574 -> 577 force true 2023-10-21T04:09:57.832+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.832+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.836+05:30 [Info] switched currmeta from 577 -> 577 force true 2023-10-21T04:09:57.857+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.859+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.875+05:30 [Info] switched currmeta from 577 -> 578 force true 2023-10-21T04:09:57.907+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.907+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.920+05:30 [Info] switched currmeta from 578 -> 579 force true 2023-10-21T04:09:57.943+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:57.943+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:57.995+05:30 [Info] switched currmeta from 579 -> 582 force true 2023-10-21T04:09:58.033+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:58.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:58.053+05:30 [Info] switched currmeta from 582 -> 585 force true 2023-10-21T04:09:58.167+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:58.167+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:58.173+05:30 [Info] switched currmeta from 585 -> 586 force true 2023-10-21T04:09:58.191+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:09:58.191+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:09:58.214+05:30 [Info] switched currmeta from 586 -> 586 force true 2023-10-21T04:09:58.450+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9959476290516206 2023-10-21T04:09:58.815+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:58.815+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [6611724767191156837 15964300471522801617] 2023-10-21T04:09:58.816+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data(shard6611724767191156837) : all deamons started 2023-10-21T04:09:58.816+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery(shard6611724767191156837) : all deamons started 2023-10-21T04:09:58.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:09:58.816+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data(shard6611724767191156837) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.816+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data(shard6611724767191156837) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.816+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery(shard6611724767191156837) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.816+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery(shard6611724767191156837) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:09:58.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : RestoreShardDone completed 2023-10-21T04:09:58.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data(shard15964300471522801617) : all deamons started 2023-10-21T04:09:58.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery(shard15964300471522801617) : all deamons started 2023-10-21T04:09:58.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:09:58.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data(shard15964300471522801617) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data(shard15964300471522801617) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery(shard15964300471522801617) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery(shard15964300471522801617) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:09:58.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : RestoreShardDone completed 2023-10-21T04:09:58.819+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [6611724767191156837 15964300471522801617], elapsed: 3.739326ms 2023-10-21T04:09:58.819+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[6611724767191156837:true 15964300471522801617:true] 2023-10-21T04:09:58.819+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 6 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 7 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 8 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 1 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 2 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 3 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 4 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 80911002600697866, partnId: 5 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8446412942681787061, partnId: 0 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2483169159760864439, partnId: 0 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15131823346681830059, partnId: 0 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14910157020177557388, partnId: 0 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14780363202971367008, partnId: 0 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4030261110898422634, partnId: 0 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 6 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 7 2023-10-21T04:09:58.820+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 8 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 1 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 2 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 3 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 4 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7410154703733445404, partnId: 5 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 3 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 4 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 5 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 6 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 7 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 8 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 1 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2921784364458449578, partnId: 2 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14607307599259773249, partnId: 0 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 1 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 2 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 3 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 4 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 5 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 6 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 7 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 249283441109236971, partnId: 8 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 7 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 8 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 1 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 2 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 3 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 4 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 5 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9664682240473767939, partnId: 6 2023-10-21T04:09:58.821+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 5 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 6 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 7 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 8 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 1 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 2 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 3 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12043401624902403496, partnId: 4 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17037906325662037075, partnId: 0 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8880461853743568138, partnId: 0 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11400459767051699632, partnId: 0 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17128008689111072444, partnId: 0 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 325705934323625324, partnId: 0 2023-10-21T04:09:58.822+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17729593977321451597, partnId: 0 2023-10-21T04:09:58.822+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.663µs 2023-10-21T04:09:58.822+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [6611724767191156837 15964300471522801617] 2023-10-21T04:09:58.822+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [6611724767191156837 15964300471522801617], errMap: map[6611724767191156837: 15964300471522801617:], elapsed: 11.183µs 2023-10-21T04:09:58.823+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:09:58.823+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:09:58.823+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_DONE] 2023-10-21T04:09:58.823+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 53.59µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:58.883+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:58.883+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [10563635432419104314 3268862397300335529] 2023-10-21T04:09:58.883+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data(shard10563635432419104314) : all deamons started 2023-10-21T04:09:58.883+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery(shard10563635432419104314) : all deamons started 2023-10-21T04:09:58.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:09:58.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:09:58.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : RestoreShardDone completed 2023-10-21T04:09:58.884+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data(shard3268862397300335529) : all deamons started 2023-10-21T04:09:58.884+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery(shard3268862397300335529) : all deamons started 2023-10-21T04:09:58.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:09:58.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:09:58.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : RestoreShardDone completed 2023-10-21T04:09:58.884+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [10563635432419104314 3268862397300335529], elapsed: 775.082µs 2023-10-21T04:09:58.884+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[3268862397300335529:true 10563635432419104314:true] 2023-10-21T04:09:58.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 3810060733504350542, partnId: 0 2023-10-21T04:09:58.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17148287981272169168, partnId: 0 2023-10-21T04:09:58.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4004220452657849790, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2288923528993975684, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11160011617777336020, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 18201836893981255880, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7597064108789640455, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 3985672602331234807, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4435591218553988516, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 8 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 1 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 2 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 3 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 4 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 5 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 6 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16726320877690809560, partnId: 7 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11756349460704871322, partnId: 0 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 8 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 1 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 2 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 3 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 4 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 5 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 6 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1532172649864318584, partnId: 7 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 8 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 1 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 2 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 3 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 4 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 5 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 6 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2170849387178940761, partnId: 7 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 6 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 7 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 8 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 1 2023-10-21T04:09:58.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 2 2023-10-21T04:09:58.886+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 3 2023-10-21T04:09:58.886+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 4 2023-10-21T04:09:58.886+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13953156762317267443, partnId: 5 2023-10-21T04:09:58.886+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13258659765240018889, partnId: 0 2023-10-21T04:09:58.886+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 6 2023-10-21T04:09:58.886+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 7 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 8 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 1 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 2 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 3 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 4 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery(shard10563635432419104314) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data(shard10563635432419104314) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data(shard10563635432419104314) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery(shard10563635432419104314) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data(shard3268862397300335529) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data(shard3268862397300335529) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery(shard3268862397300335529) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:09:58.886+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery(shard3268862397300335529) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7755999278497385093, partnId: 5 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 3292093078969249751, partnId: 0 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11739645294255241349, partnId: 0 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 5 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 6 2023-10-21T04:09:58.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 7 2023-10-21T04:09:58.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 8 2023-10-21T04:09:58.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 1 2023-10-21T04:09:58.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 2 2023-10-21T04:09:58.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 3 2023-10-21T04:09:58.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2307463109245789631, partnId: 4 2023-10-21T04:09:58.888+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.669µs 2023-10-21T04:09:58.888+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [10563635432419104314 3268862397300335529] 2023-10-21T04:09:58.888+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [10563635432419104314 3268862397300335529], errMap: map[3268862397300335529: 10563635432419104314:], elapsed: 7.957µs 2023-10-21T04:09:58.888+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:09:58.888+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:09:58.888+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_DONE] 2023-10-21T04:09:58.888+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 28.468µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:58.970+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:58.972+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:58.973+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:53:e0:a8:fa:c4:c4:85 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c Shards: [6611724767191156837 15964300471522801617] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7543997995673100641 RealInstId: 12043401624902403496 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14910157020177557388 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11400459767051699632 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6377858590284099462 RealInstId: 80911002600697866 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4030261110898422634 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 325705934323625324 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2483169159760864439 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 8446412942681787061 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 10774491569505258841 RealInstId: 7410154703733445404 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1981123602637975601 RealInstId: 249283441109236971 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4611974930531028413 RealInstId: 2921784364458449578 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10428614901675085295 RealInstId: 9664682240473767939 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14607307599259773249 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17037906325662037075 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 14780363202971367008 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 15131823346681830059 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 8880461853743568138 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17729593977321451597 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17128008689111072444 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:58.994+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:-1 5020693e9ae5c52168d414b02f922ad8:0 85232c9ba3e4451b95e9e6fc8337e027:0], batchSize: 2 2023-10-21T04:09:58.994+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardTokenc8:87:93:a:4c:bb:58:5c is in state: ShardTokenCommit 2023-10-21T04:09:58.995+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken26:d5:d0:e3:1f:91:4a:e4 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 5020693e9ae5c52168d414b02f922ad8 (127.0.0.1:9002) DestId: 31fd475f22d8ee0da906c31968c91305 (127.0.0.1:9003) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken53:c:96:ac:59:a4:5d:7b Shards: [10563635432419104314 3268862397300335529] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18201836893981255880 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4004220452657849790 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 3292093078969249751 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12845014165282783025 RealInstId: 2307463109245789631 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11756349460704871322 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 579097042483677314 RealInstId: 2170849387178940761 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4248339762727765604 RealInstId: 7755999278497385093 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8934648905917366073 RealInstId: 13953156762317267443 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2288923528993975684 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10225476268021138633 RealInstId: 16726320877690809560 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11739645294255241349 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 4164706166433630808 RealInstId: 1532172649864318584 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3810060733504350542 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3985672602331234807 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4435591218553988516 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 11160011617777336020 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17148287981272169168 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13258659765240018889 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 7597064108789640455 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:59.032+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:-2 5020693e9ae5c52168d414b02f922ad8:0 85232c9ba3e4451b95e9e6fc8337e027:0], batchSize: 2 2023-10-21T04:09:59.032+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardTokenc8:87:93:a:4c:bb:58:5c is in state: ShardTokenCommit 2023-10-21T04:09:59.032+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken80:53:e0:a8:fa:c4:c4:85 2023-10-21T04:09:59.046+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken26:d5:d0:e3:1f:91:4a:e4 2023-10-21T04:09:59.089+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken53:c:96:ac:59:a4:5d:7b MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 757549517258233573 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 2866240913596939981 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3841645022501780362 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10040529926077950124 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12427853620151898915 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2330053926472458685 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10655181962244539875 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:59.116+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:-2 5020693e9ae5c52168d414b02f922ad8:0 85232c9ba3e4451b95e9e6fc8337e027:-1], batchSize: 2 2023-10-21T04:09:59.116+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardTokenc8:87:93:a:4c:bb:58:5c is in state: ShardTokenCommit 2023-10-21T04:09:59.124+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc8:87:93:a:4c:bb:58:5c MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 281b41507b1f937a1b6c37bc4027eef6 (127.0.0.1:9001) DestId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18226445516969809850 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3231309696512952224 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 15710672812788661570 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3140239290166316345 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3878185423818313063 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17238494267014926113 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1422056348477710921 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [1] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:59.150+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[281b41507b1f937a1b6c37bc4027eef6:0 31fd475f22d8ee0da906c31968c91305:-2 5020693e9ae5c52168d414b02f922ad8:0 85232c9ba3e4451b95e9e6fc8337e027:-2], batchSize: 2 2023-10-21T04:09:59.150+05:30 [Info] ShardRebalancer::processShardTransferTokenAsMaster No Tokens Found. Mark Done. 2023-10-21T04:09:59.150+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken53:c:96:ac:59:a4:5d:7b 2023-10-21T04:09:59.150+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 11] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/15e5cffff68b4b70e641b53958f9ea44 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]} {Rev:[0 0 0 0 0 0 0 9] ID:rebalance/15e5cffff68b4b70e641b53958f9ea44 Type:task-rebalance Status:task-running IsCancelable:true Progress:1 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:15e5cffff68b4b70e641b53958f9ea44]}]} 2023-10-21T04:09:59.151+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:59.151+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:09:59.166+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 12] 2023-10-21T04:09:59.168+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 12] 2023-10-21T04:09:59.181+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:09:59.181+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:09:59.181+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress done received. progress collector stopped 2023-10-21T04:09:59.181+05:30 [Info] ShardRebalancer::updateProgress Done Received 2023-10-21T04:09:59.181+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:09:59.181+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:09:59.186+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {31fd475f22d8ee0da906c31968c91305 15e5cffff68b4b70e641b53958f9ea44 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:09:59.207+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:09:59.214+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken41:50:80:87:56:22:5c:7c MasterId: SourceId: DestId: RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken53:c:96:ac:59:a4:5d:7b SiblingTokenId: ShardToken26:d5:d0:e3:1f:91:4a:e4 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:59.215+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenbb:c4:4c:76:47:bc:ee:e4 MasterId: SourceId: DestId: RebalId: 15e5cffff68b4b70e641b53958f9ea44 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokenc8:87:93:a:4c:bb:58:5c SiblingTokenId: ShardToken80:53:e0:a8:fa:c4:c4:85 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:09:59.257+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:09:59.257+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:09:59.257+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.435µs 2023-10-21T04:09:59.257+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:09:59.257+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:09:59.257+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:09:59.262+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:09:59.262+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:09:59.262+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:09:59.263+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:09:59.263+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 31.035µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:59.269+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:09:59.269+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:09:59.269+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster true, forceUnbalanced false, err: 2023-10-21T04:09:59.269+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 12] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:09:59.269+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:59.269+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:09:59.275+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:09:59.281+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:09:59.283+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:09:59.283+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:09:59.285+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:09:59.285+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:09:59.287+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:09:59.289+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:09:59.368+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:09:59.368+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:09:59.398+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:59820 2023-10-21T04:09:59.399+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:09:59.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9318570685210743, throttleDelayMs (new, old, change): (8, 10, -2) 2023-10-21T04:09:59.525+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 115.197453ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:09:59.526+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:09:59.526+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:09:59.526+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:09:59.636+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:09:59.978+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:10:00.027+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:10:00.093+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:10:00.093+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:10:00.094+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:10:00.100+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:00.101+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:00.102+05:30 [Info] UnwatchMetadata(): indexer 281b41507b1f937a1b6c37bc4027eef6 2023-10-21T04:10:00.103+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:10:00.104+05:30 [Info] UnwatchMetadata(): finish for indexer 281b41507b1f937a1b6c37bc4027eef6 2023-10-21T04:10:00.104+05:30 [Info] UnwatchMetadata(): indexer 5020693e9ae5c52168d414b02f922ad8 2023-10-21T04:10:00.104+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:10:00.105+05:30 [Info] UnwatchMetadata(): finish for indexer 5020693e9ae5c52168d414b02f922ad8 2023-10-21T04:10:00.105+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:48348->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:10:00.106+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:48812->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:10:00.117+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:10:00.117+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:10:00.117+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:10:00.117+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:10:00.117+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:10:00.124+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:56238->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:10:00.124+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:59820->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:10:00.125+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:59820. Reason = EOF. Cleaning Pipe. 2023-10-21T04:10:00.125+05:30 [Info] messageListener.start(): message channel closed. Remove peer 10:f6:dd:ba:5c:c1:0:4c as follower. 2023-10-21T04:10:00.127+05:30 [Info] switched currmeta from 586 -> 588 force true 2023-10-21T04:10:00.127+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:10:00.128+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] ... stopped 2023-10-21T04:10:00.128+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:10:00.128+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] ... stopped 2023-10-21T04:10:00.260+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:00.260+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:00.308+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Stopping releaseConnsRoutine 2023-10-21T04:10:00.350+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Stopping releaseConnsRoutine 2023-10-21T04:10:00.364+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:00.364+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:00.406+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:00.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9349082488081144, throttleDelayMs (new, old, change): (6, 8, -2) 2023-10-21T04:10:00.837+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:10:00.837+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:00.837+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:00.845+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:00.899+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:10:00.899+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:10:01.006+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:40458 closed 2023-10-21T04:10:01.033+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:57028. Error = read tcp4 127.0.0.1:9118->127.0.0.1:57028: read: connection reset by peer. Kill Pipe. 2023-10-21T04:10:01.033+05:30 [Info] messageListener.start(): message channel closed. Remove peer b5:72:71:79:21:bb:b6:b4 as follower. 2023-10-21T04:10:01.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9999739998049986, throttleDelayMs (new, old, change): (10, 6, 4) 2023-10-21T04:10:01.585+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:01.586+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:01.588+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:01.724+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:01.725+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:01.727+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:01.735+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 182] updated cluster rebalancing status to false 2023-10-21T04:10:01.749+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:10:01.749+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:01.751+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:01.781+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:01.781+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:01.795+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:02.023+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:40464 closed 2023-10-21T04:10:02.055+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57026. Reason = EOF. Cleaning Pipe. 2023-10-21T04:10:02.055+05:30 [Info] messageListener.start(): message channel closed. Remove peer fe:72:7d:94:31:9f:3:ba as follower. 2023-10-21T04:10:02.076+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:10:02.076+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:10:02.077+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9001. 2023-10-21T04:10:02.233+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:10:02.233+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(1) 2023-10-21T04:10:02.450+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9904591653276241 2023-10-21T04:10:02.681+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:10:02.681+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(1) 2023-10-21T04:10:02.699+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:10:02.700+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(2) 2023-10-21T04:10:02.899+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:10:02.899+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:02.899+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:02.902+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:02.905+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:02.905+05:30 [Warn] validateNodesAndSvs - Failed as len(nodes): 3 != len(nodesExt): 4 2023-10-21T04:10:02.905+05:30 [Warn] HostNames Nodes: [172.31.5.112:9000 127.0.0.1:9003 127.0.0.1:9004] NodesExt: [172.31.5.112:9000 127.0.0.1:9001 127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:10:02.913+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:02.913+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:02.919+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:02.941+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:02.941+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:02.945+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:02.958+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:10:02.958+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:02.963+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:02.963+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:02.967+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:02.987+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:02.987+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:02.990+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:03.344+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 237] aggregate storage info: map[bucket_%2:map[kv:5642432 index:15809490] bucket_1:map[kv:6367424 index:15862336]] 2023-10-21T04:10:03.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.899161795751138, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:10:04.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7776793234181977, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:10:04.952+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:04.963+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:04.963+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:04.966+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:10:08.206+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is enabled 2023-10-21T04:10:10.270+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43610" doAuth() ... 2023-10-21T04:10:10.270+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43610" doReceive() ... 2023-10-21T04:10:10.303+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43616" doAuth() ... 2023-10-21T04:10:10.307+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43616" doReceive() ... 2023-10-21T04:10:10.336+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43626" doAuth() ... 2023-10-21T04:10:10.336+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43626" doReceive() ... 2023-10-21T04:10:10.370+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43632" doAuth() ... 2023-10-21T04:10:10.370+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43632" doReceive() ... 2023-10-21T04:10:10.426+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43636" doAuth() ... 2023-10-21T04:10:10.429+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43636" doReceive() ... 2023-10-21T04:10:10.432+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43638" doAuth() ... 2023-10-21T04:10:10.435+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43638" doReceive() ... 2023-10-21T04:10:10.470+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43650" doAuth() ... 2023-10-21T04:10:10.470+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43650" doReceive() ... 2023-10-21T04:10:10.545+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43658" doAuth() ... 2023-10-21T04:10:10.547+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43658" doReceive() ... 2023-10-21T04:10:10.681+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43668" doAuth() ... 2023-10-21T04:10:10.683+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43668" doReceive() ... 2023-10-21T04:10:10.698+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43672" doAuth() ... 2023-10-21T04:10:10.698+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43672" doReceive() ... 2023-10-21T04:10:10.738+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43678" doAuth() ... 2023-10-21T04:10:10.741+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43678" doReceive() ... 2023-10-21T04:10:10.768+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43682" doAuth() ... 2023-10-21T04:10:10.770+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43682" doReceive() ... 2023-10-21T04:10:10.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43684" doAuth() ... 2023-10-21T04:10:10.776+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43684" doReceive() ... 2023-10-21T04:10:10.801+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43688" doAuth() ... 2023-10-21T04:10:10.805+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43688" doReceive() ... 2023-10-21T04:10:10.849+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43690" doAuth() ... 2023-10-21T04:10:10.849+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43690" doReceive() ... 2023-10-21T04:10:10.894+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43694" doAuth() ... 2023-10-21T04:10:10.894+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43694" doReceive() ... 2023-10-21T04:10:10.898+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43696" doAuth() ... 2023-10-21T04:10:10.906+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43696" doReceive() ... 2023-10-21T04:10:10.968+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43708" doAuth() ... 2023-10-21T04:10:10.968+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43708" doReceive() ... 2023-10-21T04:10:10.984+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43710" doAuth() ... 2023-10-21T04:10:10.990+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43710" doReceive() ... 2023-10-21T04:10:11.083+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43714" doAuth() ... 2023-10-21T04:10:11.083+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43714" doReceive() ... 2023-10-21T04:10:11.088+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43716" doAuth() ... 2023-10-21T04:10:11.089+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43716" doReceive() ... 2023-10-21T04:10:11.099+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43718" doAuth() ... 2023-10-21T04:10:11.099+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43718" doReceive() ... 2023-10-21T04:10:11.221+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43736" doAuth() ... 2023-10-21T04:10:11.224+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43736" doReceive() ... 2023-10-21T04:10:11.326+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43748" doAuth() ... 2023-10-21T04:10:11.329+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43748" doReceive() ... 2023-10-21T04:10:11.361+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43758" doAuth() ... 2023-10-21T04:10:11.367+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43758" doReceive() ... 2023-10-21T04:10:11.385+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43764" doAuth() ... 2023-10-21T04:10:11.389+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43764" doReceive() ... 2023-10-21T04:10:11.414+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43768" doAuth() ... 2023-10-21T04:10:11.418+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43768" doReceive() ... 2023-10-21T04:10:11.461+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43774" doAuth() ... 2023-10-21T04:10:11.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43774" doReceive() ... 2023-10-21T04:10:11.466+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43776" doAuth() ... 2023-10-21T04:10:11.469+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43776" doReceive() ... 2023-10-21T04:10:11.555+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43790" doAuth() ... 2023-10-21T04:10:11.555+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43790" doReceive() ... 2023-10-21T04:10:11.573+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43800" doAuth() ... 2023-10-21T04:10:11.576+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43800" doReceive() ... 2023-10-21T04:10:11.729+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43830" doAuth() ... 2023-10-21T04:10:11.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43830" doReceive() ... 2023-10-21T04:10:11.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43840" doAuth() ... 2023-10-21T04:10:11.782+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43840" doReceive() ... 2023-10-21T04:10:11.797+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43844" doAuth() ... 2023-10-21T04:10:11.798+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43844" doReceive() ... 2023-10-21T04:10:11.837+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43848" doAuth() ... 2023-10-21T04:10:11.843+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43848" doReceive() ... 2023-10-21T04:10:12.005+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43860" doAuth() ... 2023-10-21T04:10:12.006+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43860" doReceive() ... 2023-10-21T04:10:12.009+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43862" doAuth() ... 2023-10-21T04:10:12.012+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43862" doReceive() ... 2023-10-21T04:10:12.063+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43866" doAuth() ... 2023-10-21T04:10:12.067+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43866" doReceive() ... 2023-10-21T04:10:12.070+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43868" doAuth() ... 2023-10-21T04:10:12.072+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43868" doReceive() ... 2023-10-21T04:10:12.086+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43874" doAuth() ... 2023-10-21T04:10:12.091+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43874" doReceive() ... 2023-10-21T04:10:12.135+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43878" doAuth() ... 2023-10-21T04:10:12.139+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43878" doReceive() ... 2023-10-21T04:10:12.144+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43880" doAuth() ... 2023-10-21T04:10:12.147+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43880" doReceive() ... 2023-10-21T04:10:12.178+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43886" doAuth() ... 2023-10-21T04:10:12.184+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43886" doReceive() ... 2023-10-21T04:10:12.221+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43892" doAuth() ... 2023-10-21T04:10:12.223+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43892" doReceive() ... 2023-10-21T04:10:12.227+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43894" doAuth() ... 2023-10-21T04:10:12.229+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43894" doReceive() ... 2023-10-21T04:10:12.231+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43896" doAuth() ... 2023-10-21T04:10:12.234+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43896" doReceive() ... 2023-10-21T04:10:12.285+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43900" doAuth() ... 2023-10-21T04:10:12.289+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43900" doReceive() ... 2023-10-21T04:10:12.348+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43908" doAuth() ... 2023-10-21T04:10:12.348+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43908" doReceive() ... 2023-10-21T04:10:12.355+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43910" doAuth() ... 2023-10-21T04:10:12.355+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43910" doReceive() ... 2023-10-21T04:10:12.363+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43912" doAuth() ... 2023-10-21T04:10:12.363+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43912" doReceive() ... 2023-10-21T04:10:12.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.958828688851995, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:10:12.465+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:10:12.465+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:10:12.466+05:30 [Info] janitor: running cleanup. 2023-10-21T04:10:12.471+05:30 [Info] memstats {"Alloc":105765520, "TotalAlloc":108590584, "Sys":126042392, "Lookups":0, "Mallocs":201105,"Frees":53910, "HeapAlloc":105765520, "HeapSys":111280128, "HeapIdle":2596864, "HeapInuse":108683264,"HeapReleased":2498560, "HeapObjects":147195,"MSpanInuse":372800, "MSpanSys": 375360, "StackInuse": 1966080,"GCSys":9908120, "LastGC":1697841551979455197,"PauseTotalNs":1595588, "PauseNs":[403672, 342145, 77128, 56629, 716014], "NumGC":5} 2023-10-21T04:10:13.242+05:30 [Info] connected with 2 indexers 2023-10-21T04:10:13.243+05:30 [Info] client stats current counts: current: 76, not current: 0 2023-10-21T04:10:13.322+05:30 [Info] num concurrent scans {0} 2023-10-21T04:10:13.324+05:30 [Info] average scan response {0 ms} 2023-10-21T04:10:13.345+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:10:13.345+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:10:13.370+05:30 [Info] Indexer::ReadMemstats Time Taken 1.67591ms 2023-10-21T04:10:13.452+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9628439924731392, throttleDelayMs (new, old, change): (2, 1, 1) 2023-10-21T04:10:15.453+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9272223496655736, throttleDelayMs (new, old, change): (0, 2, -2) 2023-10-21T04:10:18.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9800460543042406, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:10:19.385+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44206" doAuth() ... 2023-10-21T04:10:19.387+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44206" doReceive() ... 2023-10-21T04:10:19.421+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44212" doAuth() ... 2023-10-21T04:10:19.422+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44212" doReceive() ... 2023-10-21T04:10:19.445+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44220" doAuth() ... 2023-10-21T04:10:19.445+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44220" doReceive() ... 2023-10-21T04:10:19.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9618998168051295, throttleDelayMs (new, old, change): (4, 3, 1) 2023-10-21T04:10:19.623+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44242" doAuth() ... 2023-10-21T04:10:19.624+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44242" doReceive() ... 2023-10-21T04:10:19.659+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44246" doAuth() ... 2023-10-21T04:10:19.659+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44246" doReceive() ... 2023-10-21T04:10:19.707+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44258" doAuth() ... 2023-10-21T04:10:19.707+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44258" doReceive() ... 2023-10-21T04:10:19.756+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44262" doAuth() ... 2023-10-21T04:10:19.757+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44262" doReceive() ... 2023-10-21T04:10:19.788+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44270" doAuth() ... 2023-10-21T04:10:19.788+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44270" doReceive() ... 2023-10-21T04:10:19.791+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44272" doAuth() ... 2023-10-21T04:10:19.792+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44272" doReceive() ... 2023-10-21T04:10:19.795+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44274" doAuth() ... 2023-10-21T04:10:19.795+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44274" doReceive() ... 2023-10-21T04:10:19.813+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44278" doAuth() ... 2023-10-21T04:10:19.814+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44278" doReceive() ... 2023-10-21T04:10:19.817+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44280" doAuth() ... 2023-10-21T04:10:19.818+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44280" doReceive() ... 2023-10-21T04:10:19.852+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44282" doAuth() ... 2023-10-21T04:10:19.853+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44282" doReceive() ... 2023-10-21T04:10:19.888+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44286" doAuth() ... 2023-10-21T04:10:19.888+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44286" doReceive() ... 2023-10-21T04:10:19.928+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44294" doAuth() ... 2023-10-21T04:10:19.930+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44294" doReceive() ... 2023-10-21T04:10:19.967+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44298" doAuth() ... 2023-10-21T04:10:19.968+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44298" doReceive() ... 2023-10-21T04:10:19.981+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44300" doAuth() ... 2023-10-21T04:10:19.984+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44300" doReceive() ... 2023-10-21T04:10:20.007+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44308" doAuth() ... 2023-10-21T04:10:20.007+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44308" doReceive() ... 2023-10-21T04:10:20.010+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44310" doAuth() ... 2023-10-21T04:10:20.011+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44310" doReceive() ... 2023-10-21T04:10:20.081+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44318" doAuth() ... 2023-10-21T04:10:20.082+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44318" doReceive() ... 2023-10-21T04:10:20.096+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44320" doAuth() ... 2023-10-21T04:10:20.096+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44320" doReceive() ... 2023-10-21T04:10:20.129+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44322" doAuth() ... 2023-10-21T04:10:20.129+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44322" doReceive() ... 2023-10-21T04:10:20.154+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44326" doAuth() ... 2023-10-21T04:10:20.155+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44326" doReceive() ... 2023-10-21T04:10:20.168+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44328" doAuth() ... 2023-10-21T04:10:20.169+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44328" doReceive() ... 2023-10-21T04:10:20.183+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44332" doAuth() ... 2023-10-21T04:10:20.184+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44332" doReceive() ... 2023-10-21T04:10:20.187+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44334" doAuth() ... 2023-10-21T04:10:20.187+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44334" doReceive() ... 2023-10-21T04:10:20.276+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44340" doAuth() ... 2023-10-21T04:10:20.276+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44340" doReceive() ... 2023-10-21T04:10:20.279+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44342" doAuth() ... 2023-10-21T04:10:20.280+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44342" doReceive() ... 2023-10-21T04:10:20.283+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44346" doAuth() ... 2023-10-21T04:10:20.284+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44346" doReceive() ... 2023-10-21T04:10:20.300+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44348" doAuth() ... 2023-10-21T04:10:20.301+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44348" doReceive() ... 2023-10-21T04:10:20.371+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44354" doAuth() ... 2023-10-21T04:10:20.371+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44354" doReceive() ... 2023-10-21T04:10:20.375+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44356" doAuth() ... 2023-10-21T04:10:20.375+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44356" doReceive() ... 2023-10-21T04:10:20.388+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44358" doAuth() ... 2023-10-21T04:10:20.389+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44358" doReceive() ... 2023-10-21T04:10:20.455+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.798674239021732, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:10:20.496+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44366" doAuth() ... 2023-10-21T04:10:20.497+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44366" doReceive() ... 2023-10-21T04:10:20.510+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44368" doAuth() ... 2023-10-21T04:10:20.511+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44368" doReceive() ... 2023-10-21T04:10:20.513+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44370" doAuth() ... 2023-10-21T04:10:20.514+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44370" doReceive() ... 2023-10-21T04:10:20.599+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44384" doAuth() ... 2023-10-21T04:10:20.599+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44384" doReceive() ... 2023-10-21T04:10:20.628+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44390" doAuth() ... 2023-10-21T04:10:20.628+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44390" doReceive() ... 2023-10-21T04:10:20.631+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44392" doAuth() ... 2023-10-21T04:10:20.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44392" doReceive() ... 2023-10-21T04:10:20.655+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44396" doAuth() ... 2023-10-21T04:10:20.656+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44396" doReceive() ... 2023-10-21T04:10:20.658+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44398" doAuth() ... 2023-10-21T04:10:20.658+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44398" doReceive() ... 2023-10-21T04:10:20.667+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44400" doAuth() ... 2023-10-21T04:10:20.668+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44400" doReceive() ... 2023-10-21T04:10:20.694+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44404" doAuth() ... 2023-10-21T04:10:20.695+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44404" doReceive() ... 2023-10-21T04:10:20.708+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44406" doAuth() ... 2023-10-21T04:10:20.709+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44406" doReceive() ... 2023-10-21T04:10:20.755+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44414" doAuth() ... 2023-10-21T04:10:20.755+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44414" doReceive() ... 2023-10-21T04:10:24.657+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44528" doAuth() ... 2023-10-21T04:10:24.659+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44528" doReceive() ... 2023-10-21T04:10:24.717+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44540" doAuth() ... 2023-10-21T04:10:24.720+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44540" doReceive() ... 2023-10-21T04:10:24.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44550" doAuth() ... 2023-10-21T04:10:24.775+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44550" doReceive() ... 2023-10-21T04:10:24.781+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44554" doAuth() ... 2023-10-21T04:10:24.783+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44554" doReceive() ... 2023-10-21T04:10:24.785+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44556" doAuth() ... 2023-10-21T04:10:24.787+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44556" doReceive() ... 2023-10-21T04:10:24.810+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44560" doAuth() ... 2023-10-21T04:10:24.811+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44560" doReceive() ... 2023-10-21T04:10:24.869+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44568" doAuth() ... 2023-10-21T04:10:24.871+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44568" doReceive() ... 2023-10-21T04:10:24.897+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44570" doAuth() ... 2023-10-21T04:10:24.899+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44570" doReceive() ... 2023-10-21T04:10:24.901+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44572" doAuth() ... 2023-10-21T04:10:24.903+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44572" doReceive() ... 2023-10-21T04:10:24.911+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44574" doAuth() ... 2023-10-21T04:10:24.913+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44574" doReceive() ... 2023-10-21T04:10:24.916+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44576" doAuth() ... 2023-10-21T04:10:24.918+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44576" doReceive() ... 2023-10-21T04:10:24.943+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44578" doAuth() ... 2023-10-21T04:10:24.945+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44578" doReceive() ... 2023-10-21T04:10:25.015+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44580" doAuth() ... 2023-10-21T04:10:25.024+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44580" doReceive() ... 2023-10-21T04:10:25.026+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44582" doAuth() ... 2023-10-21T04:10:25.027+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44582" doReceive() ... 2023-10-21T04:10:25.033+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44584" doAuth() ... 2023-10-21T04:10:25.034+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44584" doReceive() ... 2023-10-21T04:10:25.071+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44588" doAuth() ... 2023-10-21T04:10:25.071+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44588" doReceive() ... 2023-10-21T04:10:25.074+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44590" doAuth() ... 2023-10-21T04:10:25.075+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44590" doReceive() ... 2023-10-21T04:10:25.083+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44592" doAuth() ... 2023-10-21T04:10:25.084+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44592" doReceive() ... 2023-10-21T04:10:25.086+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44594" doAuth() ... 2023-10-21T04:10:25.087+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44594" doReceive() ... 2023-10-21T04:10:25.090+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44596" doAuth() ... 2023-10-21T04:10:25.091+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44596" doReceive() ... 2023-10-21T04:10:25.102+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44598" doAuth() ... 2023-10-21T04:10:25.103+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44598" doReceive() ... 2023-10-21T04:10:25.104+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44600" doAuth() ... 2023-10-21T04:10:25.105+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44600" doReceive() ... 2023-10-21T04:10:25.117+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44602" doAuth() ... 2023-10-21T04:10:25.118+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44602" doReceive() ... 2023-10-21T04:10:25.146+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44610" doAuth() ... 2023-10-21T04:10:25.147+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44610" doReceive() ... 2023-10-21T04:10:25.174+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44618" doAuth() ... 2023-10-21T04:10:25.174+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44618" doReceive() ... 2023-10-21T04:10:25.199+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44622" doAuth() ... 2023-10-21T04:10:25.199+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44622" doReceive() ... 2023-10-21T04:10:25.210+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44626" doAuth() ... 2023-10-21T04:10:25.211+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44626" doReceive() ... 2023-10-21T04:10:25.226+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44630" doAuth() ... 2023-10-21T04:10:25.227+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44630" doReceive() ... 2023-10-21T04:10:25.230+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44632" doAuth() ... 2023-10-21T04:10:25.231+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44632" doReceive() ... 2023-10-21T04:10:25.260+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44642" doAuth() ... 2023-10-21T04:10:25.261+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44642" doReceive() ... 2023-10-21T04:10:25.264+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44644" doAuth() ... 2023-10-21T04:10:25.265+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44644" doReceive() ... 2023-10-21T04:10:25.293+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44648" doAuth() ... 2023-10-21T04:10:25.293+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44648" doReceive() ... 2023-10-21T04:10:25.299+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44652" doAuth() ... 2023-10-21T04:10:25.300+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44652" doReceive() ... 2023-10-21T04:10:25.344+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44662" doAuth() ... 2023-10-21T04:10:25.345+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44662" doReceive() ... 2023-10-21T04:10:25.353+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44664" doAuth() ... 2023-10-21T04:10:25.353+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44664" doReceive() ... 2023-10-21T04:10:25.356+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44666" doAuth() ... 2023-10-21T04:10:25.357+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44666" doReceive() ... 2023-10-21T04:10:25.362+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44670" doAuth() ... 2023-10-21T04:10:25.362+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44670" doReceive() ... 2023-10-21T04:10:25.373+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44674" doAuth() ... 2023-10-21T04:10:25.374+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44674" doReceive() ... 2023-10-21T04:10:25.394+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44682" doAuth() ... 2023-10-21T04:10:25.395+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44682" doReceive() ... 2023-10-21T04:10:25.404+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44684" doAuth() ... 2023-10-21T04:10:25.405+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44684" doReceive() ... 2023-10-21T04:10:25.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44690" doAuth() ... 2023-10-21T04:10:25.465+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44690" doReceive() ... 2023-10-21T04:10:25.473+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44692" doAuth() ... 2023-10-21T04:10:25.474+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44692" doReceive() ... 2023-10-21T04:10:25.499+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44696" doAuth() ... 2023-10-21T04:10:25.500+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44696" doReceive() ... 2023-10-21T04:10:25.503+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44698" doAuth() ... 2023-10-21T04:10:25.504+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44698" doReceive() ... 2023-10-21T04:10:25.566+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44712" doAuth() ... 2023-10-21T04:10:25.567+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44712" doReceive() ... 2023-10-21T04:10:25.569+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44714" doAuth() ... 2023-10-21T04:10:25.569+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44714" doReceive() ... 2023-10-21T04:10:25.572+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44716" doAuth() ... 2023-10-21T04:10:25.572+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44716" doReceive() ... 2023-10-21T04:10:25.614+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44728" doAuth() ... 2023-10-21T04:10:25.615+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44728" doReceive() ... 2023-10-21T04:10:25.637+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44732" doAuth() ... 2023-10-21T04:10:25.637+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44732" doReceive() ... 2023-10-21T04:10:25.640+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44734" doAuth() ... 2023-10-21T04:10:25.641+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44734" doReceive() ... 2023-10-21T04:10:25.693+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44744" doAuth() ... 2023-10-21T04:10:25.694+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44744" doReceive() ... 2023-10-21T04:10:25.703+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44746" doAuth() ... 2023-10-21T04:10:25.703+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44746" doReceive() ... 2023-10-21T04:10:25.711+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44748" doAuth() ... 2023-10-21T04:10:25.712+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44748" doReceive() ... 2023-10-21T04:10:25.728+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44752" doAuth() ... 2023-10-21T04:10:25.729+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44752" doReceive() ... 2023-10-21T04:10:25.754+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44754" doAuth() ... 2023-10-21T04:10:25.755+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44754" doReceive() ... 2023-10-21T04:10:27.911+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is enabled 2023-10-21T04:10:29.275+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:10:29.275+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:10:29.282+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. err: operation_canceled 2023-10-21T04:10:29.282+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:10:29.282+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:10:29.289+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. err: operation_canceled 2023-10-21T04:10:29.289+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:10:29.289+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:10:29.290+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:10:29.290+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:10:29.290+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:10:29.291+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:10:29.324+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44806" doAuth() ... 2023-10-21T04:10:29.325+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44806" doReceive() ... 2023-10-21T04:10:29.327+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44808" doAuth() ... 2023-10-21T04:10:29.328+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44808" doReceive() ... 2023-10-21T04:10:29.346+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44816" doAuth() ... 2023-10-21T04:10:29.346+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44816" doReceive() ... 2023-10-21T04:10:29.370+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44822" doAuth() ... 2023-10-21T04:10:29.371+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44822" doReceive() ... 2023-10-21T04:10:29.399+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44828" doAuth() ... 2023-10-21T04:10:29.400+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44828" doReceive() ... 2023-10-21T04:10:29.402+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44830" doAuth() ... 2023-10-21T04:10:29.402+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44830" doReceive() ... 2023-10-21T04:10:29.413+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44834" doAuth() ... 2023-10-21T04:10:29.413+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44834" doReceive() ... 2023-10-21T04:10:29.437+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44838" doAuth() ... 2023-10-21T04:10:29.437+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44838" doReceive() ... 2023-10-21T04:10:29.446+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44840" doAuth() ... 2023-10-21T04:10:29.447+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44840" doReceive() ... 2023-10-21T04:10:29.490+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44848" doAuth() ... 2023-10-21T04:10:29.491+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44848" doReceive() ... 2023-10-21T04:10:29.510+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44852" doAuth() ... 2023-10-21T04:10:29.511+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44852" doReceive() ... 2023-10-21T04:10:29.524+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44854" doAuth() ... 2023-10-21T04:10:29.525+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44854" doReceive() ... 2023-10-21T04:10:29.533+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44856" doAuth() ... 2023-10-21T04:10:29.533+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44856" doReceive() ... 2023-10-21T04:10:29.536+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44858" doAuth() ... 2023-10-21T04:10:29.537+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44858" doReceive() ... 2023-10-21T04:10:29.540+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44860" doAuth() ... 2023-10-21T04:10:29.540+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44860" doReceive() ... 2023-10-21T04:10:29.547+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44862" doAuth() ... 2023-10-21T04:10:29.548+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44862" doReceive() ... 2023-10-21T04:10:29.562+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44864" doAuth() ... 2023-10-21T04:10:29.562+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44864" doReceive() ... 2023-10-21T04:10:29.565+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44866" doAuth() ... 2023-10-21T04:10:29.565+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44866" doReceive() ... 2023-10-21T04:10:29.592+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44868" doAuth() ... 2023-10-21T04:10:29.593+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44868" doReceive() ... 2023-10-21T04:10:29.631+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44882" doAuth() ... 2023-10-21T04:10:29.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44882" doReceive() ... 2023-10-21T04:10:29.634+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44884" doAuth() ... 2023-10-21T04:10:29.634+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44884" doReceive() ... 2023-10-21T04:10:29.666+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44890" doAuth() ... 2023-10-21T04:10:29.667+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44890" doReceive() ... 2023-10-21T04:10:29.683+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44894" doAuth() ... 2023-10-21T04:10:29.683+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44894" doReceive() ... 2023-10-21T04:10:29.697+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44896" doAuth() ... 2023-10-21T04:10:29.698+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44896" doReceive() ... 2023-10-21T04:10:29.700+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44898" doAuth() ... 2023-10-21T04:10:29.702+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44898" doReceive() ... 2023-10-21T04:10:29.705+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44900" doAuth() ... 2023-10-21T04:10:29.707+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44900" doReceive() ... 2023-10-21T04:10:29.731+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44906" doAuth() ... 2023-10-21T04:10:29.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44906" doReceive() ... 2023-10-21T04:10:29.755+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44910" doAuth() ... 2023-10-21T04:10:29.757+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44910" doReceive() ... 2023-10-21T04:10:29.781+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44914" doAuth() ... 2023-10-21T04:10:29.781+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44914" doReceive() ... 2023-10-21T04:10:29.797+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44918" doAuth() ... 2023-10-21T04:10:29.799+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44918" doReceive() ... 2023-10-21T04:10:29.801+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44920" doAuth() ... 2023-10-21T04:10:29.803+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44920" doReceive() ... 2023-10-21T04:10:29.811+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44924" doAuth() ... 2023-10-21T04:10:29.813+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44924" doReceive() ... 2023-10-21T04:10:29.815+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44926" doAuth() ... 2023-10-21T04:10:29.817+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44926" doReceive() ... 2023-10-21T04:10:29.819+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44928" doAuth() ... 2023-10-21T04:10:29.821+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44928" doReceive() ... 2023-10-21T04:10:29.823+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44930" doAuth() ... 2023-10-21T04:10:29.825+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44930" doReceive() ... 2023-10-21T04:10:29.827+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44932" doAuth() ... 2023-10-21T04:10:29.829+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44932" doReceive() ... 2023-10-21T04:10:29.845+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44938" doAuth() ... 2023-10-21T04:10:29.847+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44938" doReceive() ... 2023-10-21T04:10:29.859+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44942" doAuth() ... 2023-10-21T04:10:29.861+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44942" doReceive() ... 2023-10-21T04:10:29.883+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44948" doAuth() ... 2023-10-21T04:10:29.885+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44948" doReceive() ... 2023-10-21T04:10:29.894+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44950" doAuth() ... 2023-10-21T04:10:29.896+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44950" doReceive() ... 2023-10-21T04:10:29.898+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44952" doAuth() ... 2023-10-21T04:10:29.900+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44952" doReceive() ... 2023-10-21T04:10:29.903+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44954" doAuth() ... 2023-10-21T04:10:29.905+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44954" doReceive() ... 2023-10-21T04:10:29.913+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44956" doAuth() ... 2023-10-21T04:10:29.915+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44956" doReceive() ... 2023-10-21T04:10:29.922+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44960" doAuth() ... 2023-10-21T04:10:29.924+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44960" doReceive() ... 2023-10-21T04:10:29.932+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44962" doAuth() ... 2023-10-21T04:10:29.934+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44962" doReceive() ... 2023-10-21T04:10:29.946+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44966" doAuth() ... 2023-10-21T04:10:29.948+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44966" doReceive() ... 2023-10-21T04:10:29.958+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44968" doAuth() ... 2023-10-21T04:10:29.960+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44968" doReceive() ... 2023-10-21T04:10:29.977+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44972" doAuth() ... 2023-10-21T04:10:29.979+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44972" doReceive() ... 2023-10-21T04:10:30.006+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44978" doAuth() ... 2023-10-21T04:10:30.008+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44978" doReceive() ... 2023-10-21T04:10:30.026+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44982" doAuth() ... 2023-10-21T04:10:30.028+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44982" doReceive() ... 2023-10-21T04:10:30.048+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44984" doAuth() ... 2023-10-21T04:10:30.050+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44984" doReceive() ... 2023-10-21T04:10:30.077+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44986" doAuth() ... 2023-10-21T04:10:30.079+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:44986" doReceive() ... 2023-10-21T04:10:30.192+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45000" doAuth() ... 2023-10-21T04:10:30.195+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45000" doReceive() ... 2023-10-21T04:10:30.198+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45002" doAuth() ... 2023-10-21T04:10:30.200+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45002" doReceive() ... 2023-10-21T04:10:30.983+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45026" doAuth() ... 2023-10-21T04:10:30.985+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45026" doReceive() ... 2023-10-21T04:10:34.222+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45122" doAuth() ... 2023-10-21T04:10:34.223+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45122" doReceive() ... 2023-10-21T04:10:34.226+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45124" doAuth() ... 2023-10-21T04:10:34.226+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45124" doReceive() ... 2023-10-21T04:10:34.239+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45126" doAuth() ... 2023-10-21T04:10:34.240+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45126" doReceive() ... 2023-10-21T04:10:34.242+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45128" doAuth() ... 2023-10-21T04:10:34.242+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45128" doReceive() ... 2023-10-21T04:10:34.245+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45130" doAuth() ... 2023-10-21T04:10:34.246+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45130" doReceive() ... 2023-10-21T04:10:34.248+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45132" doAuth() ... 2023-10-21T04:10:34.248+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45132" doReceive() ... 2023-10-21T04:10:34.254+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45136" doAuth() ... 2023-10-21T04:10:34.254+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45136" doReceive() ... 2023-10-21T04:10:34.291+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45146" doAuth() ... 2023-10-21T04:10:34.292+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45146" doReceive() ... 2023-10-21T04:10:34.294+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45148" doAuth() ... 2023-10-21T04:10:34.295+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45148" doReceive() ... 2023-10-21T04:10:34.297+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45150" doAuth() ... 2023-10-21T04:10:34.298+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45150" doReceive() ... 2023-10-21T04:10:34.300+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45152" doAuth() ... 2023-10-21T04:10:34.301+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45152" doReceive() ... 2023-10-21T04:10:34.303+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45154" doAuth() ... 2023-10-21T04:10:34.304+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45154" doReceive() ... 2023-10-21T04:10:34.341+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45164" doAuth() ... 2023-10-21T04:10:34.342+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45164" doReceive() ... 2023-10-21T04:10:34.358+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45168" doAuth() ... 2023-10-21T04:10:34.358+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45168" doReceive() ... 2023-10-21T04:10:34.366+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45170" doAuth() ... 2023-10-21T04:10:34.367+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45170" doReceive() ... 2023-10-21T04:10:34.406+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45182" doAuth() ... 2023-10-21T04:10:34.407+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45182" doReceive() ... 2023-10-21T04:10:34.426+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45184" doAuth() ... 2023-10-21T04:10:34.427+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45184" doReceive() ... 2023-10-21T04:10:34.443+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45188" doAuth() ... 2023-10-21T04:10:34.444+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45188" doReceive() ... 2023-10-21T04:10:34.475+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45190" doAuth() ... 2023-10-21T04:10:34.476+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45190" doReceive() ... 2023-10-21T04:10:34.492+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45200" doAuth() ... 2023-10-21T04:10:34.493+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45200" doReceive() ... 2023-10-21T04:10:34.537+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45208" doAuth() ... 2023-10-21T04:10:34.538+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45208" doReceive() ... 2023-10-21T04:10:34.549+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45212" doAuth() ... 2023-10-21T04:10:34.550+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45212" doReceive() ... 2023-10-21T04:10:34.553+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45214" doAuth() ... 2023-10-21T04:10:34.553+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45214" doReceive() ... 2023-10-21T04:10:34.575+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45218" doAuth() ... 2023-10-21T04:10:34.576+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45218" doReceive() ... 2023-10-21T04:10:34.698+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45238" doAuth() ... 2023-10-21T04:10:34.699+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45238" doReceive() ... 2023-10-21T04:10:34.701+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45240" doAuth() ... 2023-10-21T04:10:34.702+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45240" doReceive() ... 2023-10-21T04:10:34.725+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45246" doAuth() ... 2023-10-21T04:10:34.728+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45246" doReceive() ... 2023-10-21T04:10:34.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45248" doAuth() ... 2023-10-21T04:10:34.734+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45248" doReceive() ... 2023-10-21T04:10:34.744+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45250" doAuth() ... 2023-10-21T04:10:34.747+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45250" doReceive() ... 2023-10-21T04:10:34.759+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45252" doAuth() ... 2023-10-21T04:10:34.762+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45252" doReceive() ... 2023-10-21T04:10:34.771+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45254" doAuth() ... 2023-10-21T04:10:34.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45254" doReceive() ... 2023-10-21T04:10:34.776+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45256" doAuth() ... 2023-10-21T04:10:34.778+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45256" doReceive() ... 2023-10-21T04:10:34.792+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45260" doAuth() ... 2023-10-21T04:10:34.793+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45260" doReceive() ... 2023-10-21T04:10:34.802+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45262" doAuth() ... 2023-10-21T04:10:34.802+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45262" doReceive() ... 2023-10-21T04:10:34.810+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45264" doAuth() ... 2023-10-21T04:10:34.811+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45264" doReceive() ... 2023-10-21T04:10:34.820+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45268" doAuth() ... 2023-10-21T04:10:34.823+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45268" doReceive() ... 2023-10-21T04:10:34.845+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45270" doAuth() ... 2023-10-21T04:10:34.847+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45270" doReceive() ... 2023-10-21T04:10:34.856+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45272" doAuth() ... 2023-10-21T04:10:34.858+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45272" doReceive() ... 2023-10-21T04:10:34.864+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45276" doAuth() ... 2023-10-21T04:10:34.867+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45276" doReceive() ... 2023-10-21T04:10:34.869+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45278" doAuth() ... 2023-10-21T04:10:34.870+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45278" doReceive() ... 2023-10-21T04:10:34.873+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45280" doAuth() ... 2023-10-21T04:10:34.873+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45280" doReceive() ... 2023-10-21T04:10:34.891+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45288" doAuth() ... 2023-10-21T04:10:34.891+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45288" doReceive() ... 2023-10-21T04:10:34.894+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45290" doAuth() ... 2023-10-21T04:10:34.895+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45290" doReceive() ... 2023-10-21T04:10:34.930+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45294" doAuth() ... 2023-10-21T04:10:34.932+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45294" doReceive() ... 2023-10-21T04:10:34.946+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45296" doAuth() ... 2023-10-21T04:10:34.948+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45296" doReceive() ... 2023-10-21T04:10:34.980+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45302" doAuth() ... 2023-10-21T04:10:34.982+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45302" doReceive() ... 2023-10-21T04:10:35.024+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45312" doAuth() ... 2023-10-21T04:10:35.026+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45312" doReceive() ... 2023-10-21T04:10:35.040+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45314" doAuth() ... 2023-10-21T04:10:35.042+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45314" doReceive() ... 2023-10-21T04:10:35.093+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45318" doAuth() ... 2023-10-21T04:10:35.094+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45318" doReceive() ... 2023-10-21T04:10:35.126+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45320" doAuth() ... 2023-10-21T04:10:35.127+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45320" doReceive() ... 2023-10-21T04:10:35.130+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45322" doAuth() ... 2023-10-21T04:10:35.131+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45322" doReceive() ... 2023-10-21T04:10:35.133+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45324" doAuth() ... 2023-10-21T04:10:35.136+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45324" doReceive() ... 2023-10-21T04:10:35.153+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45328" doAuth() ... 2023-10-21T04:10:35.155+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45328" doReceive() ... 2023-10-21T04:10:35.176+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45332" doAuth() ... 2023-10-21T04:10:35.177+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45332" doReceive() ... 2023-10-21T04:10:35.203+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45334" doAuth() ... 2023-10-21T04:10:35.205+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45334" doReceive() ... 2023-10-21T04:10:38.855+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45382" doAuth() ... 2023-10-21T04:10:38.856+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45382" doReceive() ... 2023-10-21T04:10:38.882+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45390" doAuth() ... 2023-10-21T04:10:38.882+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45390" doReceive() ... 2023-10-21T04:10:38.890+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45392" doAuth() ... 2023-10-21T04:10:38.891+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45392" doReceive() ... 2023-10-21T04:10:38.893+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45394" doAuth() ... 2023-10-21T04:10:38.894+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45394" doReceive() ... 2023-10-21T04:10:38.945+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45402" doAuth() ... 2023-10-21T04:10:38.945+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45402" doReceive() ... 2023-10-21T04:10:38.975+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45408" doAuth() ... 2023-10-21T04:10:38.976+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45408" doReceive() ... 2023-10-21T04:10:39.011+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45416" doAuth() ... 2023-10-21T04:10:39.011+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45416" doReceive() ... 2023-10-21T04:10:39.028+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45420" doAuth() ... 2023-10-21T04:10:39.029+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45420" doReceive() ... 2023-10-21T04:10:39.043+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45426" doAuth() ... 2023-10-21T04:10:39.044+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45426" doReceive() ... 2023-10-21T04:10:39.077+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45428" doAuth() ... 2023-10-21T04:10:39.077+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45428" doReceive() ... 2023-10-21T04:10:39.113+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45432" doAuth() ... 2023-10-21T04:10:39.113+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45432" doReceive() ... 2023-10-21T04:10:39.126+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45434" doAuth() ... 2023-10-21T04:10:39.127+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45434" doReceive() ... 2023-10-21T04:10:39.130+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45438" doAuth() ... 2023-10-21T04:10:39.130+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45438" doReceive() ... 2023-10-21T04:10:39.133+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45440" doAuth() ... 2023-10-21T04:10:39.134+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45440" doReceive() ... 2023-10-21T04:10:39.145+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45446" doAuth() ... 2023-10-21T04:10:39.146+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45446" doReceive() ... 2023-10-21T04:10:39.148+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45448" doAuth() ... 2023-10-21T04:10:39.149+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45448" doReceive() ... 2023-10-21T04:10:39.152+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45450" doAuth() ... 2023-10-21T04:10:39.152+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45450" doReceive() ... 2023-10-21T04:10:39.176+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45454" doAuth() ... 2023-10-21T04:10:39.176+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45454" doReceive() ... 2023-10-21T04:10:39.200+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45458" doAuth() ... 2023-10-21T04:10:39.200+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45458" doReceive() ... 2023-10-21T04:10:39.271+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45468" doAuth() ... 2023-10-21T04:10:39.272+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45468" doReceive() ... 2023-10-21T04:10:39.289+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45472" doAuth() ... 2023-10-21T04:10:39.289+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45472" doReceive() ... 2023-10-21T04:10:39.297+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45474" doAuth() ... 2023-10-21T04:10:39.297+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45474" doReceive() ... 2023-10-21T04:10:39.311+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45480" doAuth() ... 2023-10-21T04:10:39.312+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45480" doReceive() ... 2023-10-21T04:10:39.332+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45486" doAuth() ... 2023-10-21T04:10:39.333+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45486" doReceive() ... 2023-10-21T04:10:39.347+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45488" doAuth() ... 2023-10-21T04:10:39.347+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45488" doReceive() ... 2023-10-21T04:10:39.367+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45496" doAuth() ... 2023-10-21T04:10:39.367+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45496" doReceive() ... 2023-10-21T04:10:39.388+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45500" doAuth() ... 2023-10-21T04:10:39.388+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45500" doReceive() ... 2023-10-21T04:10:39.397+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45502" doAuth() ... 2023-10-21T04:10:39.397+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45502" doReceive() ... 2023-10-21T04:10:39.421+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45504" doAuth() ... 2023-10-21T04:10:39.422+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45504" doReceive() ... 2023-10-21T04:10:39.481+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45520" doAuth() ... 2023-10-21T04:10:39.482+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45520" doReceive() ... 2023-10-21T04:10:39.515+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45526" doAuth() ... 2023-10-21T04:10:39.516+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45526" doReceive() ... 2023-10-21T04:10:39.518+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45528" doAuth() ... 2023-10-21T04:10:39.519+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45528" doReceive() ... 2023-10-21T04:10:39.532+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45530" doAuth() ... 2023-10-21T04:10:39.533+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45530" doReceive() ... 2023-10-21T04:10:39.592+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45536" doAuth() ... 2023-10-21T04:10:39.593+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45536" doReceive() ... 2023-10-21T04:10:39.626+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45540" doAuth() ... 2023-10-21T04:10:39.627+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45540" doReceive() ... 2023-10-21T04:10:39.643+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45542" doAuth() ... 2023-10-21T04:10:39.643+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45542" doReceive() ... 2023-10-21T04:10:39.660+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45544" doAuth() ... 2023-10-21T04:10:39.660+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45544" doReceive() ... 2023-10-21T04:10:39.663+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45546" doAuth() ... 2023-10-21T04:10:39.664+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45546" doReceive() ... 2023-10-21T04:10:39.673+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45548" doAuth() ... 2023-10-21T04:10:39.673+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45548" doReceive() ... 2023-10-21T04:10:39.689+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45552" doAuth() ... 2023-10-21T04:10:39.690+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45552" doReceive() ... 2023-10-21T04:10:39.727+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45558" doAuth() ... 2023-10-21T04:10:39.728+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45558" doReceive() ... 2023-10-21T04:10:39.734+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45562" doAuth() ... 2023-10-21T04:10:39.735+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45562" doReceive() ... 2023-10-21T04:10:41.365+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 18201836893981255880 &{0} 2023-10-21T04:10:41.365+05:30 [Info] Indexer::handleDropIndex - IndexInstId 18201836893981255880 2023-10-21T04:10:41.366+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:10:41.366+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:10:41.366+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18201836893981255880 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:10:41.367+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [18201836893981255880] 2023-10-21T04:10:41.368+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [18201836893981255880] 2023-10-21T04:10:41.368+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:10:41.368+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [18201836893981255880] 2023-10-21T04:10:41.369+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 62064 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:10:41.369+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 18201836893981255880 Defn: DefnId: 14889401051926000176 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 62064 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:10:41.370+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.596872ms 2023-10-21T04:10:41.370+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 18201836893981255880, PartitionId 0, IndexDefnId 14889401051926000176. 2023-10-21T04:10:41.370+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:10:41.370+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 18201836893981255880 2023-10-21T04:10:41.370+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:10:41.370+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:10:41.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex] 2023-10-21T04:10:41.370+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex 2023-10-21T04:10:41.370+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex] 2023-10-21T04:10:41.370+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex] 2023-10-21T04:10:41.370+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex 2023-10-21T04:10:41.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex stopped 2023-10-21T04:10:41.370+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:10:41.370+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:10:41.370+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:10:41.370+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:10:41.375+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId Instances [18201836893981255880] 2023-10-21T04:10:41.378+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:10:41.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex] 2023-10-21T04:10:41.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex closed 2023-10-21T04:10:41.378+05:30 [Info] bucket_1/idx_secondary/Mainstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:10:41.379+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:10:41.379+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:10:41.379+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:10:41.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex] 2023-10-21T04:10:41.379+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex 2023-10-21T04:10:41.379+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex] 2023-10-21T04:10:41.379+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex] 2023-10-21T04:10:41.379+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex 2023-10-21T04:10:41.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex stopped 2023-10-21T04:10:41.379+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:10:41.379+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:10:41.379+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:10:41.379+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:10:41.382+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:10:41.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex] 2023-10-21T04:10:41.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex closed 2023-10-21T04:10:41.383+05:30 [Info] bucket_1/idx_secondary/Backstore#18201836893981255880:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:10:41.383+05:30 [Info] Indexer::cleanupIndexData IndexInst 18201836893981255880 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:10:41.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex ... 2023-10-21T04:10:41.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex ... 2023-10-21T04:10:41.384+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.384+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.386+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId 2023-10-21T04:10:41.386+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 62064 2023-10-21T04:10:41.387+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 16.948599ms 2023-10-21T04:10:41.387+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 21.610045ms 2023-10-21T04:10:41.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:10:41.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/mainIndex sucessfully destroyed 2023-10-21T04:10:41.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex ... 2023-10-21T04:10:41.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex ... 2023-10-21T04:10:41.391+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:10:41.391+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_18201836893981255880_0.index destroyed 2023-10-21T04:10:41.391+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 18201836893981255880, PartitionId 0, IndexDefnId 14889401051926000176. 2023-10-21T04:10:41.391+05:30 [Info] Indexer::cleanupIndexData IndexInst 18201836893981255880 Partition 0 Destroy Done 2023-10-21T04:10:41.394+05:30 [Info] switched currmeta from 588 -> 589 force true 2023-10-21T04:10:41.404+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.407+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.405+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 57.718564ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:10:41.412+05:30 [Info] switched currmeta from 589 -> 590 force true 2023-10-21T04:10:41.418+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.418+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.423+05:30 [Info] switched currmeta from 590 -> 590 force true 2023-10-21T04:10:41.458+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.458+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.463+05:30 [Info] switched currmeta from 590 -> 591 force true 2023-10-21T04:10:41.501+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.501+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.514+05:30 [Info] switched currmeta from 591 -> 592 force true 2023-10-21T04:10:41.523+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.523+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.529+05:30 [Info] switched currmeta from 592 -> 592 force true 2023-10-21T04:10:41.601+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4030261110898422634 &{0} 2023-10-21T04:10:41.601+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4030261110898422634 2023-10-21T04:10:41.602+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:10:41.602+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:10:41.602+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4030261110898422634 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:10:41.603+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4030261110898422634] 2023-10-21T04:10:41.603+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4030261110898422634] 2023-10-21T04:10:41.603+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:10:41.603+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4030261110898422634] 2023-10-21T04:10:41.604+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 9997 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:10:41.604+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4030261110898422634 Defn: DefnId: 4860549750398470299 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 9997 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:10:41.605+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4030261110898422634, PartitionId 0, IndexDefnId 4860549750398470299. 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:10:41.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex] 2023-10-21T04:10:41.605+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex] 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex] 2023-10-21T04:10:41.605+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex 2023-10-21T04:10:41.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex stopped 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:10:41.605+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:10:41.606+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId Instances [4030261110898422634] 2023-10-21T04:10:41.606+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4030261110898422634 2023-10-21T04:10:41.611+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId 2023-10-21T04:10:41.611+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 9997 2023-10-21T04:10:41.611+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 1.881688ms 2023-10-21T04:10:41.611+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 9.74587ms 2023-10-21T04:10:41.604+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.302385ms 2023-10-21T04:10:41.621+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:10:41.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex] 2023-10-21T04:10:41.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex closed 2023-10-21T04:10:41.622+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:10:41.622+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:10:41.622+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:10:41.622+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:10:41.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex] 2023-10-21T04:10:41.622+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex 2023-10-21T04:10:41.623+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex] 2023-10-21T04:10:41.623+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex] 2023-10-21T04:10:41.623+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex 2023-10-21T04:10:41.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex stopped 2023-10-21T04:10:41.623+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:10:41.623+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:10:41.623+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:10:41.623+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:10:41.626+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:10:41.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex] 2023-10-21T04:10:41.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex closed 2023-10-21T04:10:41.626+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#14910157020177557388:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:10:41.626+05:30 [Info] Indexer::cleanupIndexData IndexInst 4030261110898422634 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:10:41.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex ... 2023-10-21T04:10:41.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex ... 2023-10-21T04:10:41.627+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 33.981554ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:10:41.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:10:41.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/docIndex sucessfully destroyed 2023-10-21T04:10:41.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex ... 2023-10-21T04:10:41.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex ... 2023-10-21T04:10:41.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:10:41.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_4030261110898422634_0.index destroyed 2023-10-21T04:10:41.637+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4030261110898422634, PartitionId 0, IndexDefnId 4860549750398470299. 2023-10-21T04:10:41.637+05:30 [Info] Indexer::cleanupIndexData IndexInst 4030261110898422634 Partition 0 Destroy Done 2023-10-21T04:10:41.643+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.643+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.646+05:30 [Info] switched currmeta from 592 -> 594 force true 2023-10-21T04:10:41.655+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.655+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.658+05:30 [Info] switched currmeta from 594 -> 595 force true 2023-10-21T04:10:41.665+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.665+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.671+05:30 [Info] switched currmeta from 595 -> 595 force true 2023-10-21T04:10:41.687+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.687+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.700+05:30 [Info] switched currmeta from 595 -> 596 force true 2023-10-21T04:10:41.707+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:41.707+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:41.717+05:30 [Info] switched currmeta from 596 -> 596 force true 2023-10-21T04:10:42.469+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:10:45.239+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:10:45.245+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:10:45.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:10:45.248+05:30 [Info] switched currmeta from 596 -> 596 force true 2023-10-21T04:10:59.291+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. err: operation_canceled 2023-10-21T04:10:59.291+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:10:59.291+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:10:59.292+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:10:59.292+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:10:59.292+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:10:59.293+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:11:02.679+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 5465835793930010953 2023-10-21T04:11:02.679+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 348.159µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:02.719+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47474 2023-10-21T04:11:02.722+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47474 2023-10-21T04:11:02.726+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47474 2023-10-21T04:11:02.839+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 5465835793930010953, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:11:02.839+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 5465835793930010953, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:11:02.868+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 16338040286194952893, indexDefn DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:11:02.868+05:30 [Info] Indexer::handleCreateIndex InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:11:02.868+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16338040286194952893 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:11:02.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:11:02.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:11:02.872+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:11:02.872+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:11:02.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:11:02.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:11:02.879+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:11:02.880+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:11:02.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 82 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex 2023-10-21T04:11:02.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:11:02.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:11:02.886+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex 2023-10-21T04:11:02.886+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex 2023-10-21T04:11:02.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex started 2023-10-21T04:11:02.886+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:11:02.886+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:11:02.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 92 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex 2023-10-21T04:11:02.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:11:02.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:11:02.891+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex 2023-10-21T04:11:02.891+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex 2023-10-21T04:11:02.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex started 2023-10-21T04:11:02.892+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16338040286194952893 partitionId 0 WriterThreads 4 2023-10-21T04:11:02.892+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16338040286194952893 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index Index: 16338040286194952893 Partition: 0 2023-10-21T04:11:02.892+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 16338040286194952893 2023-10-21T04:11:02.893+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:02.893+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:11:02.893+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:11:02.893+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:11:02.894+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:02.895+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:02.896+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 28.313862ms 2023-10-21T04:11:02.896+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 28.40424ms 2023-10-21T04:11:02.896+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 16338040286194952893, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], indexDefn DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:11:02.897+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:02.897+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:02.903+05:30 [Info] switched currmeta from 596 -> 599 force true 2023-10-21T04:11:02.909+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:02.909+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:02.920+05:30 [Info] switched currmeta from 599 -> 600 force true 2023-10-21T04:11:02.923+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [16338040286194952893] &{0} 2023-10-21T04:11:02.923+05:30 [Info] Indexer::handleBuildIndex [16338040286194952893] 2023-10-21T04:11:02.924+05:30 [Info] bulkUpdateRState: Index instance 16338040286194952893 rstate moved to ACTIVE 2023-10-21T04:11:02.925+05:30 [Info] Indexer::handleBuildIndex Added Index: [16338040286194952893] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:11:02.926+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:02.927+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:02.927+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:02.928+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:11:02.928+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:02.928+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:02.929+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57460 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:02.929+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:11:02.929+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:11:02.929+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:11:02.930+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57460 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:02.930+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:11:02.930+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:02.930+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:02.931+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:11:02.931+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:11:02.931+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:02.932+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:02.932+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:11:02.932+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:10] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:11:02.933+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 16338040286194952893 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:11:02.933+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:11:02.934+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [ InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 57461 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:11:02.937+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:02.937+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:02.939+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:11:02.939+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:11:02.944+05:30 [Info] switched currmeta from 600 -> 601 force true 2023-10-21T04:11:02.945+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:11:02.944+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:11:02.946+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 22.860957ms 2023-10-21T04:11:02.946+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 22.990737ms 2023-10-21T04:11:02.947+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [16338040286194952893] 2023-10-21T04:11:02.952+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 5465835793930010953 2023-10-21T04:11:02.953+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 113.726723ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:02.962+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:11:02.974+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.957198ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:03.007+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35158" doAuth() ... 2023-10-21T04:11:03.007+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35158" auth successful 2023-10-21T04:11:03.008+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:35158" +1 2023-10-21T04:11:03.029+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:03.029+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:03.034+05:30 [Info] switched currmeta from 601 -> 605 force true 2023-10-21T04:11:03.054+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:03.054+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:03.062+05:30 [Info] switched currmeta from 605 -> 605 force true 2023-10-21T04:11:03.319+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 186517626268899 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.319+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 178840478062605 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.323+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 28605411869539 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.323+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 148075163102199 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.323+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 215374859564337 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.325+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 39552632604610 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.329+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 50961953886801 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.329+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 191903564840858 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.331+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 225280996447309 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.336+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 242327631330054 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.336+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 4429827396609 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.336+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 274410200699189 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.336+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 66881023717063 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.338+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 183148377680315 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.340+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 226814108379280 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.340+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 127185710837163 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.342+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 257031483891250 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.342+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 55290066231400 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 242578104468179 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 210306088158507 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 230239359684242 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.353+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 209771427122473 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.358+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 260036488774725 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.358+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 216462437971699 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 280874278672777 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 160961818008256 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.364+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 219132834127088 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 102578907139247 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 184056281482149 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 45957426777107 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.368+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 99433130332778 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.371+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 82469662608686 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.378+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 217381278987664 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.379+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 70213628416850 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.381+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 136958029391999 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.383+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 253845164956592 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.384+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 28417669987133 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.384+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 253494845579900 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.386+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 188107826286720 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.387+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 5915864305121 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.388+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 222555662236724 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.389+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 252931432434966 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.390+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 146267938201125 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.392+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 158545118983293 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.394+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 242994859770444 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.398+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 259381390638966 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.398+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 271074273345339 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.398+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 12301831552869 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.406+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 227378777358533 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.406+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 215467668600630 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.411+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 207367350302991 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.411+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 200436537417141 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.411+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 188718367400378 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.413+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 204867593700407 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.420+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 264835102027079 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.420+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 279362078165491 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.423+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 129411666769864 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.429+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 170056388266420 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.429+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 16044440367455 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.439+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 75566139929862 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.439+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 125009072873642 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.439+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 106932309485550 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.451+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c1 InstanceIds [16338040286194952893] 2023-10-21T04:11:03.451+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 57461 2023-10-21T04:11:03.451+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:11:03.451+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:11:03.451+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_1:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:11:03.451+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 62 Crc64: 2113421542146274774 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 55 0 55} { 1 a2a78b2a980d 2 54 0 54} { 2 86ac6f7f2bf7 0 0 0 0} { 3 c3e1ddff7531 2 35 0 35} { 4 1a043769c363 2 58 0 58} { 5 23f9109dc7c2 0 0 0 0} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 2 63 0 63} { 8 2e5981289251 2 30 0 45} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 2 32 0 51} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} { 61 61411e5267ee 0 0 0 0} HWT bucket: bucket_1, scopeId: , collectionId: , vbuckets: 62 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 55 0 55} { 1 a2a78b2a980d 2 54 0 54} { 2 86ac6f7f2bf7 0 0 0 0} { 3 c3e1ddff7531 2 35 0 35} { 4 1a043769c363 2 58 0 58} { 5 23f9109dc7c2 0 0 0 0} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 2 63 0 63} { 8 2e5981289251 2 30 0 45} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 2 32 0 51} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} { 61 61411e5267ee 0 0 0 0} 2023-10-21T04:11:03.451+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:11:03.451+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:11:03.451+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:11:03.452+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 10527297481171 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.452+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 133748675726909 0 57461 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:03.453+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:03.462+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:11:03.462+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:11:03.471+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:34418 2023-10-21T04:11:03.471+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:11:03.484+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:34420 2023-10-21T04:11:03.527+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 28.561084ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:03.528+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:11:03.528+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:11:03.528+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:11:03.533+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 32.015983ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:03.597+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 29.698µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:03.624+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:34420. Reason = EOF. Cleaning Pipe. 2023-10-21T04:11:03.624+05:30 [Info] messageListener.start(): message channel closed. Remove peer 1a:9e:e9:4e:9a:71:70:99 as follower. 2023-10-21T04:11:03.807+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:11:03.902+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:11:03.903+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:11:03.903+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:11:03.903+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:11:03.903+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:11:03.903+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:11:03.903+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:34418->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:11:03.903+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:59048->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:11:03.903+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:34418. Reason = EOF. Cleaning Pipe. 2023-10-21T04:11:03.903+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4f:a3:78:a4:93:3e:40:12 as follower. 2023-10-21T04:11:04.030+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:11:04.035+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:11:04.215+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:11:04.218+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16338040286194952893 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:04.218+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16338040286194952893 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.468µs SnapOpenDur 94.675µs 2023-10-21T04:11:04.218+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 16338040286194952893 INDEX_STATE_CATCHUP 2023-10-21T04:11:04.218+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 16338040286194952893 true 2023-10-21T04:11:04.218+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 16338040286194952893 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 57461 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] 2023-10-21T04:11:04.219+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:11:04.221+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 57461 2023-10-21T04:11:04.221+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 57461 2023-10-21T04:11:04.222+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:04.222+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:04.222+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:04.223+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:11:04.223+05:30 [Info] send progress stats to clients 2023-10-21T04:11:04.223+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:11:04.223+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:11:04.223+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:11] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:11:04.223+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 57461 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:11:04.229+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:11:04.235+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.556479ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:04.238+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 2023-10-21T04:11:04.239+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57461 2023-10-21T04:11:04.240+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57461 2023-10-21T04:11:04.240+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:11:04.240+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:11:04.240+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 16338040286194952893 false 2023-10-21T04:11:04.240+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 16338040286194952893 2023-10-21T04:11:04.240+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:11:04.240+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 57461 2023-10-21T04:11:04.238+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 SnapshotId 3 Created recovery point (took 19.622191ms) 2023-10-21T04:11:04.248+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:04.248+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:04.260+05:30 [Info] switched currmeta from 605 -> 606 force true 2023-10-21T04:11:04.279+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:04.280+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:04.282+05:30 [Info] switched currmeta from 606 -> 607 force true 2023-10-21T04:11:04.457+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:11:04.457+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:11:04.457+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 16338040286194952893 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 57461, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 55 0 55} { 1 a2a78b2a980d 2 54 0 54} { 2 86ac6f7f2bf7 2 54 0 54} { 3 c3e1ddff7531 2 35 0 35} { 4 1a043769c363 2 58 0 58} { 5 23f9109dc7c2 2 52 0 52} { 6 dc654c627b06 2 62 0 62} { 7 ae8907d03f9a 2 63 0 63} { 8 2e5981289251 2 45 0 45} { 9 cce451c4944d 2 53 0 53} { 10 f9931a891135 2 54 0 54} { 11 3cd3f3a92ec7 2 51 0 51} { 12 4076649d401 2 42 0 42} { 13 a6928dfe29bb 2 46 0 46} { 14 ce49465c2890 2 72 0 72} { 15 73acbb1d6dab 2 50 0 50} { 16 e9c4ce4b1e32 2 55 0 55} { 17 324938c1e068 2 51 0 51} { 18 dc9f9dbf42d3 2 63 0 63} { 19 bf45b3adc52b 2 48 0 48} { 20 d166c73f4e92 2 50 0 50} { 21 bec93765cd29 2 47 0 47} { 22 ec80768ac445 2 51 0 51} { 23 c4df16b8f6f3 2 57 0 57} { 24 9264d7fc1ac0 2 59 0 59} { 25 ff74239b5989 2 71 0 71} { 26 c74cd6bc9cf0 2 52 0 52} { 27 a765f146c3a5 2 49 0 49} { 28 5d4b83524caf 2 58 0 58} { 29 29cc4c190c13 2 57 0 57} { 30 5a6f1469926a 2 58 0 58} { 31 c5b505ea3590 2 67 0 67} { 32 4b0176f1cd2e 2 60 0 60} { 33 7c9006b0e87f 2 65 0 65} { 34 19d8811ffb3d 2 58 0 58} { 35 3fdbe2615352 2 47 0 47} { 36 e6deeee29bb0 2 58 0 58} { 37 e68d5e39227c 2 54 0 54} { 38 ab15442a5080 2 60 0 60} { 39 ca69c730bc34 2 67 0 67} { 40 56164fd8de1 2 60 0 60} { 41 ebe7efa67f76 2 58 0 58} { 42 e60a302ecd16 2 46 0 46} { 43 8507a87f9e25 2 55 0 55} { 44 9032297f747d 2 67 0 67} { 45 b303e4cbf65 2 63 0 63} { 46 dd00a64c824c 2 54 0 54} { 47 f68a65c53f3b 2 53 0 53} { 48 ceccbf4100c5 2 57 0 57} { 49 c3f779d8f736 2 34 0 34} { 50 bc997940b90f 2 64 0 64} { 51 b64bc49cddb5 2 54 0 54} { 52 aba36b4061ba 2 56 0 56} { 53 ba5374260437 2 56 0 56} { 54 fe140d6ec9f3 2 55 0 55} { 55 9aaa569d49b4 2 70 0 70} { 56 f0ddba2e2d47 2 52 0 52} { 57 75b3006d77c8 2 55 0 55} { 58 e97a32b315f 2 53 0 53} { 59 71b1f161e0aa 2 53 0 53} { 60 44ba1ced7506 2 56 0 56} { 61 61411e5267ee 2 53 0 53} { 62 99313d3a9d3 2 57 0 57} { 63 79a4ca4e4e3d 2 56 0 56} 2023-10-21T04:11:04.457+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 16338040286194952893 INDEX_STATE_ACTIVE 2023-10-21T04:11:04.457+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:11:04.457+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:11:04.458+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:11:04.458+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 0 New 0 2023-10-21T04:11:04.458+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 57461 2023-10-21T04:11:04.459+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:04.460+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:04.461+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:11:04.461+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57461 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:04.462+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 16338040286194952893 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:11:04.462+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:11:04.462+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:11:04.462+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:11:04.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:04.462+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:04.462+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57461 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:04.462+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:11:04.462+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:18 1:14 2:15 3:7 4:19 5:8 6:18 7:18 8:9 9:11 10:17 11:12 12:14 13:15 14:21 15:12 16:10 17:12 18:18 19:8 20:15 21:8 22:14 23:13 24:11 25:23 26:11 27:16 28:17 29:18 30:14 31:19 32:17 33:17 34:17 35:15 36:14 37:15 38:16 39:25 40:22 41:17 42:11 43:13 44:33 45:16 46:17 47:14 48:17 49:14 50:22 51:23 52:17 53:13 54:23 55:22 56:10 57:13 58:14 59:13 60:16 61:14 62:19 63:16] 2023-10-21T04:11:04.462+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:36 1:34 2:36 3:21 4:38 5:38 6:46 7:46 8:34 9:31 10:42 11:34 12:34 13:31 14:48 15:36 16:35 17:34 18:45 19:29 20:35 21:32 22:32 23:34 24:36 25:50 26:29 27:35 28:36 29:38 30:44 31:44 32:41 33:43 34:41 35:37 36:37 37:41 38:38 39:46 40:43 41:38 42:34 43:34 44:55 45:41 46:42 47:37 48:42 49:28 50:47 51:43 52:40 53:38 54:45 55:46 56:39 57:38 58:33 59:41 60:41 61:36 62:41 63:40] 2023-10-21T04:11:04.463+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:11:04.463+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:35158: use of closed network connection 2023-10-21T04:11:04.463+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:35158" exit: read tcp4 127.0.0.1:9121->127.0.0.1:35158: use of closed network connection 2023-10-21T04:11:04.463+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35158" closed ! 2023-10-21T04:11:04.463+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:11:04.463+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:04.463+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:04.463+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:04.463+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:04.463+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:11:04.463+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:11:04.464+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 57461 2023-10-21T04:11:04.467+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:11:04.468+05:30 [Info] switched currmeta from 607 -> 608 force true 2023-10-21T04:11:04.480+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.911222ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:04.487+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:04.488+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:04.493+05:30 [Info] switched currmeta from 608 -> 609 force true 2023-10-21T04:11:04.527+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:11:04.527+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:11:04.527+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:11:04.527+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57461 2023-10-21T04:11:04.527+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57461 2023-10-21T04:11:04.527+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16338040286194952893:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13420534524606213282:[0]]. New rollbacktime: 1697841557509288105, Old rollbacktime: 0 2023-10-21T04:11:05.348+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:05.354+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:05.354+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:05.356+05:30 [Info] switched currmeta from 609 -> 609 force true 2023-10-21T04:11:05.452+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 8 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.45µs SnapOpenDur 133.775µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.553µs SnapOpenDur 46.572µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 2 Creating New Snapshot SnapshotInfo: count:113 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:113 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.664µs SnapOpenDur 46.534µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:132 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.368µs SnapOpenDur 43.11µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 4 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:124 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.185µs SnapOpenDur 41.389µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 5 Creating New Snapshot SnapshotInfo: count:111 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:111 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.96µs SnapOpenDur 41.06µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 6 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:140 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.475µs SnapOpenDur 54.803µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:125 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.474µs SnapOpenDur 45.622µs 2023-10-21T04:11:05.977+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3810060733504350542 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:05.977+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3810060733504350542 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.763µs SnapOpenDur 83.114µs 2023-10-21T04:11:05.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3292093078969249751 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3292093078969249751 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.391µs SnapOpenDur 63.631µs 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 2 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:120 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.649µs SnapOpenDur 60.286µs 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 3 Creating New Snapshot SnapshotInfo: count:139 committed:true 2023-10-21T04:11:05.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:139 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.504µs SnapOpenDur 51.404µs 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 4 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:11:05.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:112 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.437µs SnapOpenDur 48.304µs 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 5 Creating New Snapshot SnapshotInfo: count:133 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:133 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.223µs SnapOpenDur 42.351µs 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 6 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.565µs SnapOpenDur 42.609µs 2023-10-21T04:11:05.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 7 Creating New Snapshot SnapshotInfo: count:119 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:119 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.328µs SnapOpenDur 40.986µs 2023-10-21T04:11:05.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 8 Creating New Snapshot SnapshotInfo: count:122 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:122 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.13µs SnapOpenDur 41.092µs 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.3µs SnapOpenDur 55.303µs 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16338040286194952893 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:05.978+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16338040286194952893 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.87µs SnapOpenDur 62.181µs 2023-10-21T04:11:05.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.978+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17148287981272169168 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11160011617777336020 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11160011617777336020 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.259µs SnapOpenDur 64.675µs 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13258659765240018889 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13258659765240018889 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:21 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.603µs SnapOpenDur 53.863µs 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 8 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:128 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.039µs SnapOpenDur 62.771µs 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 1 Creating New Snapshot SnapshotInfo: count:135 committed:true 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:135 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.088µs SnapOpenDur 38.332µs 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 2 Creating New Snapshot SnapshotInfo: count:107 committed:true 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:107 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.328µs SnapOpenDur 43.848µs 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 3 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:138 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.354µs SnapOpenDur 45.604µs 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 4 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:137 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.468µs SnapOpenDur 46.507µs 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 5 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:11:05.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:131 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.222µs SnapOpenDur 36.502µs 2023-10-21T04:11:05.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 6 Creating New Snapshot SnapshotInfo: count:115 committed:true 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:115 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.979µs SnapOpenDur 35.053µs 2023-10-21T04:11:05.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17148287981272169168 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.222µs SnapOpenDur 58.485µs 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 7 Creating New Snapshot SnapshotInfo: count:109 committed:true 2023-10-21T04:11:05.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:109 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.969µs SnapOpenDur 83.488µs 2023-10-21T04:11:05.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4004220452657849790 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:05.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4004220452657849790 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.696µs SnapOpenDur 55.822µs 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.981+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:05.981+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:06.029+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 SnapshotId 2 Created recovery point (took 50.996412ms) 2023-10-21T04:11:06.030+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:11:06.031+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:11:06.067+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 SnapshotId 2 Created recovery point (took 88.427954ms) 2023-10-21T04:11:06.068+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:06.086+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 SnapshotId 2 Created recovery point (took 107.230513ms) 2023-10-21T04:11:06.087+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:06.126+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 SnapshotId 2 Created recovery point (took 147.784149ms) 2023-10-21T04:11:06.128+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:11:06.128+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:11:06.287+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 SnapshotId 2 Created recovery point (took 307.896863ms) 2023-10-21T04:11:06.288+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:11:06.289+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:11:06.337+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 SnapshotId 2 Created recovery point (took 357.695866ms) 2023-10-21T04:11:06.339+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:11:06.341+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:11:06.402+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 SnapshotId 2 Created recovery point (took 421.930283ms) 2023-10-21T04:11:06.404+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:06.435+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 SnapshotId 2 Created recovery point (took 455.386507ms) 2023-10-21T04:11:06.436+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:11:06.437+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:11:06.460+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 SnapshotId 2 Created recovery point (took 479.745019ms) 2023-10-21T04:11:06.461+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:14 committed:false. num RPs 3. 2023-10-21T04:11:06.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 SnapshotId 2 Created recovery point (took 597.203629ms) 2023-10-21T04:11:06.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:11:06.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:11:06.605+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 SnapshotId 2 Created recovery point (took 627.318315ms) 2023-10-21T04:11:06.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:11:06.607+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:11:06.641+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 SnapshotId 2 Created recovery point (took 662.865098ms) 2023-10-21T04:11:06.642+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:11:06.643+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:11:06.674+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 SnapshotId 2 Created recovery point (took 695.98863ms) 2023-10-21T04:11:06.676+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:11:06.676+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:11:06.689+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 SnapshotId 2 Created recovery point (took 710.575499ms) 2023-10-21T04:11:06.690+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:06.691+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:06.705+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 SnapshotId 2 Created recovery point (took 726.460533ms) 2023-10-21T04:11:06.706+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:11:06.707+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:11:06.732+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 SnapshotId 2 Created recovery point (took 752.842178ms) 2023-10-21T04:11:06.734+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:11:06.735+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:11:06.749+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 SnapshotId 2 Created recovery point (took 770.14889ms) 2023-10-21T04:11:06.751+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:11:06.751+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:11:06.768+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 SnapshotId 2 Created recovery point (took 788.907898ms) 2023-10-21T04:11:06.769+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:11:06.770+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:11:06.795+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 SnapshotId 2 Created recovery point (took 815.982274ms) 2023-10-21T04:11:06.796+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:11:06.797+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:11:06.818+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 SnapshotId 2 Created recovery point (took 838.661735ms) 2023-10-21T04:11:06.819+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:11:06.820+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:11:06.839+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 SnapshotId 2 Created recovery point (took 859.471879ms) 2023-10-21T04:11:06.841+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:11:06.841+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:11:06.851+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 SnapshotId 4 Created recovery point (took 871.503573ms) 2023-10-21T04:11:06.861+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 SnapshotId 2 Created recovery point (took 881.504744ms) 2023-10-21T04:11:06.862+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:11:06.863+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:11:06.872+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 SnapshotId 2 Created recovery point (took 891.957277ms) 2023-10-21T04:11:06.873+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:11:06.874+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:11:06.893+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 SnapshotId 2 Created recovery point (took 913.032822ms) 2023-10-21T04:11:06.894+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:11:06.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:11:06.928+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 SnapshotId 2 Created recovery point (took 947.354108ms) 2023-10-21T04:11:06.929+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:06.929+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:06.957+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 SnapshotId 2 Created recovery point (took 976.476393ms) 2023-10-21T04:11:06.958+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:11:06.959+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:11:06.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 SnapshotId 2 Created recovery point (took 997.671088ms) 2023-10-21T04:11:06.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:11:06.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:11:06.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 SnapshotId 2 Created recovery point (took 1.011632539s) 2023-10-21T04:11:06.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:11:06.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:11:06.999+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 SnapshotId 2 Created recovery point (took 1.018376659s) 2023-10-21T04:11:07.000+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:11:07.001+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:11:07.018+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 SnapshotId 2 Created recovery point (took 1.037997147s) 2023-10-21T04:11:07.019+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:11:07.019+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:11:10.818+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 12195715103909237684 2023-10-21T04:11:10.820+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 1.581273ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:10.864+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47474 2023-10-21T04:11:10.868+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47474 2023-10-21T04:11:10.871+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47474 2023-10-21T04:11:11.018+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12195715103909237684, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:11:11.019+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12195715103909237684, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:11:11.041+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 18361169415695152662, indexDefn DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:11:11.042+05:30 [Info] Indexer::handleCreateIndex InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:11:11.043+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 18361169415695152662 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:11:11.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:11:11.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:11:11.057+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:11:11.058+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:11:11.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:11:11.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:11:11.066+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:11:11.066+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:11:11.069+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:11.070+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:11.075+05:30 [Info] switched currmeta from 609 -> 612 force true 2023-10-21T04:11:11.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 92 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex 2023-10-21T04:11:11.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:11:11.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:11:11.078+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex 2023-10-21T04:11:11.078+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex 2023-10-21T04:11:11.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex started 2023-10-21T04:11:11.078+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:11:11.078+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:11:11.083+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:11.083+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:11.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 82 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex 2023-10-21T04:11:11.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:11:11.091+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:11:11.091+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex 2023-10-21T04:11:11.092+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex 2023-10-21T04:11:11.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex started 2023-10-21T04:11:11.092+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:11:11.092+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 18361169415695152662 partitionId 0 WriterThreads 4 2023-10-21T04:11:11.092+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 18361169415695152662 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index Index: 18361169415695152662 Partition: 0 2023-10-21T04:11:11.093+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 18361169415695152662 2023-10-21T04:11:11.093+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:11.093+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:11:11.093+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:11.094+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:11:11.094+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:11.094+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 51.949716ms 2023-10-21T04:11:11.094+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 51.991996ms 2023-10-21T04:11:11.094+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 18361169415695152662, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], indexDefn DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:11:11.101+05:30 [Info] switched currmeta from 612 -> 613 force true 2023-10-21T04:11:11.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:11.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:11.123+05:30 [Info] switched currmeta from 613 -> 615 force true 2023-10-21T04:11:11.126+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [18361169415695152662] &{0} 2023-10-21T04:11:11.127+05:30 [Info] Indexer::handleBuildIndex [18361169415695152662] 2023-10-21T04:11:11.128+05:30 [Info] bulkUpdateRState: Index instance 18361169415695152662 rstate moved to ACTIVE 2023-10-21T04:11:11.128+05:30 [Info] Indexer::handleBuildIndex Added Index: [18361169415695152662] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:11:11.129+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:11.131+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:11.132+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:11.133+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:11:11.134+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:11.135+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:11.136+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11732 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:11.136+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:11:11.137+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:11:11.138+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:11:11.139+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11732 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:11.142+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:11:11.144+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:11.145+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:11.145+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:11:11.145+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:11:11.146+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:11.146+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:11.146+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:11:11.147+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:11] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:11:11.147+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 18361169415695152662 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:11:11.147+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:11:11.147+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [ InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 11733 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:11:11.148+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:11:11.148+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:11:11.153+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:11:11.157+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 30.429189ms 2023-10-21T04:11:11.157+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:11:11.157+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [18361169415695152662] 2023-10-21T04:11:11.157+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 30.741264ms 2023-10-21T04:11:11.160+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:11:11.164+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 12195715103909237684 2023-10-21T04:11:11.164+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 145.694694ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:11.173+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.045662ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:11.200+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:11.200+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:11.203+05:30 [Info] switched currmeta from 615 -> 618 force true 2023-10-21T04:11:11.211+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:11.212+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:11.215+05:30 [Info] switched currmeta from 618 -> 618 force true 2023-10-21T04:11:11.227+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35334" doAuth() ... 2023-10-21T04:11:11.227+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35334" auth successful 2023-10-21T04:11:11.228+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:35334" +1 2023-10-21T04:11:11.474+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:34590 2023-10-21T04:11:11.512+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 263268055970459 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.512+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 176639880977228 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.512+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 35403626585217 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.515+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 215749515705641 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.519+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 159113185399719 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.519+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 3874519919278 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 170288557775498 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.523+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 236398693961229 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.524+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 264975169139761 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.526+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:11:11.526+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:11:11.526+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 234758907181738 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.528+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 44.708942ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:11.529+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 102350060311343 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.529+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 116104661063478 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.529+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 206533515052028 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.532+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 140236896721046 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.534+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 200752780087280 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.534+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 162158720181435 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.536+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 261493429196201 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.537+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 98820809009123 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.537+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 207134287529256 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.540+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:34596 2023-10-21T04:11:11.540+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 208836052671833 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.540+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 92956357968546 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.541+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:11:11.545+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 192215088742243 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.545+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 27979933591532 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.546+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 271667630317209 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.546+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 154934968850942 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.550+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 168740244208559 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 120989102157727 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.555+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 130231725986856 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.555+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 60545336618717 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.555+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 236953356346174 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.556+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5715487545729 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.562+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 67362620867705 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.562+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 73432771436458 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.563+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 278426329668882 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 24368176313355 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 51776324008509 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.565+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 10.853103ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:11.565+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:11:11.565+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:11:11.565+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:11:11.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 153478150002150 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 83647976044922 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 43594649874772 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 181267443431019 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 100659545510465 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.575+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 45479338375128 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.575+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 116785907755771 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.579+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:11:11.580+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 166675095997741 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.580+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 249919763471169 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.580+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 49796766757346 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.580+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 124194574848641 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.582+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 149022976193937 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.587+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 30.189µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:11.589+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 199728167534433 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.589+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 19274425379955 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.589+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 184487117398647 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.602+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:34590. Error = read tcp4 127.0.0.1:9118->127.0.0.1:34590: read: connection reset by peer. Kill Pipe. 2023-10-21T04:11:11.603+05:30 [Info] messageListener.start(): message channel closed. Remove peer 72:e5:85:a3:5f:39:87:1d as follower. 2023-10-21T04:11:11.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 158076890855634 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.613+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 138935694816848 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.623+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 196777743741606 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.631+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c1 InstanceIds [18361169415695152662] 2023-10-21T04:11:11.631+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 11733 2023-10-21T04:11:11.625+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 29071415952852 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 24856128127173 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 61798873988101 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 56626109228904 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.634+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 191719959781498 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 103336286678362 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 140942236111856 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 119871837460216 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.653+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:11:11.654+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:11:11.654+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:11:11.654+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:11:11.654+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:11.655+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:11:11.655+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 177105814270799 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.655+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 252537041863196 0 11733 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:11:11.702+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:11:11.702+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:11:11.702+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:11:11.702+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:11:11.702+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:11:11.703+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:34596->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:11:11.703+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:34596. Reason = EOF. Cleaning Pipe. 2023-10-21T04:11:11.703+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2c:26:f3:aa:ae:d7:9e:c6 as follower. 2023-10-21T04:11:11.709+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:11:11.709+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:59204->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:11:12.032+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:11:12.067+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:11:12.307+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:11:12.312+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 18361169415695152662 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:12.312+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 18361169415695152662 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.990213ms SnapOpenDur 120.873µs 2023-10-21T04:11:12.312+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 18361169415695152662 INDEX_STATE_CATCHUP 2023-10-21T04:11:12.312+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 18361169415695152662 true 2023-10-21T04:11:12.312+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 18361169415695152662 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 11733 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] 2023-10-21T04:11:12.312+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 SnapshotId 1 Creating recovery point ... 2023-10-21T04:11:12.312+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 11733 2023-10-21T04:11:12.312+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 11733 2023-10-21T04:11:12.313+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:12.314+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:12.314+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:12.315+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:11:12.315+05:30 [Info] send progress stats to clients 2023-10-21T04:11:12.316+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:11:12.316+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:11:12.316+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:11] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:11:12.316+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 11733 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:11:12.332+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:11:12.358+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 40.112035ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:12.374+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:12.374+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:12.376+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 SnapshotId 1 Created recovery point (took 63.487522ms) 2023-10-21T04:11:12.377+05:30 [Info] switched currmeta from 618 -> 620 force true 2023-10-21T04:11:12.388+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:12.388+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:12.388+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 2023-10-21T04:11:12.388+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11733 2023-10-21T04:11:12.388+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11733 2023-10-21T04:11:12.388+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:11:12.388+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:11:12.388+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 18361169415695152662 false 2023-10-21T04:11:12.388+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 18361169415695152662 2023-10-21T04:11:12.388+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:11:12.388+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 11733 2023-10-21T04:11:12.392+05:30 [Info] switched currmeta from 620 -> 620 force true 2023-10-21T04:11:12.466+05:30 [Info] janitor: running cleanup. 2023-10-21T04:11:12.466+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[14889401051926000176 4860549750398470299]} 2023-10-21T04:11:12.472+05:30 [Info] memstats {"Alloc":206156016, "TotalAlloc":1602834568, "Sys":323052872, "Lookups":0, "Mallocs":15219264,"Frees":14294147, "HeapAlloc":206156016, "HeapSys":291700736, "HeapIdle":67395584, "HeapInuse":224305152,"HeapReleased":1794048, "HeapObjects":925117,"MSpanInuse":1658080, "MSpanSys": 2513280, "StackInuse": 10289152,"GCSys":15552480, "LastGC":1697841611909602208,"PauseTotalNs":40947677, "PauseNs":[1493923, 388373, 2666881, 20386290, 10974280, 110066, 187655, 305972, 443159, 405492, 1420769, 569229], "NumGC":17} 2023-10-21T04:11:12.472+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:11:12.655+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:11:12.655+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:11:12.655+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 18361169415695152662 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 11733, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 52 0 52} { 1 a0a72d5bdb4c 2 58 0 58} { 2 20330c961c81 2 50 0 50} { 3 c439193e7d29 2 45 0 45} { 4 90b66ce513a7 2 58 0 58} { 5 3861b622aae 2 54 0 54} { 6 edd3aef555a9 2 60 0 60} { 7 9ae064feda8a 2 52 0 52} { 8 f0fe56d4b431 2 56 0 56} { 9 d700dc2dce0d 2 62 0 62} { 10 bbd754cc83fc 2 50 0 50} { 11 d58311453aaa 2 56 0 56} { 12 6998b9145736 2 55 0 55} { 13 5d163afcd32f 2 53 0 53} { 14 7f8b7269d896 2 60 0 60} { 15 937b84e9e4bb 2 67 0 67} { 16 b69566250ff0 2 51 0 51} { 17 59e08337a7e3 2 44 0 44} { 18 bc6335a10528 2 46 0 46} { 19 bdef6ebc5d59 2 45 0 45} { 20 548b16b242a2 2 50 0 50} { 21 19729600a7ec 2 54 0 54} { 22 aed19015bf63 2 55 0 55} { 23 8ce99bc819fe 2 51 0 51} { 24 f7148c9a1e99 2 52 0 52} { 25 9977e650bbaf 2 59 0 59} { 26 6e09f874af9f 2 59 0 59} { 27 532bd9ad981 2 54 0 54} { 28 d78200a2233e 2 54 0 54} { 29 7671efc41428 2 49 0 49} { 30 3710cf482edd 2 46 0 46} { 31 42c966419faa 2 60 0 60} { 32 3d4415168479 2 67 0 67} { 33 fd3a2e79dd12 2 54 0 54} { 34 2f171d667a3d 2 60 0 60} { 35 1629a87e340b 2 50 0 50} { 36 27a62b9eb154 2 52 0 52} { 37 4c13cfe5f97a 2 59 0 59} { 38 a4dc9d93366b 2 53 0 53} { 39 5b8ca0741241 2 49 0 49} { 40 8b966a9c95e6 2 68 0 68} { 41 6a37568beafb 2 55 0 55} { 42 295cfbcef7d8 2 55 0 55} { 43 e34cfab95b41 2 64 0 64} { 44 979711e3cd2d 2 57 0 57} { 45 2d4a3697f1e2 2 58 0 58} { 46 70f44d845281 2 54 0 54} { 47 87891d8f3591 2 55 0 55} { 48 1187ad472c73 2 55 0 55} { 49 a7ca41192277 2 55 0 55} { 50 b5a6d67a3f61 2 66 0 66} { 51 b2f7e383f2a6 2 51 0 51} { 52 8fc524ed38d2 2 54 0 54} { 53 7e5c7cbc7650 2 61 0 61} { 54 169b44b000c5 2 60 0 60} { 55 3834abee7c05 2 67 0 67} { 56 1a70b76bc5d4 2 61 0 61} { 57 ae5e4819607a 2 63 0 63} { 58 5dfbdaa8ed5a 2 46 0 46} { 59 802fabead3f0 2 63 0 63} { 60 33804b202768 2 46 0 46} { 61 6d05d64a56f8 2 39 0 39} { 62 a113a925b34f 2 58 0 58} { 63 e5ae5cac421c 2 73 0 73} 2023-10-21T04:11:12.655+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 18361169415695152662 INDEX_STATE_ACTIVE 2023-10-21T04:11:12.656+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:11:12.656+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:11:12.656+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:11:12.656+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 3 CurrMaint 0 New 0 2023-10-21T04:11:12.656+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 11733 2023-10-21T04:11:12.656+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:12.657+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:11:12.657+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:11:12.657+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11733 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:12.657+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 18361169415695152662 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:11:12.657+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:11:12.657+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:11:12.657+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:11:12.657+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11733 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:11:12.657+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:11:12.658+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:17 1:18 2:15 3:14 4:21 5:14 6:21 7:13 8:16 9:14 10:14 11:13 12:12 13:14 14:10 15:20 16:18 17:14 18:12 19:13 20:12 21:16 22:13 23:17 24:16 25:10 26:15 27:17 28:20 29:16 30:11 31:17 32:20 33:12 34:18 35:20 36:11 37:12 38:12 39:9 40:17 41:15 42:19 43:14 44:15 45:15 46:16 47:17 48:15 49:14 50:18 51:15 52:17 53:24 54:16 55:18 56:14 57:16 58:11 59:19 60:17 61:14 62:19 63:28] 2023-10-21T04:11:12.658+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:37 1:38 2:35 3:33 4:41 5:34 6:47 7:33 8:42 9:41 10:34 11:37 12:35 13:37 14:38 15:47 16:39 17:27 18:34 19:32 20:33 21:36 22:37 23:40 24:33 25:33 26:38 27:47 28:39 29:36 30:32 31:40 32:47 33:38 34:45 35:39 36:33 37:33 38:39 39:29 40:45 41:41 42:46 43:41 44:34 45:34 46:42 47:38 48:38 49:42 50:44 51:36 52:37 53:50 54:41 55:47 56:36 57:41 58:32 59:46 60:37 61:30 62:43 63:50] 2023-10-21T04:11:12.658+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:11:12.658+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:35334: use of closed network connection 2023-10-21T04:11:12.658+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:35334" exit: read tcp4 127.0.0.1:9121->127.0.0.1:35334: use of closed network connection 2023-10-21T04:11:12.658+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35334" closed ! 2023-10-21T04:11:12.658+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:11:12.658+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:35334" already gone 2023-10-21T04:11:12.658+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:12.658+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:12.658+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:11:12.658+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:11:12.658+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:11:12.658+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:11:12.659+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 11733 2023-10-21T04:11:12.660+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:11:12.679+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.06859ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:12.692+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:11:12.692+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:11:12.692+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:11:12.692+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11733 2023-10-21T04:11:12.692+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11733 2023-10-21T04:11:12.692+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:11:12.703+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:12.703+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:12.710+05:30 [Info] switched currmeta from 620 -> 621 force true 2023-10-21T04:11:12.767+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:12.767+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:12.780+05:30 [Info] switched currmeta from 621 -> 622 force true 2023-10-21T04:11:13.090+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18361169415695152662:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:11:13.090+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18361169415695152662:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:11:13.217+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14736339311583908837:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:11:13.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14736339311583908837:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:11:13.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14736339311583908837:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:11:13.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14736339311583908837:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:11:13.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14736339311583908837:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:11:13.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14736339311583908837:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:11:13.218+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14736339311583908837:[0]]. New rollbacktime: 1697841557322373572, Old rollbacktime: 0 2023-10-21T04:11:13.241+05:30 [Info] connected with 2 indexers 2023-10-21T04:11:13.241+05:30 [Info] client stats current counts: current: 76, not current: 0 2023-10-21T04:11:13.309+05:30 [Info] num concurrent scans {0} 2023-10-21T04:11:13.309+05:30 [Info] average scan response {0 ms} 2023-10-21T04:11:13.391+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:11:13.391+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:11:13.654+05:30 [Info] Indexer::ReadMemstats Time Taken 195.135µs 2023-10-21T04:11:13.655+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:11:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17729593977321451597 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 1 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:11:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17729593977321451597 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.504µs SnapOpenDur 144.203µs 2023-10-21T04:11:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.221µs SnapOpenDur 58.423µs 2023-10-21T04:11:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 2 Creating New Snapshot SnapshotInfo: count:110 committed:true 2023-10-21T04:11:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:110 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.077µs SnapOpenDur 39.588µs 2023-10-21T04:11:14.252+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 3 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:11:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.302µs SnapOpenDur 36.647µs 2023-10-21T04:11:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 4 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:11:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:140 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.365µs SnapOpenDur 41.537µs 2023-10-21T04:11:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 5 Creating New Snapshot SnapshotInfo: count:116 committed:true 2023-10-21T04:11:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:116 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.257µs SnapOpenDur 42.231µs 2023-10-21T04:11:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:11:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:127 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.451µs SnapOpenDur 43.301µs 2023-10-21T04:11:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:11:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:125 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.478µs SnapOpenDur 45.013µs 2023-10-21T04:11:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 8 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:11:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:124 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.393µs SnapOpenDur 55.093µs 2023-10-21T04:11:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2483169159760864439 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2483169159760864439 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.586µs SnapOpenDur 61.737µs 2023-10-21T04:11:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17128008689111072444 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14607307599259773249 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11400459767051699632 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:14.253+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 18361169415695152662 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14780363202971367008 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 4 Creating New Snapshot SnapshotInfo: count:130 committed:true 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.252+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.259+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17128008689111072444 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.259µs SnapOpenDur 6.288642ms 2023-10-21T04:11:14.260+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14607307599259773249 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.981µs SnapOpenDur 7.68084ms 2023-10-21T04:11:14.260+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11400459767051699632 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.784µs SnapOpenDur 6.30904ms 2023-10-21T04:11:14.261+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.712µs SnapOpenDur 7.593749ms 2023-10-21T04:11:14.261+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 4 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:11:14.261+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:138 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.66µs SnapOpenDur 51µs 2023-10-21T04:11:14.261+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 5 Creating New Snapshot SnapshotInfo: count:117 committed:true 2023-10-21T04:11:14.261+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:117 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.766µs SnapOpenDur 50.456µs 2023-10-21T04:11:14.261+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 6 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:11:14.261+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:137 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.298µs SnapOpenDur 45.454µs 2023-10-21T04:11:14.261+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 7 Creating New Snapshot SnapshotInfo: count:102 committed:true 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 18361169415695152662 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.636µs SnapOpenDur 8.134246ms 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14780363202971367008 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:21 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.383µs SnapOpenDur 8.225195ms 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:130 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.439µs SnapOpenDur 8.327439ms 2023-10-21T04:11:14.262+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 5 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:128 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.526µs SnapOpenDur 44.919µs 2023-10-21T04:11:14.262+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 6 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:106 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.64µs SnapOpenDur 30.161µs 2023-10-21T04:11:14.262+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 7 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.625µs SnapOpenDur 42.881µs 2023-10-21T04:11:14.262+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 8 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.925µs SnapOpenDur 48.116µs 2023-10-21T04:11:14.262+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 1 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:120 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.433µs SnapOpenDur 41.069µs 2023-10-21T04:11:14.262+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 2 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:11:14.263+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.745µs SnapOpenDur 39.419µs 2023-10-21T04:11:14.263+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:11:14.263+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.529µs SnapOpenDur 44.929µs 2023-10-21T04:11:14.263+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.263+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.263+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.263+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 SnapshotId 2 Created recovery point (took 11.105778ms) 2023-10-21T04:11:14.264+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.264+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.264+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.264+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.264+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.264+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.264+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.265+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:14.265+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.262+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:102 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.875µs SnapOpenDur 45.046µs 2023-10-21T04:11:14.266+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 8 Creating New Snapshot SnapshotInfo: count:148 committed:true 2023-10-21T04:11:14.266+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:148 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.52µs SnapOpenDur 61.357µs 2023-10-21T04:11:14.266+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 1 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:11:14.266+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:112 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.246µs SnapOpenDur 44.51µs 2023-10-21T04:11:14.266+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.266+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.266+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:11:14.266+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.524µs SnapOpenDur 138.506µs 2023-10-21T04:11:14.267+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:11:14.304+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 SnapshotId 2 Created recovery point (took 42.723745ms) 2023-10-21T04:11:14.305+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:14.356+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 SnapshotId 2 Created recovery point (took 94.548858ms) 2023-10-21T04:11:14.358+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:12 committed:false. num RPs 3. 2023-10-21T04:11:14.497+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 SnapshotId 2 Created recovery point (took 233.997603ms) 2023-10-21T04:11:14.499+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:14.743+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 SnapshotId 2 Created recovery point (took 479.705469ms) 2023-10-21T04:11:14.744+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:11:14.745+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:11:14.759+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 SnapshotId 2 Created recovery point (took 498.566576ms) 2023-10-21T04:11:14.760+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:14.760+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:14.797+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 SnapshotId 2 Created recovery point (took 535.01342ms) 2023-10-21T04:11:14.798+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:11:14.798+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:11:14.836+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 SnapshotId 2 Created recovery point (took 574.509339ms) 2023-10-21T04:11:14.861+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 SnapshotId 2 Created recovery point (took 598.497275ms) 2023-10-21T04:11:14.862+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:11:14.863+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:11:14.883+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 SnapshotId 2 Created recovery point (took 619.951968ms) 2023-10-21T04:11:14.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 4. 2023-10-21T04:11:14.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:11:14.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 4. 2023-10-21T04:11:14.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:11:14.897+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 SnapshotId 2 Created recovery point (took 633.919106ms) 2023-10-21T04:11:14.898+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:11:14.898+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:11:14.911+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 SnapshotId 2 Created recovery point (took 648.497415ms) 2023-10-21T04:11:14.912+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:11:14.913+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:11:14.931+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 SnapshotId 2 Created recovery point (took 668.119203ms) 2023-10-21T04:11:14.932+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:11:14.933+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:11:14.964+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 SnapshotId 2 Created recovery point (took 701.181418ms) 2023-10-21T04:11:14.965+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:11:14.966+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:11:14.979+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 SnapshotId 2 Created recovery point (took 715.496699ms) 2023-10-21T04:11:14.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:11:14.980+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:11:14.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 SnapshotId 2 Created recovery point (took 730.343097ms) 2023-10-21T04:11:14.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:11:14.996+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:11:15.003+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 SnapshotId 2 Created recovery point (took 739.252112ms) 2023-10-21T04:11:15.004+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:11:15.004+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:11:15.016+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 SnapshotId 2 Created recovery point (took 752.377732ms) 2023-10-21T04:11:15.017+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:11:15.017+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:11:15.034+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 SnapshotId 2 Created recovery point (took 769.912176ms) 2023-10-21T04:11:15.034+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:11:15.035+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:11:15.053+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 SnapshotId 2 Created recovery point (took 789.344094ms) 2023-10-21T04:11:15.054+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:15.055+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:11:15.079+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 SnapshotId 2 Created recovery point (took 815.46743ms) 2023-10-21T04:11:15.080+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:11:15.083+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:11:15.094+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 SnapshotId 2 Created recovery point (took 829.903337ms) 2023-10-21T04:11:15.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:11:15.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:11:15.112+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 SnapshotId 2 Created recovery point (took 847.654281ms) 2023-10-21T04:11:15.112+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:11:15.113+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:11:15.124+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 SnapshotId 2 Created recovery point (took 860.282652ms) 2023-10-21T04:11:15.125+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:11:15.126+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:11:15.139+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 SnapshotId 2 Created recovery point (took 874.430338ms) 2023-10-21T04:11:15.139+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:11:15.140+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:11:15.160+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 SnapshotId 2 Created recovery point (took 895.558671ms) 2023-10-21T04:11:15.161+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:11:15.161+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:11:15.191+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 SnapshotId 2 Created recovery point (took 926.811687ms) 2023-10-21T04:11:15.192+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:11:15.193+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:11:15.212+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 SnapshotId 2 Created recovery point (took 947.132352ms) 2023-10-21T04:11:15.213+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:11:15.213+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:11:15.232+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 SnapshotId 2 Created recovery point (took 965.721211ms) 2023-10-21T04:11:15.233+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:11:15.234+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:11:15.243+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 SnapshotId 2 Created recovery point (took 976.343548ms) 2023-10-21T04:11:15.244+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:11:15.245+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:11:15.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 SnapshotId 2 Created recovery point (took 988.682469ms) 2023-10-21T04:11:15.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:11:15.257+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:11:15.401+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:15.405+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:15.405+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:15.407+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:27.598+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:11:27.701+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 1 2023-10-21T04:11:27.702+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:27.702+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:27.705+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:27.832+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:27.838+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:27.838+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:27.845+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:28.446+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:11:28.493+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:28.493+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:28.497+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:28.503+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:28.513+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:28.514+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:28.523+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:29.293+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. err: operation_canceled 2023-10-21T04:11:29.293+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:11:29.293+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:11:29.294+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:11:29.294+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:11:29.294+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[31fd475f22d8ee0da906c31968c91305 85232c9ba3e4451b95e9e6fc8337e027] IsBalanced:true Messages:[]} 2023-10-21T04:11:29.295+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:11:37.506+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:11:37.555+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 182] updated cluster rebalancing status to true 2023-10-21T04:11:37.556+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 182] updating rebalancing status to true 2023-10-21T04:11:37.563+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:37.564+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:37.585+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:37.638+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:37.643+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:37.656+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:37.656+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:37.659+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:37.668+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:37.668+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:37.670+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:37.939+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:11:37.996+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:11:37.996+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:37.996+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:37.998+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:38.150+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:38.165+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:38.165+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:38.176+05:30 [Info] switched currmeta from 622 -> 622 force true 2023-10-21T04:11:39.619+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 238] /regulator/report/s/index/281b41507b1f937a1b6c37bc4027eef6 has 0-length value 2023-10-21T04:11:39.654+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 238] /regulator/report/s/index/5020693e9ae5c52168d414b02f922ad8 has 0-length value 2023-10-21T04:11:39.745+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {a03babfad26a7c7d717571a6ae235db1 [] topology-change-rebalance [{{f2b29e6bf10e9349c06775f41b68f8ff 7060000 } recovery-full} {{31fd475f22d8ee0da906c31968c91305 7060000 } recovery-full}] [{85232c9ba3e4451b95e9e6fc8337e027 7060000 }]} 2023-10-21T04:11:39.753+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 26.489µs 2023-10-21T04:11:39.753+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:11:39.753+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:11:39.753+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:11:39.756+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:11:39.756+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:11:39.756+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 3.261534ms 2023-10-21T04:11:39.756+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:11:39.756+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:11:39.756+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 11.03µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:39.756+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 14] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:11:39.756+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. taskList: &{Rev:[0 0 0 0 0 0 0 14] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:11:39.760+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 14] 2023-10-21T04:11:39.762+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 14] 2023-10-21T04:11:39.788+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:11:39.835+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:39.835+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:39.835+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:11:39.901+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:11:39.986+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:11:39.991+05:30 [Info] switched currmeta from 622 -> 623 force true 2023-10-21T04:11:39.991+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:40.000+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:40.000+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:40.004+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:11:40.004+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:11:40.004+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:11:40.005+05:30 [Info] switched currmeta from 623 -> 623 force true 2023-10-21T04:11:40.005+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:11:40.017+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35506 2023-10-21T04:11:40.046+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.582432ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:40.047+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 69.371µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:40.159+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47102" doAuth() ... 2023-10-21T04:11:40.160+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47102" doReceive() ... 2023-10-21T04:11:40.177+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {a03babfad26a7c7d717571a6ae235db1 [] topology-change-rebalance [{{f2b29e6bf10e9349c06775f41b68f8ff 7060000 } recovery-full} {{31fd475f22d8ee0da906c31968c91305 7060000 } recovery-full}] [{85232c9ba3e4451b95e9e6fc8337e027 7060000 }]} 2023-10-21T04:11:40.179+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:11:40.207+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"31fd475f22d8ee0da906c31968c91305","RebalId":"a03babfad26a7c7d717571a6ae235db1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:11:40.209+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{31fd475f22d8ee0da906c31968c91305 a03babfad26a7c7d717571a6ae235db1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:11:40.211+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{31fd475f22d8ee0da906c31968c91305 a03babfad26a7c7d717571a6ae235db1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:11:40.230+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:11:40.230+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:11:40.247+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:11:40.248+05:30 [Info] NewShardRebalancer nodeId 31fd475f22d8ee0da906c31968c91305 rebalToken &{31fd475f22d8ee0da906c31968c91305 a03babfad26a7c7d717571a6ae235db1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:11:40.249+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:11:40.248+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 14] call. taskList: &{Rev:[0 0 0 0 0 0 0 15] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:11:40.249+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:11:40.248+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 15] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:11:40.257+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 15] 2023-10-21T04:11:40.258+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 15] 2023-10-21T04:11:40.260+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:11:40.260+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 15] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:11:40.260+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:11:40.260+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 15] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:11:40.266+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 15] 2023-10-21T04:11:40.268+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 15] 2023-10-21T04:11:40.356+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{f2b29e6bf10e9349c06775f41b68f8ff f2b29e6bf10e9349c06775f41b68f8ff plasma map[] [] [] 0 0 0 false} {31fd475f22d8ee0da906c31968c91305 31fd475f22d8ee0da906c31968c91305 plasma map[] [{7 bucket_1 _system _query [{bucket_1 _system _query #primary 12833670934079048728 [{13258659765240018889 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314] []}] 1 0 1 1 false plasma 0}]}]} {27 bucket_%2 _default c2% [{bucket_%2 _default c2% idx_secondary_defer 8669649359784409226 [{14910157020177557388 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default c2% idx_secondary 14741949230702299196 [{2483169159760864439 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned_defer 2374548411389959576 [{249283441109236971 1 0 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned 15991242037348127852 [{9664682240473767939 4 1 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default c2% #primary_defer 1844709904168202817 [{17037906325662037075 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default c2% #primary 7874378322029374196 [{17128008689111072444 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837] []}] 1 0 1 0 false plasma 0}]}]} {35 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned_defer 2872726210871813325 [{2307463109245789631 1 0 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary_defer 18323460706027642575 [{2288923528993975684 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned 16013578430151540990 [{16726320877690809560 4 1 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c1 #primary_defer 17489537288579938718 [{3985672602331234807 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default c1 #primary 1571318351652255745 [{17148287981272169168 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 5465835793930010953 [{16338040286194952893 4 1 [{0 0 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 0 0 false plasma 0}]}]} {30 bucket_1 _default c2% [{bucket_1 _default c2% idx_secondary 12140458060295645238 [{4004220452657849790 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default c2% idx_secondary_defer 6985049432225870291 [{11756349460704871322 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default c2% idx_partitioned 14026476026078401783 [{2170849387178940761 4 1 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c2% idx_partitioned_defer 10585832761158485697 [{7755999278497385093 1 0 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default c2% #primary_defer 9619308266130016313 [{4435591218553988516 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default c2% #primary 5103684059183126577 [{11160011617777336020 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314] []}] 1 0 1 0 false plasma 0}]}]} {30 bucket_%2 _default _default [{bucket_%2 _default _default idx_secondary 12027054384338714804 [{11400459767051699632 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default _default idx_partitioned 16162012140118277065 [{80911002600697866 4 1 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default _default idx_secondary_defer 18046371353728341033 [{8446412942681787061 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default _default idx_partitioned_defer 15431181238515168303 [{2921784364458449578 1 0 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default _default #primary_defer 13856704970496601958 [{8880461853743568138 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default _default #primary 8857269605176042330 [{17729593977321451597 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837] []}] 1 0 1 1 false plasma 0}]}]} {30 bucket_1 _default _default [{bucket_1 _default _default idx_secondary 14292583894719865678 [{3292093078969249751 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default _default idx_partitioned_defer 17656144409930763141 [{13953156762317267443 1 0 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default _default idx_secondary_defer 15945117906390072636 [{11739645294255241349 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default _default idx_partitioned 573521267164877540 [{1532172649864318584 4 1 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314 3268862397300335529] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default _default #primary 12338036531800130039 [{3810060733504350542 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default _default #primary_defer 11619165464560280276 [{7597064108789640455 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [10563635432419104314] []}] 1 0 1 0 false plasma 0}]}]} {7 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3674078063353481907 [{14780363202971367008 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837] []}] 1 0 1 1 false plasma 0}]}]} {38 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned_defer 15649171661394925140 [{12043401624902403496 1 0 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary_defer 15968368593314043432 [{325705934323625324 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned 14340086159258068923 [{7410154703733445404 4 1 [{1 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {2 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {3 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {4 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {5 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {6 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {7 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []} {8 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default c1 #primary 5197198800322775808 [{14607307599259773249 4 1 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default c1 #primary_defer 17878371269610391239 [{15131823346681830059 1 0 [{0 1 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12195715103909237684 [{18361169415695152662 4 1 [{0 0 {[{0 0 31fd475f22d8ee0da906c31968c91305}]} {[] []} [6611724767191156837 15964300471522801617] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {85232c9ba3e4451b95e9e6fc8337e027 85232c9ba3e4451b95e9e6fc8337e027 plasma map[] [{35 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned_defer 15649171661394925140 [{1955535609704140162 1 0 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary_defer 15968368593314043432 [{17161083103924027797 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned 14340086159258068923 [{3052172645459386575 4 1 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default c1 #primary 5197198800322775808 [{13718648355797887482 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default c1 #primary_defer 17878371269610391239 [{7287869093983910291 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12195715103909237684 [{14736339311583908837 4 1 [{0 0 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 0 false plasma 0}]}]} {27 bucket_1 _default c2% [{bucket_1 _default c2% idx_partitioned_defer 10585832761158485697 [{9358717944830195392 1 0 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c2% idx_secondary 12140458060295645238 [{3513301448177380059 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default c2% idx_partitioned 14026476026078401783 [{6393061446487756912 4 1 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default c2% idx_secondary_defer 6985049432225870291 [{4138705102427425280 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default c2% #primary_defer 9619308266130016313 [{11484519016652644658 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default c2% #primary 5103684059183126577 [{16983881020783103544 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630] []}] 1 0 1 1 false plasma 0}]}]} {30 bucket_1 _default _default [{bucket_1 _default _default idx_secondary 14292583894719865678 [{4362827867398787577 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default _default idx_partitioned 573521267164877540 [{8205593268191821683 4 1 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default _default idx_partitioned_defer 17656144409930763141 [{7291286939424384589 1 0 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default _default idx_secondary_defer 15945117906390072636 [{10455069697192970620 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default _default #primary 12338036531800130039 [{4503814398859864266 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630] []}] 1 0 1 0 false plasma 0}]} {bucket_1 _default _default #primary_defer 11619165464560280276 [{11158580445292211864 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630] []}] 1 0 1 1 false plasma 0}]}]} {30 bucket_%2 _default c2% [{bucket_%2 _default c2% idx_secondary_defer 8669649359784409226 [{6213797968262183765 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned 15991242037348127852 [{9010726441942672253 4 1 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default c2% idx_secondary 14741949230702299196 [{8899653570021014071 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default c2% idx_partitioned_defer 2374548411389959576 [{16044079815412989935 1 0 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default c2% #primary 7874378322029374196 [{11675990026580235111 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default c2% #primary_defer 1844709904168202817 [{1938940178723796232 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479] []}] 1 0 1 0 false plasma 0}]}]} {38 bucket_1 _default c1 [{bucket_1 _default c1 idx_secondary_defer 18323460706027642575 [{14991206072146508370 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned 16013578430151540990 [{5299824419646640495 4 1 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 2872726210871813325 [{18089525033132576654 1 0 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default c1 #primary 1571318351652255745 [{6796752657857722550 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default c1 #primary_defer 17489537288579938718 [{9134754361302452675 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630] []}] 1 0 1 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 5465835793930010953 [{13420534524606213282 4 1 [{0 0 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 1 false plasma 0}]}]} {30 bucket_%2 _default _default [{bucket_%2 _default _default idx_secondary_defer 18046371353728341033 [{18184041269291378080 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default _default idx_partitioned 16162012140118277065 [{18051244886420988269 4 1 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default _default idx_secondary 12027054384338714804 [{9246878583854294580 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 1 1 false plasma 0}]} {bucket_%2 _default _default idx_partitioned_defer 15431181238515168303 [{10574086758347311295 1 0 [{1 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default _default #primary_defer 13856704970496601958 [{15283305021611177781 1 0 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479] []}] 1 0 1 0 false plasma 0}]} {bucket_%2 _default _default #primary 8857269605176042330 [{16398781184179534607 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479] []}] 1 0 1 0 false plasma 0}]}]} {7 bucket_1 _system _query [{bucket_1 _system _query #primary 12833670934079048728 [{4894899676975150934 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [2864853750198994630] []}] 1 0 1 0 false plasma 0}]}]} {7 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3674078063353481907 [{360999437388707481 4 1 [{0 1 {[{0 0 85232c9ba3e4451b95e9e6fc8337e027}]} {[] []} [10890719122371003479] []}] 1 0 1 0 false plasma 0}]}]}] [DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false}] map[]} 2023-10-21T04:11:40.397+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43132 2023-10-21T04:11:40.400+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:11:40.400+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:11:40.400+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:11:40.401+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43132 2023-10-21T04:11:40.407+05:30 [Info] Planner::processDeleteToken: processing 2 tokens for node 127.0.0.1:9002 2023-10-21T04:11:40.407+05:30 [Info] Planner::processDeleteToken: processing 2 tokens for node 127.0.0.1:9004 2023-10-21T04:11:40.407+05:30 [Info] Planner::processDeleteToken: processing 2 tokens for node 127.0.0.1:9003 2023-10-21T04:11:40.408+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43132 2023-10-21T04:11:40.412+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:11:40.412+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:11:40.412+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9004] 2023-10-21T04:11:40.524+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9004] 2023-10-21T04:11:40.524+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9003] 2023-10-21T04:11:40.524+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9002] 2023-10-21T04:11:40.524+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9002] 2023-10-21T04:11:40.524+05:30 [Info] Moving index 8669649359784409226:6213797968262183765:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 11619165464560280276:11158580445292211864:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 2872726210871813325:18089525033132576654:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14026476026078401783:6393061446487756912:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 14741949230702299196:8899653570021014071:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 7874378322029374196:11675990026580235111:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 18323460706027642575:14991206072146508370:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 17489537288579938718:9134754361302452675:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 3674078063353481907:360999437388707481:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 12140458060295645238:3513301448177380059:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 1844709904168202817:1938940178723796232:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 5465835793930010953:13420534524606213282:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.524+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16013578430151540990:5299824419646640495:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 12027054384338714804:9246878583854294580:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 12833670934079048728:4894899676975150934:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 15968368593314043432:17161083103924027797:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 6985049432225870291:4138705102427425280:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 5103684059183126577:16983881020783103544:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 16162012140118277065:18051244886420988269:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 10585832761158485697:9358717944830195392:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 5197198800322775808:13718648355797887482:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.525+05:30 [Info] Moving index 15945117906390072636:10455069697192970620:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 12195715103909237684:14736339311583908837:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 9619308266130016313:11484519016652644658:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15991242037348127852:9010726441942672253:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15431181238515168303:10574086758347311295:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 1571318351652255745:6796752657857722550:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 8857269605176042330:16398781184179534607:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17878371269610391239:7287869093983910291:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 573521267164877540:8205593268191821683:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 2374548411389959576:16044079815412989935:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 17656144409930763141:7291286939424384589:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 12338036531800130039:4503814398859864266:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 18046371353728341033:18184041269291378080:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 13856704970496601958:15283305021611177781:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.526+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 15649171661394925140:1955535609704140162:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:1 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:2 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:3 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:4 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:5 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:6 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:7 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14340086159258068923:3052172645459386575:8 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Moving index 14292583894719865678:4362827867398787577:0 from source 127.0.0.1:9004 to dest 127.0.0.1:9002 2023-10-21T04:11:40.527+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG Group 1 Memory 17138834 Units 0 2023-10-21T04:11:40.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 18700280 Units 0 Tenants 2 2023-10-21T04:11:40.527+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG Group 1 Memory 1561466 Units 0 Tenants 0 2023-10-21T04:11:40.527+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9003 127.0.0.1:9002]] 2023-10-21T04:11:40.527+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:11:40.527+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:11:40.527+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:11:40.527+05:30 [Info] ---------------------------------------- 2023-10-21T04:11:40.527+05:30 [Info] 2023-10-21T04:11:40.527+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:31fd475f22d8ee0da906c31968c91305, useLiveData:false 2023-10-21T04:11:40.527+05:30 [Info] Indexer total memory:19296765748 (17.9715G), mem:2569153 (2.45014M), overhead:19294196595 (17.9691G), min:19296765748 (17.9715G), data:2569153 (2.45014M) cpu:1.0234, io:0 (0), scan:0 drain:0 numIndexes:122 shardCompatVersion: 1 2023-10-21T04:11:40.527+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:11:40.527+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.527+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:5197198800322775808, instId:14607307599259773249, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:11:40.527+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0754 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.527+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.527+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.527+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:12140458060295645238, instId:4004220452657849790, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157415760 (150.123M), mem:112000 (109.375K), overhead:157303760 (150.017M), min:157415760 (150.123M), data:112000 (109.375K) cpu:0.0420 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:5103684059183126577, instId:11160011617777336020, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0364 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157327665 (150.039M), mem:35728 (34.8906K), overhead:157291937 (150.005M), min:157327665 (150.039M), data:35728 (34.8906K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157329303 (150.041M), mem:37146 (36.2754K), overhead:157292157 (150.005M), min:157329303 (150.041M), data:37146 (36.2754K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157328735 (150.04M), mem:36654 (35.7949K), overhead:157292081 (150.005M), min:157328735 (150.04M), data:36654 (35.7949K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157329116 (150.041M), mem:36984 (36.1172K), overhead:157292132 (150.005M), min:157329116 (150.041M), data:36984 (36.1172K) cpu:0.0132 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157328426 (150.04M), mem:36387 (35.5342K), overhead:157292039 (150.005M), min:157328426 (150.04M), data:36387 (35.5342K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157328965 (150.041M), mem:36853 (35.9893K), overhead:157292112 (150.005M), min:157328965 (150.041M), data:36853 (35.9893K) cpu:0.0132 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157326573 (150.038M), mem:34782 (33.9668K), overhead:157291791 (150.005M), min:157326573 (150.038M), data:34782 (33.9668K) cpu:0.0098 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:7410154703733445404, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157330502 (150.042M), mem:38184 (37.2891K), overhead:157292318 (150.006M), min:157330502 (150.042M), data:38184 (37.2891K) cpu:0.0142 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.528+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3674078063353481907, instId:14780363202971367008, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:11:40.528+05:30 [Info] Index total memory:157296174 (150.009M), mem:8463 (8.26465K), overhead:157287711 (150.001M), min:157296174 (150.009M), data:8463 (8.26465K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.528+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.528+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.529+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:12195715103909237684, instId:18361169415695152662, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.529+05:30 [Info] Index total memory:157415760 (150.123M), mem:112000 (109.375K), overhead:157303760 (150.017M), min:157415760 (150.123M), data:112000 (109.375K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.529+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.529+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.529+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:15968368593314043432, instId:325705934323625324, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.529+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.529+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.529+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.529+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:8669649359784409226, instId:14910157020177557388, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.529+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.529+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.529+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.529+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.529+05:30 [Info] Index total memory:157327224 (150.039M), mem:35346 (34.5176K), overhead:157291878 (150.005M), min:157327224 (150.039M), data:35346 (34.5176K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.529+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.529+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.529+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.529+05:30 [Info] Index total memory:157326293 (150.038M), mem:34540 (33.7305K), overhead:157291753 (150.005M), min:157326293 (150.038M), data:34540 (33.7305K) cpu:0.0092 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.529+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.529+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.529+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.529+05:30 [Info] Index total memory:157327075 (150.039M), mem:35217 (34.3916K), overhead:157291858 (150.005M), min:157327075 (150.039M), data:35217 (34.3916K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.529+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.529+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157328280 (150.04M), mem:36260 (35.4102K), overhead:157292020 (150.005M), min:157328280 (150.04M), data:36260 (35.4102K) cpu:0.0116 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157326861 (150.039M), mem:35032 (34.2109K), overhead:157291829 (150.005M), min:157326861 (150.039M), data:35032 (34.2109K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157327031 (150.039M), mem:35179 (34.3545K), overhead:157291852 (150.005M), min:157327031 (150.039M), data:35179 (34.3545K) cpu:0.0106 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157326825 (150.039M), mem:35000 (34.1797K), overhead:157291825 (150.005M), min:157326825 (150.039M), data:35000 (34.1797K) cpu:0.0104 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:80911002600697866, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157327647 (150.039M), mem:35712 (34.875K), overhead:157291935 (150.005M), min:157327647 (150.039M), data:35712 (34.875K) cpu:0.0104 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:6985049432225870291, instId:11756349460704871322, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157328967 (150.041M), mem:36855 (35.9912K), overhead:157292112 (150.005M), min:157328967 (150.041M), data:36855 (35.9912K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157327183 (150.039M), mem:35310 (34.4824K), overhead:157291873 (150.005M), min:157327183 (150.039M), data:35310 (34.4824K) cpu:0.0040 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157329435 (150.041M), mem:37260 (36.3867K), overhead:157292175 (150.006M), min:157329435 (150.041M), data:37260 (36.3867K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.530+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.530+05:30 [Info] Index total memory:157329123 (150.041M), mem:36990 (36.123K), overhead:157292133 (150.005M), min:157329123 (150.041M), data:36990 (36.123K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.530+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.530+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.531+05:30 [Info] Index total memory:157328614 (150.04M), mem:36549 (35.6924K), overhead:157292065 (150.005M), min:157328614 (150.04M), data:36549 (35.6924K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.531+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.531+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.531+05:30 [Info] Index total memory:157328505 (150.04M), mem:36455 (35.6006K), overhead:157292050 (150.005M), min:157328505 (150.04M), data:36455 (35.6006K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.531+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.531+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.531+05:30 [Info] Index total memory:157327819 (150.04M), mem:35861 (35.0205K), overhead:157291958 (150.005M), min:157327819 (150.04M), data:35861 (35.0205K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.531+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.531+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:2170849387178940761, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.531+05:30 [Info] Index total memory:157327943 (150.04M), mem:35968 (35.125K), overhead:157291975 (150.005M), min:157327943 (150.04M), data:35968 (35.125K) cpu:0.0048 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.531+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.531+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:12338036531800130039, instId:3810060733504350542, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:11:40.531+05:30 [Info] Index total memory:157315275 (150.028M), mem:25000 (24.4141K), overhead:157290275 (150.004M), min:157315275 (150.028M), data:25000 (24.4141K) cpu:0.0754 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.531+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.531+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_1, scope:_default, collection:_default, defnId:11619165464560280276, instId:7597064108789640455, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:11:40.531+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.531+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.531+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.531+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.531+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.531+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.531+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:249283441109236971, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:157328342 (150.04M), mem:36314 (35.4629K), overhead:157292028 (150.005M), min:157328342 (150.04M), data:36314 (35.4629K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:157328811 (150.04M), mem:36720 (35.8594K), overhead:157292091 (150.005M), min:157328811 (150.04M), data:36720 (35.8594K) cpu:0.0106 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:157329104 (150.041M), mem:36974 (36.1074K), overhead:157292130 (150.005M), min:157329104 (150.041M), data:36974 (36.1074K) cpu:0.0122 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.532+05:30 [Info] Index total memory:157327536 (150.039M), mem:35616 (34.7812K), overhead:157291920 (150.005M), min:157327536 (150.039M), data:35616 (34.7812K) cpu:0.0098 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.532+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.532+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.532+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157328490 (150.04M), mem:36442 (35.5879K), overhead:157292048 (150.005M), min:157328490 (150.04M), data:36442 (35.5879K) cpu:0.0116 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157328466 (150.04M), mem:36421 (35.5674K), overhead:157292045 (150.005M), min:157328466 (150.04M), data:36421 (35.5674K) cpu:0.0106 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157328183 (150.04M), mem:36176 (35.3281K), overhead:157292007 (150.005M), min:157328183 (150.04M), data:36176 (35.3281K) cpu:0.0104 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:16726320877690809560, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157328532 (150.04M), mem:36478 (35.623K), overhead:157292054 (150.005M), min:157328532 (150.04M), data:36478 (35.623K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:12027054384338714804, instId:11400459767051699632, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0770 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:_default, defnId:14292583894719865678, instId:3292093078969249751, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0450 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157328257 (150.04M), mem:36240 (35.3906K), overhead:157292017 (150.005M), min:157328257 (150.04M), data:36240 (35.3906K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157327820 (150.04M), mem:35862 (35.0215K), overhead:157291958 (150.005M), min:157327820 (150.04M), data:35862 (35.0215K) cpu:0.0102 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.533+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.533+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.533+05:30 [Info] Index total memory:157328326 (150.04M), mem:36300 (35.4492K), overhead:157292026 (150.005M), min:157328326 (150.04M), data:36300 (35.4492K) cpu:0.0106 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.533+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:157328291 (150.04M), mem:36270 (35.4199K), overhead:157292021 (150.005M), min:157328291 (150.04M), data:36270 (35.4199K) cpu:0.0104 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:157327943 (150.04M), mem:35968 (35.125K), overhead:157291975 (150.005M), min:157327943 (150.04M), data:35968 (35.125K) cpu:0.0102 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:157326434 (150.038M), mem:34662 (33.8496K), overhead:157291772 (150.005M), min:157326434 (150.038M), data:34662 (33.8496K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:157328326 (150.04M), mem:36300 (35.4492K), overhead:157292026 (150.005M), min:157328326 (150.04M), data:36300 (35.4492K) cpu:0.0106 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9664682240473767939, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:157328593 (150.04M), mem:36531 (35.6748K), overhead:157292062 (150.005M), min:157328593 (150.04M), data:36531 (35.6748K) cpu:0.0098 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:18046371353728341033, instId:8446412942681787061, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:9619308266130016313, instId:4435591218553988516, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_1, scope:_default, collection:_default, defnId:15945117906390072636, instId:11739645294255241349, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.534+05:30 [Info] Index total memory:157327878 (150.04M), mem:35912 (35.0703K), overhead:157291966 (150.005M), min:157327878 (150.04M), data:35912 (35.0703K) cpu:0.0048 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.534+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.534+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.534+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157326337 (150.038M), mem:34578 (33.7676K), overhead:157291759 (150.005M), min:157326337 (150.038M), data:34578 (33.7676K) cpu:0.0040 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157327411 (150.039M), mem:35508 (34.6758K), overhead:157291903 (150.005M), min:157327411 (150.039M), data:35508 (34.6758K) cpu:0.0046 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157328220 (150.04M), mem:36208 (35.3594K), overhead:157292012 (150.005M), min:157328220 (150.04M), data:36208 (35.3594K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157326399 (150.038M), mem:34632 (33.8203K), overhead:157291767 (150.005M), min:157326399 (150.038M), data:34632 (33.8203K) cpu:0.0040 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157328118 (150.04M), mem:36120 (35.2734K), overhead:157291998 (150.005M), min:157328118 (150.04M), data:36120 (35.2734K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157327258 (150.039M), mem:35375 (34.5459K), overhead:157291883 (150.005M), min:157327258 (150.039M), data:35375 (34.5459K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:1532172649864318584, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157328186 (150.04M), mem:36179 (35.3311K), overhead:157292007 (150.005M), min:157328186 (150.04M), data:36179 (35.3311K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_1, scope:_default, collection:c1, defnId:17489537288579938718, instId:3985672602331234807, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:1571318351652255745, instId:17148287981272169168, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0398 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.535+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.535+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.535+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.535+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:7755999278497385093, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:13856704970496601958, instId:8880461853743568138, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.536+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.536+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.536+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.536+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.537+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:12043401624902403496, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.537+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:14741949230702299196, instId:2483169159760864439, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:157415760 (150.123M), mem:112000 (109.375K), overhead:157303760 (150.017M), min:157415760 (150.123M), data:112000 (109.375K) cpu:0.0902 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.537+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_1, scope:_default, collection:c1, defnId:18323460706027642575, instId:2288923528993975684, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.537+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:1844709904168202817, instId:17037906325662037075, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.537+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:8857269605176042330, instId:17729593977321451597, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0364 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.537+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.537+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.537+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.537+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.537+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:13953156762317267443, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:12833670934079048728, instId:13258659765240018889, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:157296393 (150.01M), mem:8652 (8.44922K), overhead:157287741 (150.001M), min:157296393 (150.01M), data:8652 (8.44922K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:17878371269610391239, instId:15131823346681830059, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:5465835793930010953, instId:16338040286194952893, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.538+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.538+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.538+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.538+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.539+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.539+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.539+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.539+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.539+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.539+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.539+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.539+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.539+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.539+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.539+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.539+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.539+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.539+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.539+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.539+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.539+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.539+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.539+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:2307463109245789631, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10563635432419104314 3268862397300335529], shardIdsInDefn: map[] 2023-10-21T04:11:40.539+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.539+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.539+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.539+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:7874378322029374196, instId:17128008689111072444, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837], shardIdsInDefn: map[] 2023-10-21T04:11:40.539+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0854 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.539+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.539+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.539+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.539+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.540+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.540+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.540+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.540+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.540+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.540+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.540+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.540+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.540+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.540+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.540+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.540+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.540+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:2921784364458449578, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [6611724767191156837 15964300471522801617], shardIdsInDefn: map[] 2023-10-21T04:11:40.540+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.540+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.540+05:30 [Info] 2023-10-21T04:11:40.540+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9002, nodeUUID:f2b29e6bf10e9349c06775f41b68f8ff, useLiveData:false 2023-10-21T04:11:40.540+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:122 shardCompatVersion: 1 2023-10-21T04:11:40.541+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:8669649359784409226, instId:6213797968262183765, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:11619165464560280276, instId:11158580445292211864, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.541+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.541+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.541+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.541+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:2872726210871813325, instId:18089525033132576654, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157329123 (150.041M), mem:36990 (36.123K), overhead:157292133 (150.005M), min:157329123 (150.041M), data:36990 (36.123K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157327183 (150.039M), mem:35310 (34.4824K), overhead:157291873 (150.005M), min:157327183 (150.039M), data:35310 (34.4824K) cpu:0.0094 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157329275 (150.041M), mem:37122 (36.252K), overhead:157292153 (150.005M), min:157329275 (150.041M), data:37122 (36.252K) cpu:0.0120 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157329123 (150.041M), mem:36990 (36.123K), overhead:157292133 (150.005M), min:157329123 (150.041M), data:36990 (36.123K) cpu:0.0120 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157328614 (150.04M), mem:36549 (35.6924K), overhead:157292065 (150.005M), min:157328614 (150.04M), data:36549 (35.6924K) cpu:0.0114 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157328372 (150.04M), mem:36340 (35.4883K), overhead:157292032 (150.005M), min:157328372 (150.04M), data:36340 (35.4883K) cpu:0.0100 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157327819 (150.04M), mem:35861 (35.0205K), overhead:157291958 (150.005M), min:157327819 (150.04M), data:35861 (35.0205K) cpu:0.0094 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c2%, defnId:14026476026078401783, instId:6393061446487756912, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157328386 (150.04M), mem:36352 (35.5K), overhead:157292034 (150.005M), min:157328386 (150.04M), data:36352 (35.5K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:14741949230702299196, instId:8899653570021014071, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157415760 (150.123M), mem:112000 (109.375K), overhead:157303760 (150.017M), min:157415760 (150.123M), data:112000 (109.375K) cpu:0.0748 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:7874378322029374196, instId:11675990026580235111, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0784 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.542+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.542+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.542+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:18323460706027642575, instId:14991206072146508370, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.542+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:17489537288579938718, instId:9134754361302452675, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3674078063353481907, instId:360999437388707481, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157296150 (150.009M), mem:8442 (8.24414K), overhead:157287708 (150.001M), min:157296150 (150.009M), data:8442 (8.24414K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:12140458060295645238, instId:3513301448177380059, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0812 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_%2, scope:_default, collection:c2%, defnId:1844709904168202817, instId:1938940178723796232, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:5465835793930010953, instId:13420534524606213282, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157328342 (150.04M), mem:36314 (35.4629K), overhead:157292028 (150.005M), min:157328342 (150.04M), data:36314 (35.4629K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157328395 (150.04M), mem:36360 (35.5078K), overhead:157292035 (150.005M), min:157328395 (150.04M), data:36360 (35.5078K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157328944 (150.041M), mem:36835 (35.9717K), overhead:157292109 (150.005M), min:157328944 (150.041M), data:36835 (35.9717K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157327536 (150.039M), mem:35616 (34.7812K), overhead:157291920 (150.005M), min:157327536 (150.039M), data:35616 (34.7812K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157328490 (150.04M), mem:36442 (35.5879K), overhead:157292048 (150.005M), min:157328490 (150.04M), data:36442 (35.5879K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157328606 (150.04M), mem:36542 (35.6855K), overhead:157292064 (150.005M), min:157328606 (150.04M), data:36542 (35.6855K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.543+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.543+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.543+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.543+05:30 [Info] Index total memory:157328320 (150.04M), mem:36295 (35.4443K), overhead:157292025 (150.005M), min:157328320 (150.04M), data:36295 (35.4443K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16013578430151540990, instId:5299824419646640495, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:157328673 (150.04M), mem:36600 (35.7422K), overhead:157292073 (150.005M), min:157328673 (150.04M), data:36600 (35.7422K) cpu:0.0046 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:12027054384338714804, instId:9246878583854294580, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0436 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:12833670934079048728, instId:4894899676975150934, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:157296320 (150.009M), mem:8589 (8.3877K), overhead:157287731 (150.001M), min:157296320 (150.009M), data:8589 (8.3877K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15968368593314043432, instId:17161083103924027797, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:6985049432225870291, instId:4138705102427425280, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:5103684059183126577, instId:16983881020783103544, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0932 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:157326926 (150.039M), mem:35088 (34.2656K), overhead:157291838 (150.005M), min:157326926 (150.039M), data:35088 (34.2656K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:157326166 (150.038M), mem:34430 (33.623K), overhead:157291736 (150.005M), min:157326166 (150.038M), data:34430 (33.623K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.544+05:30 [Info] Index total memory:157326777 (150.039M), mem:34959 (34.1396K), overhead:157291818 (150.005M), min:157326777 (150.039M), data:34959 (34.1396K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.544+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.544+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.544+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:157327956 (150.04M), mem:35980 (35.1367K), overhead:157291976 (150.005M), min:157327956 (150.04M), data:35980 (35.1367K) cpu:0.0056 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:157326727 (150.038M), mem:34916 (34.0977K), overhead:157291811 (150.005M), min:157326727 (150.038M), data:34916 (34.0977K) cpu:0.0046 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:157326591 (150.038M), mem:34798 (33.9824K), overhead:157291793 (150.005M), min:157326591 (150.038M), data:34798 (33.9824K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:157326680 (150.038M), mem:34875 (34.0576K), overhead:157291805 (150.005M), min:157326680 (150.038M), data:34875 (34.0576K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:18051244886420988269, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:157327504 (150.039M), mem:35588 (34.7539K), overhead:157291916 (150.005M), min:157327504 (150.039M), data:35588 (34.7539K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.545+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.545+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.545+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.545+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c2%, defnId:10585832761158485697, instId:9358717944830195392, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:5197198800322775808, instId:13718648355797887482, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0894 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:15945117906390072636, instId:10455069697192970620, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:12195715103909237684, instId:14736339311583908837, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_1, scope:_default, collection:c2%, defnId:9619308266130016313, instId:11484519016652644658, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:157328395 (150.04M), mem:36360 (35.5078K), overhead:157292035 (150.005M), min:157328395 (150.04M), data:36360 (35.5078K) cpu:0.0100 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:157327820 (150.04M), mem:35862 (35.0215K), overhead:157291958 (150.005M), min:157327820 (150.04M), data:35862 (35.0215K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:157328326 (150.04M), mem:36300 (35.4492K), overhead:157292026 (150.005M), min:157328326 (150.04M), data:36300 (35.4492K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:157328141 (150.04M), mem:36140 (35.293K), overhead:157292001 (150.005M), min:157328141 (150.04M), data:36140 (35.293K) cpu:0.0110 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.546+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.546+05:30 [Info] Index total memory:157327795 (150.039M), mem:35840 (35K), overhead:157291955 (150.005M), min:157327795 (150.039M), data:35840 (35K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.546+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.546+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:157326312 (150.038M), mem:34556 (33.7461K), overhead:157291756 (150.005M), min:157326312 (150.038M), data:34556 (33.7461K) cpu:0.0090 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:157328021 (150.04M), mem:36036 (35.1914K), overhead:157291985 (150.005M), min:157328021 (150.04M), data:36036 (35.1914K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c2%, defnId:15991242037348127852, instId:9010726441942672253, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:157328735 (150.04M), mem:36654 (35.7949K), overhead:157292081 (150.005M), min:157328735 (150.04M), data:36654 (35.7949K) cpu:0.0104 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.547+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.547+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.547+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.547+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:10574086758347311295, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:1571318351652255745, instId:6796752657857722550, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0852 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:8857269605176042330, instId:16398781184179534607, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157315275 (150.028M), mem:25000 (24.4141K), overhead:157290275 (150.004M), min:157315275 (150.028M), data:25000 (24.4141K) cpu:0.0922 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17878371269610391239, instId:7287869093983910291, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157327568 (150.039M), mem:35644 (34.8086K), overhead:157291924 (150.005M), min:157327568 (150.039M), data:35644 (34.8086K) cpu:0.0062 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157326598 (150.038M), mem:34804 (33.9883K), overhead:157291794 (150.005M), min:157326598 (150.038M), data:34804 (33.9883K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157327259 (150.039M), mem:35376 (34.5469K), overhead:157291883 (150.005M), min:157327259 (150.039M), data:35376 (34.5469K) cpu:0.0060 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157328363 (150.04M), mem:36332 (35.4805K), overhead:157292031 (150.005M), min:157328363 (150.04M), data:36332 (35.4805K) cpu:0.0056 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157326399 (150.038M), mem:34632 (33.8203K), overhead:157291767 (150.005M), min:157326399 (150.038M), data:34632 (33.8203K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.548+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.548+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.548+05:30 [Info] Index total memory:157328118 (150.04M), mem:36120 (35.2734K), overhead:157291998 (150.005M), min:157328118 (150.04M), data:36120 (35.2734K) cpu:0.0064 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.548+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:157327113 (150.039M), mem:35250 (34.4238K), overhead:157291863 (150.005M), min:157327113 (150.039M), data:35250 (34.4238K) cpu:0.0058 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:8205593268191821683, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:157327907 (150.04M), mem:35937 (35.0947K), overhead:157291970 (150.005M), min:157327907 (150.04M), data:35937 (35.0947K) cpu:0.0056 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.549+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.549+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.549+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.549+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c2%, defnId:2374548411389959576, instId:16044079815412989935, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.550+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.550+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.550+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7291286939424384589, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.550+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_default, collection:_default, defnId:12338036531800130039, instId:4503814398859864266, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:157316430 (150.029M), mem:26000 (25.3906K), overhead:157290430 (150.004M), min:157316430 (150.029M), data:26000 (25.3906K) cpu:0.0446 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:18046371353728341033, instId:18184041269291378080, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:13856704970496601958, instId:15283305021611177781, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.551+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.551+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.551+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.551+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:15649171661394925140, instId:1955535609704140162, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157327536 (150.039M), mem:35616 (34.7812K), overhead:157291920 (150.005M), min:157327536 (150.039M), data:35616 (34.7812K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157329445 (150.041M), mem:37269 (36.3955K), overhead:157292176 (150.006M), min:157329445 (150.041M), data:37269 (36.3955K) cpu:0.0082 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157329161 (150.041M), mem:37023 (36.1553K), overhead:157292138 (150.005M), min:157329161 (150.041M), data:37023 (36.1553K) cpu:0.0082 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157329435 (150.041M), mem:37260 (36.3867K), overhead:157292175 (150.006M), min:157329435 (150.041M), data:37260 (36.3867K) cpu:0.0092 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157328426 (150.04M), mem:36387 (35.5342K), overhead:157292039 (150.005M), min:157328426 (150.04M), data:36387 (35.5342K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157329281 (150.041M), mem:37127 (36.2568K), overhead:157292154 (150.005M), min:157329281 (150.041M), data:37127 (36.2568K) cpu:0.0092 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157326808 (150.039M), mem:34986 (34.166K), overhead:157291822 (150.005M), min:157326808 (150.039M), data:34986 (34.166K) cpu:0.0068 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14340086159258068923, instId:3052172645459386575, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:11:40.552+05:30 [Info] Index total memory:157330502 (150.042M), mem:38184 (37.2891K), overhead:157292318 (150.006M), min:157330502 (150.042M), data:38184 (37.2891K) cpu:0.0100 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.552+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.552+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:11:40.552+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:14292583894719865678, instId:4362827867398787577, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:11:40.553+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0370 io:0 (0) scan:0 drain:0 2023-10-21T04:11:40.553+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:11:40.553+05:30 [Info] **************************************** 2023-10-21T04:11:40.553+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 18700280 Units 0 Tenants 2 2023-10-21T04:11:40.553+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9002 SG Group 1 Memory 1561466 Units 0 Tenants 0 2023-10-21T04:11:40.554+05:30 [Info] Generating Shard Transfer Token (ShardTokenff:15:2d:39:84:e6:37:31) for rebalance ( MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: Region: InstId: 6213797968262183765 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 6213797968262183765 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 8899653570021014071 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 9246878583854294580 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 17161083103924027797 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 17161083103924027797 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 7170604883654040991 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 18051244886420988269 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 14736339311583908837 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14736339311583908837 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 6137244366124413283 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 9010726441942672253 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 6054633421341911295 RealInstId: 10574086758347311295 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 10574086758347311295 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 16060708638314595734 RealInstId: 16044079815412989935 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 16044079815412989935 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 18184041269291378080 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 18184041269291378080 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 13518927892577684793 RealInstId: 1955535609704140162 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 1955535609704140162 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 6452766258658064766 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 3052172645459386575 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 11675990026580235111 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 360999437388707481 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 360999437388707481 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 1938940178723796232 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 1938940178723796232 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 13718648355797887482 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 16398781184179534607 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 16398781184179534607 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 7287869093983910291 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 7287869093983910291 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 15283305021611177781 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 15283305021611177781 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:11:40.556+05:30 [Info] Generating Shard Transfer Token (ShardToken47:ec:19:47:fd:b7:cd:73) for rebalance ( MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: Region: InstId: 5653766467314045229 RealInstId: 18089525033132576654 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 18089525033132576654 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 17533500296092519815 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 6393061446487756912 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 14991206072146508370 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 14991206072146508370 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 3513301448177380059 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 13420534524606213282 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 13420534524606213282 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 6492262833116702924 RealInstId: 5299824419646640495 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 5299824419646640495 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 4138705102427425280 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 4138705102427425280 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 13207161782265087251 RealInstId: 9358717944830195392 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 9358717944830195392 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 10455069697192970620 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 10455069697192970620 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 10819337149286705062 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 8205593268191821683 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 5179062340836596587 RealInstId: 7291286939424384589 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 7291286939424384589 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 4362827867398787577 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 11158580445292211864 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 11158580445292211864 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 9134754361302452675 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 9134754361302452675 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 4894899676975150934 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 4894899676975150934 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 16983881020783103544 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 11484519016652644658 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 11484519016652644658 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 6796752657857722550 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 6796752657857722550 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 4503814398859864266 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:11:40.558+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:11:40.558+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:11:40.570+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenff:15:2d:39:84:e6:37:31 2023-10-21T04:11:40.592+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken47:ec:19:47:fd:b7:cd:73 2023-10-21T04:11:40.592+05:30 [Info] ShardRebalancer::observeRebalance &{31fd475f22d8ee0da906c31968c91305 a03babfad26a7c7d717571a6ae235db1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:11:40.604+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:40.620+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:40.620+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"31fd475f22d8ee0da906c31968c91305","RebalId":"a03babfad26a7c7d717571a6ae235db1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:11:40.620+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:11:40.633+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:40.676+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:40.742+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:40.749+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:40.749+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:11:40.755+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{31fd475f22d8ee0da906c31968c91305 a03babfad26a7c7d717571a6ae235db1 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:11:40.755+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 2 2023-10-21T04:11:40.756+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{31fd475f22d8ee0da906c31968c91305 a03babfad26a7c7d717571a6ae235db1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:11:40.808+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[85232c9ba3e4451b95e9e6fc8337e027:ShardToken47:ec:19:47:fd:b7:cd:73], currActiveTransfers: map[85232c9ba3e4451b95e9e6fc8337e027:2] 2023-10-21T04:11:40.808+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"31fd475f22d8ee0da906c31968c91305","RebalId":"a03babfad26a7c7d717571a6ae235db1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:11:40.808+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:11:40.808+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:11:40.808+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:11:40.808+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:11:40.808+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:11:40.810+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 55.064µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:11:40.810+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:40.819+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:41.071+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:41.071+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[85232c9ba3e4451b95e9e6fc8337e027:1 f2b29e6bf10e9349c06775f41b68f8ff:1], batchSize: 2 2023-10-21T04:11:41.179+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:41.179+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[85232c9ba3e4451b95e9e6fc8337e027:1 f2b29e6bf10e9349c06775f41b68f8ff:0], batchSize: 2 2023-10-21T04:11:41.238+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:41.238+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:41.267+05:30 [Info] switched currmeta from 623 -> 624 force true 2023-10-21T04:11:41.322+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:41.322+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[85232c9ba3e4451b95e9e6fc8337e027:0 f2b29e6bf10e9349c06775f41b68f8ff:1], batchSize: 2 2023-10-21T04:11:41.381+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:11:41.381+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[85232c9ba3e4451b95e9e6fc8337e027:0 f2b29e6bf10e9349c06775f41b68f8ff:0], batchSize: 2 2023-10-21T04:11:41.453+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:41.453+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:41.459+05:30 [Info] switched currmeta from 624 -> 625 force true 2023-10-21T04:11:41.754+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:41.754+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:41.756+05:30 [Info] switched currmeta from 625 -> 626 force true 2023-10-21T04:11:42.006+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:42.006+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:42.010+05:30 [Info] switched currmeta from 626 -> 627 force true 2023-10-21T04:11:42.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:42.195+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:42.198+05:30 [Info] switched currmeta from 627 -> 628 force true 2023-10-21T04:11:42.228+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:42.228+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:42.245+05:30 [Info] switched currmeta from 628 -> 629 force true 2023-10-21T04:11:42.345+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:42.345+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:42.348+05:30 [Info] switched currmeta from 629 -> 630 force true 2023-10-21T04:11:42.476+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:11:42.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:42.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:42.508+05:30 [Info] switched currmeta from 630 -> 632 force true 2023-10-21T04:11:42.514+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:42.514+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:42.521+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:11:43.263+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:43.263+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:43.277+05:30 [Info] switched currmeta from 632 -> 633 force true 2023-10-21T04:11:43.480+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:43.480+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:43.484+05:30 [Info] switched currmeta from 633 -> 634 force true 2023-10-21T04:11:43.603+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:43.603+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:43.614+05:30 [Info] switched currmeta from 634 -> 635 force true 2023-10-21T04:11:43.631+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:43.631+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:43.638+05:30 [Info] switched currmeta from 635 -> 636 force true 2023-10-21T04:11:44.319+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:44.319+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:44.325+05:30 [Info] switched currmeta from 636 -> 637 force true 2023-10-21T04:11:44.537+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:44.537+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:44.547+05:30 [Info] switched currmeta from 637 -> 638 force true 2023-10-21T04:11:44.576+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:44.576+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:44.587+05:30 [Info] switched currmeta from 638 -> 639 force true 2023-10-21T04:11:44.802+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 99.94366197183263 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:44.802+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 99.94366197183263 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:44.802+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:44.802+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:44.802+05:30 [Info] ShardRebalancer::computeProgress 0.7029999555226094 2023-10-21T04:11:45.044+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.044+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.051+05:30 [Info] switched currmeta from 639 -> 640 force true 2023-10-21T04:11:45.250+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 15] call. taskList: &{Rev:[0 0 0 0 0 0 0 16] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029999555226094 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:11:45.250+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 16] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:11:45.250+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 16] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:11:45.250+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 15] call. taskList: &{Rev:[0 0 0 0 0 0 0 16] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029999555226094 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:11:45.254+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 16] 2023-10-21T04:11:45.254+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 16] 2023-10-21T04:11:45.286+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.286+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.290+05:30 [Info] switched currmeta from 640 -> 641 force true 2023-10-21T04:11:45.344+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.344+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.351+05:30 [Info] switched currmeta from 641 -> 642 force true 2023-10-21T04:11:45.393+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.393+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.404+05:30 [Info] switched currmeta from 642 -> 643 force true 2023-10-21T04:11:45.452+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.452+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.456+05:30 [Info] switched currmeta from 643 -> 644 force true 2023-10-21T04:11:45.642+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.642+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.644+05:30 [Info] switched currmeta from 644 -> 646 force true 2023-10-21T04:11:45.651+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:45.654+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.654+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.667+05:30 [Info] switched currmeta from 646 -> 647 force true 2023-10-21T04:11:45.676+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.676+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.678+05:30 [Info] switched currmeta from 647 -> 647 force true 2023-10-21T04:11:45.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:45.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:45.686+05:30 [Info] switched currmeta from 647 -> 647 force true 2023-10-21T04:11:46.449+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:46.450+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:46.453+05:30 [Info] switched currmeta from 647 -> 648 force true 2023-10-21T04:11:46.491+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:46.491+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:46.495+05:30 [Info] switched currmeta from 648 -> 649 force true 2023-10-21T04:11:46.664+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:46.664+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:46.667+05:30 [Info] switched currmeta from 649 -> 650 force true 2023-10-21T04:11:46.682+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:46.683+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:46.689+05:30 [Info] switched currmeta from 650 -> 651 force true 2023-10-21T04:11:47.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.455+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.459+05:30 [Info] switched currmeta from 651 -> 652 force true 2023-10-21T04:11:47.480+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.480+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.485+05:30 [Info] switched currmeta from 652 -> 653 force true 2023-10-21T04:11:47.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.517+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.521+05:30 [Info] switched currmeta from 653 -> 655 force true 2023-10-21T04:11:47.532+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.532+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.535+05:30 [Info] switched currmeta from 655 -> 655 force true 2023-10-21T04:11:47.589+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.589+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.595+05:30 [Info] switched currmeta from 655 -> 656 force true 2023-10-21T04:11:47.617+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.624+05:30 [Info] switched currmeta from 656 -> 657 force true 2023-10-21T04:11:47.666+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.666+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.670+05:30 [Info] switched currmeta from 657 -> 659 force true 2023-10-21T04:11:47.678+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.678+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.682+05:30 [Info] switched currmeta from 659 -> 659 force true 2023-10-21T04:11:47.691+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.691+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.693+05:30 [Info] switched currmeta from 659 -> 660 force true 2023-10-21T04:11:47.723+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.723+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.727+05:30 [Info] switched currmeta from 660 -> 661 force true 2023-10-21T04:11:47.740+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.740+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.745+05:30 [Info] switched currmeta from 661 -> 662 force true 2023-10-21T04:11:47.841+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.841+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.845+05:30 [Info] switched currmeta from 662 -> 663 force true 2023-10-21T04:11:47.870+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.870+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.875+05:30 [Info] switched currmeta from 663 -> 664 force true 2023-10-21T04:11:47.884+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:47.884+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:47.889+05:30 [Info] switched currmeta from 664 -> 665 force true 2023-10-21T04:11:48.222+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.222+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.229+05:30 [Info] switched currmeta from 665 -> 667 force true 2023-10-21T04:11:48.234+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.234+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.243+05:30 [Info] switched currmeta from 667 -> 668 force true 2023-10-21T04:11:48.248+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.248+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.253+05:30 [Info] switched currmeta from 668 -> 668 force true 2023-10-21T04:11:48.619+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.619+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.622+05:30 [Info] switched currmeta from 668 -> 669 force true 2023-10-21T04:11:48.649+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.649+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.668+05:30 [Info] switched currmeta from 669 -> 670 force true 2023-10-21T04:11:48.751+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.751+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.761+05:30 [Info] switched currmeta from 670 -> 674 force true 2023-10-21T04:11:48.780+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.780+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.783+05:30 [Info] switched currmeta from 674 -> 674 force true 2023-10-21T04:11:48.822+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.823+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.834+05:30 [Info] switched currmeta from 674 -> 675 force true 2023-10-21T04:11:48.875+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.875+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.881+05:30 [Info] switched currmeta from 675 -> 676 force true 2023-10-21T04:11:48.889+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.889+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.892+05:30 [Info] switched currmeta from 676 -> 677 force true 2023-10-21T04:11:48.907+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.908+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.913+05:30 [Info] switched currmeta from 677 -> 678 force true 2023-10-21T04:11:48.946+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:48.946+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:48.953+05:30 [Info] switched currmeta from 678 -> 679 force true 2023-10-21T04:11:49.002+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:49.003+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:49.008+05:30 [Info] switched currmeta from 679 -> 680 force true 2023-10-21T04:11:49.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:49.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:49.074+05:30 [Info] switched currmeta from 680 -> 681 force true 2023-10-21T04:11:49.292+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:49.293+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 99.94366197183263 InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:49.293+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:49.294+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:49.294+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:49.295+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:49.295+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:49.295+05:30 [Info] ShardRebalancer::computeProgress 0.7029999777613049 2023-10-21T04:11:49.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11675990026580235111:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:49.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11675990026580235111:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 {Index inst : [partitions]}: map[16983881020783103544:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:49.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11675990026580235111:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 {Index inst : [partitions]}: map[6393061446487756912:[5 6 7 8 1 2 3 4] 16983881020783103544:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:49.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 11675990026580235111:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 {Index inst : [partitions]}: map[6393061446487756912:[5 6 7 8 1 2 3 4] 16983881020783103544:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:49.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9010726441942672253:[2 3 4 5 6 7 8 1] 11675990026580235111:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 {Index inst : [partitions]}: map[6393061446487756912:[5 6 7 8 1 2 3 4] 16983881020783103544:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:49.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8899653570021014071:[0] 9010726441942672253:[2 3 4 5 6 7 8 1] 11675990026580235111:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 {Index inst : [partitions]}: map[3513301448177380059:[0] 6393061446487756912:[5 6 7 8 1 2 3 4] 16983881020783103544:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:49.895+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:49.896+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:49.899+05:30 [Info] switched currmeta from 681 -> 682 force true 2023-10-21T04:11:50.032+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:50.032+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:50.035+05:30 [Info] switched currmeta from 682 -> 683 force true 2023-10-21T04:11:50.058+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:50.058+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:50.063+05:30 [Info] switched currmeta from 683 -> 684 force true 2023-10-21T04:11:50.249+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 16] call. taskList: &{Rev:[0 0 0 0 0 0 0 17] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 2] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029999777613049 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:11:50.250+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 17] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:11:50.253+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 17] 2023-10-21T04:11:50.253+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 17] 2023-10-21T04:11:50.280+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:50.280+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:50.284+05:30 [Info] switched currmeta from 684 -> 685 force true 2023-10-21T04:11:50.388+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:50.388+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:50.391+05:30 [Info] switched currmeta from 685 -> 686 force true 2023-10-21T04:11:50.625+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:50.631+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:50.631+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:50.633+05:30 [Info] switched currmeta from 686 -> 686 force true 2023-10-21T04:11:50.925+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:50.925+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:50.929+05:30 [Info] switched currmeta from 686 -> 687 force true 2023-10-21T04:11:51.091+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:51.091+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:51.095+05:30 [Info] switched currmeta from 687 -> 688 force true 2023-10-21T04:11:51.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:51.517+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:51.525+05:30 [Info] switched currmeta from 688 -> 689 force true 2023-10-21T04:11:51.723+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:51.724+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:51.728+05:30 [Info] switched currmeta from 689 -> 690 force true 2023-10-21T04:11:51.934+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:51.934+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:51.939+05:30 [Info] switched currmeta from 690 -> 691 force true 2023-10-21T04:11:51.964+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:51.964+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:51.967+05:30 [Info] switched currmeta from 691 -> 692 force true 2023-10-21T04:11:52.047+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:52.047+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:52.051+05:30 [Info] switched currmeta from 692 -> 693 force true 2023-10-21T04:11:52.118+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:52.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:52.128+05:30 [Info] switched currmeta from 693 -> 694 force true 2023-10-21T04:11:52.155+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:52.155+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:52.159+05:30 [Info] switched currmeta from 694 -> 695 force true 2023-10-21T04:11:52.306+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:52.306+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:52.311+05:30 [Info] switched currmeta from 695 -> 696 force true 2023-10-21T04:11:53.040+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:53.040+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:53.045+05:30 [Info] switched currmeta from 696 -> 697 force true 2023-10-21T04:11:53.084+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:53.084+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:53.090+05:30 [Info] switched currmeta from 697 -> 698 force true 2023-10-21T04:11:53.132+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:53.132+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:53.136+05:30 [Info] switched currmeta from 698 -> 699 force true 2023-10-21T04:11:53.196+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:53.196+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:53.199+05:30 [Info] switched currmeta from 699 -> 700 force true 2023-10-21T04:11:53.256+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:53.256+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:53.262+05:30 [Info] switched currmeta from 700 -> 701 force true 2023-10-21T04:11:53.782+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.782+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 99.94366197183263 InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.782+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 99.94366197183263 InstId: 18051244886420988269 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 16398781184179534607 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:53.783+05:30 [Info] ShardRebalancer::computeProgress 0.7029999555226094 2023-10-21T04:11:54.158+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:54.158+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:54.166+05:30 [Info] switched currmeta from 701 -> 702 force true 2023-10-21T04:11:54.207+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:54.207+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:54.212+05:30 [Info] switched currmeta from 702 -> 703 force true 2023-10-21T04:11:54.233+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:54.234+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:54.237+05:30 [Info] switched currmeta from 703 -> 704 force true 2023-10-21T04:11:54.278+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:54.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:54.284+05:30 [Info] switched currmeta from 704 -> 705 force true 2023-10-21T04:11:54.321+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:54.321+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:54.324+05:30 [Info] switched currmeta from 705 -> 706 force true 2023-10-21T04:11:55.169+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.169+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.174+05:30 [Info] switched currmeta from 706 -> 707 force true 2023-10-21T04:11:55.207+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.207+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.214+05:30 [Info] switched currmeta from 707 -> 710 force true 2023-10-21T04:11:55.230+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.230+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.241+05:30 [Info] switched currmeta from 710 -> 710 force true 2023-10-21T04:11:55.251+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 18] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:11:55.251+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 17] call. taskList: &{Rev:[0 0 0 0 0 0 0 18] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 3] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029999555226094 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:11:55.254+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 18] 2023-10-21T04:11:55.255+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 18] 2023-10-21T04:11:55.283+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.283+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.293+05:30 [Info] switched currmeta from 710 -> 711 force true 2023-10-21T04:11:55.310+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.310+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.315+05:30 [Info] switched currmeta from 711 -> 712 force true 2023-10-21T04:11:55.329+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.329+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.333+05:30 [Info] switched currmeta from 712 -> 713 force true 2023-10-21T04:11:55.357+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.357+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.360+05:30 [Info] switched currmeta from 713 -> 714 force true 2023-10-21T04:11:55.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.378+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.380+05:30 [Info] switched currmeta from 714 -> 715 force true 2023-10-21T04:11:55.410+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.410+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.416+05:30 [Info] switched currmeta from 715 -> 716 force true 2023-10-21T04:11:55.510+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.510+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.516+05:30 [Info] switched currmeta from 716 -> 717 force true 2023-10-21T04:11:55.530+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.530+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.533+05:30 [Info] switched currmeta from 717 -> 719 force true 2023-10-21T04:11:55.539+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.539+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.541+05:30 [Info] switched currmeta from 719 -> 719 force true 2023-10-21T04:11:55.687+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:11:55.694+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.694+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.698+05:30 [Info] switched currmeta from 719 -> 719 force true 2023-10-21T04:11:55.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.899+05:30 [Info] switched currmeta from 719 -> 720 force true 2023-10-21T04:11:55.919+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.919+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.925+05:30 [Info] switched currmeta from 720 -> 722 force true 2023-10-21T04:11:55.931+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:55.931+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:55.934+05:30 [Info] switched currmeta from 722 -> 722 force true 2023-10-21T04:11:56.342+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:56.342+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:56.345+05:30 [Info] switched currmeta from 722 -> 723 force true 2023-10-21T04:11:56.370+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:56.377+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:56.385+05:30 [Info] switched currmeta from 723 -> 725 force true 2023-10-21T04:11:56.408+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:56.408+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:56.412+05:30 [Info] switched currmeta from 725 -> 725 force true 2023-10-21T04:11:56.911+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:56.912+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:56.916+05:30 [Info] switched currmeta from 725 -> 727 force true 2023-10-21T04:11:56.932+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:56.933+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:56.938+05:30 [Info] switched currmeta from 727 -> 728 force true 2023-10-21T04:11:56.947+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:56.948+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:56.951+05:30 [Info] switched currmeta from 728 -> 728 force true 2023-10-21T04:11:57.295+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.295+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.305+05:30 [Info] switched currmeta from 728 -> 730 force true 2023-10-21T04:11:57.314+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.314+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.319+05:30 [Info] switched currmeta from 730 -> 730 force true 2023-10-21T04:11:57.353+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.355+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.365+05:30 [Info] switched currmeta from 730 -> 732 force true 2023-10-21T04:11:57.374+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.374+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.379+05:30 [Info] switched currmeta from 732 -> 734 force true 2023-10-21T04:11:57.398+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.398+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.401+05:30 [Info] switched currmeta from 734 -> 734 force true 2023-10-21T04:11:57.440+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:57.440+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:57.440+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:57.440+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:57.440+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16398781184179534607:[0] 18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:57.440+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16398781184179534607:[0] 18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:57.440+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9246878583854294580:[0] 16398781184179534607:[0] 18051244886420988269:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:11:57.475+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.476+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.483+05:30 [Info] switched currmeta from 734 -> 735 force true 2023-10-21T04:11:57.557+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.557+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.562+05:30 [Info] switched currmeta from 735 -> 736 force true 2023-10-21T04:11:57.581+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.581+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.593+05:30 [Info] switched currmeta from 736 -> 737 force true 2023-10-21T04:11:57.612+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.612+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.624+05:30 [Info] switched currmeta from 737 -> 739 force true 2023-10-21T04:11:57.631+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.631+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.633+05:30 [Info] switched currmeta from 739 -> 739 force true 2023-10-21T04:11:57.682+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.682+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.690+05:30 [Info] switched currmeta from 739 -> 740 force true 2023-10-21T04:11:57.720+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:57.720+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:57.724+05:30 [Info] switched currmeta from 740 -> 741 force true 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 18051244886420988269 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 16398781184179534607 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 4894899676975150934 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.297+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:11:58.298+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:11:58.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.436+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4503814398859864266:[0] 8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.436+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4503814398859864266:[0] 8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.436+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4503814398859864266:[0] 8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.436+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4503814398859864266:[0] 8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.436+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4503814398859864266:[0] 8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.436+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.436+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4362827867398787577:[0] 4503814398859864266:[0] 8205593268191821683:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:11:58.586+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:58.586+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:58.591+05:30 [Info] switched currmeta from 741 -> 743 force true 2023-10-21T04:11:58.603+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:58.604+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:58.609+05:30 [Info] switched currmeta from 743 -> 743 force true 2023-10-21T04:11:58.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:58.629+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:58.633+05:30 [Info] switched currmeta from 743 -> 744 force true 2023-10-21T04:11:58.728+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:58.728+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:58.731+05:30 [Info] switched currmeta from 744 -> 745 force true 2023-10-21T04:11:58.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:58.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:58.759+05:30 [Info] switched currmeta from 745 -> 746 force true 2023-10-21T04:11:59.625+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:11:59.625+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:11:59.634+05:30 [Info] switched currmeta from 746 -> 747 force true 2023-10-21T04:12:00.185+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:00.185+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:00.188+05:30 [Info] switched currmeta from 747 -> 749 force true 2023-10-21T04:12:00.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:00.195+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:00.197+05:30 [Info] switched currmeta from 749 -> 749 force true 2023-10-21T04:12:00.250+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 19] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:00.250+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 18] call. taskList: &{Rev:[0 0 0 0 0 0 0 19] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 4] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:12:00.252+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 19] 2023-10-21T04:12:00.253+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 19] 2023-10-21T04:12:00.643+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:00.646+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:00.651+05:30 [Info] switched currmeta from 749 -> 750 force true 2023-10-21T04:12:00.671+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:00.675+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:00.687+05:30 [Info] switched currmeta from 750 -> 751 force true 2023-10-21T04:12:00.755+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:00.762+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:00.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:00.765+05:30 [Info] switched currmeta from 751 -> 751 force true 2023-10-21T04:12:00.947+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:00.947+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:00.953+05:30 [Info] switched currmeta from 751 -> 752 force true 2023-10-21T04:12:01.186+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:01.186+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:01.195+05:30 [Info] switched currmeta from 752 -> 753 force true 2023-10-21T04:12:01.702+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:01.702+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:01.710+05:30 [Info] switched currmeta from 753 -> 754 force true 2023-10-21T04:12:01.742+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:01.742+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:01.749+05:30 [Info] switched currmeta from 754 -> 755 force true 2023-10-21T04:12:01.796+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:01.797+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:01.799+05:30 [Info] switched currmeta from 755 -> 756 force true 2023-10-21T04:12:01.810+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:01.810+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:01.813+05:30 [Info] switched currmeta from 756 -> 757 force true 2023-10-21T04:12:02.450+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:02.450+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:02.452+05:30 [Info] switched currmeta from 757 -> 758 force true 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.453+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4894899676975150934:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:02.729+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:02.729+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:02.732+05:30 [Info] switched currmeta from 758 -> 759 force true 2023-10-21T04:12:02.786+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:02.786+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:02.789+05:30 [Info] switched currmeta from 759 -> 760 force true 2023-10-21T04:12:02.856+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:02.856+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:02.858+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.858+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.858+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.858+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.859+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 4894899676975150934 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.859+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.859+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.859+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.859+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.859+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 18051244886420988269 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.859+05:30 [Info] switched currmeta from 760 -> 761 force true 2023-10-21T04:12:02.859+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 99.94366197183263 InstId: 14736339311583908837 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.861+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.861+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 99.94366197183263 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.861+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.861+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 99.94366197183263 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.861+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 16398781184179534607 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:02.861+05:30 [Info] ShardRebalancer::computeProgress 0.7029999332839141 2023-10-21T04:12:02.871+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:02.872+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:02.874+05:30 [Info] switched currmeta from 761 -> 762 force true 2023-10-21T04:12:02.979+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:02.979+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:02.982+05:30 [Info] switched currmeta from 762 -> 763 force true 2023-10-21T04:12:03.068+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 237] aggregate storage info: map[bucket_%2:map[kv:5667008 index:31449631] bucket_1:map[kv:6392000 index:32755557]] 2023-10-21T04:12:03.779+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:03.779+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:03.781+05:30 [Info] switched currmeta from 763 -> 764 force true 2023-10-21T04:12:03.807+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:03.807+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:03.809+05:30 [Info] switched currmeta from 764 -> 765 force true 2023-10-21T04:12:03.819+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:03.819+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:03.822+05:30 [Info] switched currmeta from 765 -> 766 force true 2023-10-21T04:12:03.847+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:03.847+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:03.849+05:30 [Info] switched currmeta from 766 -> 767 force true 2023-10-21T04:12:03.898+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:03.898+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:03.902+05:30 [Info] switched currmeta from 767 -> 768 force true 2023-10-21T04:12:03.925+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:03.925+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:03.930+05:30 [Info] switched currmeta from 768 -> 769 force true 2023-10-21T04:12:03.981+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:03.982+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:04.001+05:30 [Info] switched currmeta from 769 -> 772 force true 2023-10-21T04:12:04.019+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:04.032+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:04.036+05:30 [Info] switched currmeta from 772 -> 772 force true 2023-10-21T04:12:04.171+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:04.171+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:04.175+05:30 [Info] switched currmeta from 772 -> 773 force true 2023-10-21T04:12:04.561+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:04.561+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:04.568+05:30 [Info] switched currmeta from 773 -> 775 force true 2023-10-21T04:12:04.587+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:04.587+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:04.613+05:30 [Info] switched currmeta from 775 -> 776 force true 2023-10-21T04:12:04.619+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:04.619+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:04.623+05:30 [Info] switched currmeta from 776 -> 776 force true 2023-10-21T04:12:05.037+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:05.037+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:05.045+05:30 [Info] switched currmeta from 776 -> 777 force true 2023-10-21T04:12:05.250+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 20] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:05.250+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 19] call. taskList: &{Rev:[0 0 0 0 0 0 0 20] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 5] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029999332839141 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:12:05.253+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 20] 2023-10-21T04:12:05.253+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 20] 2023-10-21T04:12:05.352+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:05.352+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:05.357+05:30 [Info] switched currmeta from 777 -> 778 force true 2023-10-21T04:12:05.583+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:05.583+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:05.595+05:30 [Info] switched currmeta from 778 -> 780 force true 2023-10-21T04:12:05.605+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:05.605+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:05.609+05:30 [Info] switched currmeta from 780 -> 781 force true 2023-10-21T04:12:05.617+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:05.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:05.620+05:30 [Info] switched currmeta from 781 -> 781 force true 2023-10-21T04:12:05.797+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:05.803+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:05.804+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:05.806+05:30 [Info] switched currmeta from 781 -> 781 force true 2023-10-21T04:12:05.993+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_1 Forcing Overdue Commit 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3292093078969249751 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4004220452657849790 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:05.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4004220452657849790 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.183µs SnapOpenDur 107.847µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16338040286194952893 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16338040286194952893 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.437µs SnapOpenDur 57.396µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17148287981272169168 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17148287981272169168 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.672µs SnapOpenDur 55.853µs 2023-10-21T04:12:05.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 5 Creating New Snapshot SnapshotInfo: count:133 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:133 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.121µs SnapOpenDur 74.892µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 8 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 6 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.294µs SnapOpenDur 49.442µs 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.774µs SnapOpenDur 36.387µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 1 Creating New Snapshot SnapshotInfo: count:135 committed:true 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 7 Creating New Snapshot SnapshotInfo: count:119 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:135 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.349µs SnapOpenDur 38.852µs 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:119 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.407µs SnapOpenDur 38.631µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 2 Creating New Snapshot SnapshotInfo: count:107 committed:true 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 8 Creating New Snapshot SnapshotInfo: count:122 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:107 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.766µs SnapOpenDur 40.139µs 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:122 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.397µs SnapOpenDur 38.768µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.355µs SnapOpenDur 41.669µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 2 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:120 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.671µs SnapOpenDur 46.864µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 3 Creating New Snapshot SnapshotInfo: count:139 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:139 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.877µs SnapOpenDur 37.13µs 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 4 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.564µs SnapOpenDur 39.175µs 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3810060733504350542 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:05.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3810060733504350542 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.714µs SnapOpenDur 969.617µs 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 SnapshotId 5 Creating recovery point ... 2023-10-21T04:12:05.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 2 Creating New Snapshot SnapshotInfo: count:113 committed:true 2023-10-21T04:12:05.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:113 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.041µs SnapOpenDur 56.909µs 2023-10-21T04:12:05.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:12:05.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.406µs SnapOpenDur 42.765µs 2023-10-21T04:12:05.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 4 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:124 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.92µs SnapOpenDur 33.455µs 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 5 Creating New Snapshot SnapshotInfo: count:111 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:111 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.37µs SnapOpenDur 36.232µs 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 6 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:140 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.456µs SnapOpenDur 49.644µs 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.962µs SnapOpenDur 39.154µs 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 8 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:12:05.996+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.144µs SnapOpenDur 37.002µs 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.185µs SnapOpenDur 46.924µs 2023-10-21T04:12:05.996+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13258659765240018889 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13258659765240018889 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:21 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.492µs SnapOpenDur 60.756µs 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11160011617777336020 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:05.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 3 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11160011617777336020 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.251µs SnapOpenDur 52.002µs 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.879µs SnapOpenDur 2.033856ms 2023-10-21T04:12:05.996+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.996+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.996+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 4 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:12:05.996+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:137 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.79µs SnapOpenDur 157.685µs 2023-10-21T04:12:05.996+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 5 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:12:05.996+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:131 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.373µs SnapOpenDur 38.85µs 2023-10-21T04:12:05.997+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 6 Creating New Snapshot SnapshotInfo: count:115 committed:true 2023-10-21T04:12:05.997+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:115 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.398µs SnapOpenDur 36.01µs 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 7 Creating New Snapshot SnapshotInfo: count:109 committed:true 2023-10-21T04:12:05.997+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:109 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.356µs SnapOpenDur 34.924µs 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3292093078969249751 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.833µs SnapOpenDur 75.716µs 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:05.997+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:06.018+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 SnapshotId 3 Created recovery point (took 23.953355ms) 2023-10-21T04:12:06.019+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.024+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:06.024+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:06.030+05:30 [Info] switched currmeta from 781 -> 784 force true 2023-10-21T04:12:06.036+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:06.037+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:06.042+05:30 [Info] switched currmeta from 784 -> 784 force true 2023-10-21T04:12:06.095+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:06.095+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:06.105+05:30 [Info] switched currmeta from 784 -> 785 force true 2023-10-21T04:12:06.110+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 SnapshotId 3 Created recovery point (took 115.375168ms) 2023-10-21T04:12:06.112+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.157+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:06.157+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:06.164+05:30 [Info] switched currmeta from 785 -> 786 force true 2023-10-21T04:12:06.207+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 SnapshotId 3 Created recovery point (took 211.11681ms) 2023-10-21T04:12:06.209+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:12:06.209+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:12:06.214+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:06.214+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:06.218+05:30 [Info] switched currmeta from 786 -> 788 force true 2023-10-21T04:12:06.224+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:06.224+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:06.231+05:30 [Info] switched currmeta from 788 -> 788 force true 2023-10-21T04:12:06.337+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 SnapshotId 4 Created recovery point (took 339.583073ms) 2023-10-21T04:12:06.337+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:12:06.344+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 SnapshotId 3 Created recovery point (took 347.176912ms) 2023-10-21T04:12:06.346+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.398+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 SnapshotId 3 Created recovery point (took 403.843803ms) 2023-10-21T04:12:06.399+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:12:06.400+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:12:06.415+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 SnapshotId 3 Created recovery point (took 419.971822ms) 2023-10-21T04:12:06.416+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:12:06.416+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:12:06.424+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 SnapshotId 3 Created recovery point (took 428.728955ms) 2023-10-21T04:12:06.424+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:12:06.425+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 13718648355797887482:[0] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 13718648355797887482:[0] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 13718648355797887482:[0] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.434+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3052172645459386575:[4 5 6 7 8 1 2 3] 13718648355797887482:[0] 14736339311583908837:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:06.438+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 SnapshotId 3 Created recovery point (took 443.089018ms) 2023-10-21T04:12:06.441+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:12:06.442+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:12:06.447+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 SnapshotId 3 Created recovery point (took 451.736951ms) 2023-10-21T04:12:06.448+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:12:06.448+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:12:06.454+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 SnapshotId 3 Created recovery point (took 459.154318ms) 2023-10-21T04:12:06.455+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:12:06.458+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:12:06.460+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 SnapshotId 3 Created recovery point (took 465.106027ms) 2023-10-21T04:12:06.461+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:12:06.461+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:12:06.466+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 SnapshotId 5 Created recovery point (took 470.76724ms) 2023-10-21T04:12:06.467+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.467+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.472+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 SnapshotId 3 Created recovery point (took 476.699804ms) 2023-10-21T04:12:06.473+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:12:06.474+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:12:06.478+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 SnapshotId 3 Created recovery point (took 482.42133ms) 2023-10-21T04:12:06.479+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:12:06.480+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:12:06.488+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 SnapshotId 3 Created recovery point (took 491.264681ms) 2023-10-21T04:12:06.490+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:12:06.490+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:12:06.501+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 SnapshotId 3 Created recovery point (took 504.438755ms) 2023-10-21T04:12:06.502+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:12:06.502+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:12:06.520+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 SnapshotId 3 Created recovery point (took 523.901827ms) 2023-10-21T04:12:06.521+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:12:06.522+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:12:06.539+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 SnapshotId 3 Created recovery point (took 542.337214ms) 2023-10-21T04:12:06.540+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:12:06.541+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:12:06.554+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 SnapshotId 3 Created recovery point (took 557.375294ms) 2023-10-21T04:12:06.555+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:12:06.555+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:12:06.564+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 SnapshotId 3 Created recovery point (took 567.734288ms) 2023-10-21T04:12:06.565+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:12:06.566+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:12:06.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 SnapshotId 3 Created recovery point (took 579.585028ms) 2023-10-21T04:12:06.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:12:06.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:12:06.582+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 SnapshotId 3 Created recovery point (took 585.565103ms) 2023-10-21T04:12:06.583+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:12:06.583+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:12:06.593+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 SnapshotId 3 Created recovery point (took 596.033933ms) 2023-10-21T04:12:06.594+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:12:06.594+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:12:06.601+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 SnapshotId 3 Created recovery point (took 604.35405ms) 2023-10-21T04:12:06.602+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:12:06.603+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:12:06.609+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 SnapshotId 3 Created recovery point (took 612.339964ms) 2023-10-21T04:12:06.610+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:12:06.611+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:12:06.617+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 SnapshotId 3 Created recovery point (took 619.679192ms) 2023-10-21T04:12:06.618+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:12:06.619+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:12:06.625+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 SnapshotId 3 Created recovery point (took 628.172012ms) 2023-10-21T04:12:06.626+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:12:06.627+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:12:06.637+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 SnapshotId 3 Created recovery point (took 643.644298ms) 2023-10-21T04:12:06.638+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.639+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.652+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 SnapshotId 3 Created recovery point (took 658.557369ms) 2023-10-21T04:12:06.654+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.654+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:06.668+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 SnapshotId 3 Created recovery point (took 671.261306ms) 2023-10-21T04:12:06.670+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:12:06.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:12:07.135+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:07.135+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:07.140+05:30 [Info] switched currmeta from 788 -> 790 force true 2023-10-21T04:12:07.150+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:07.150+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:07.153+05:30 [Info] switched currmeta from 790 -> 790 force true 2023-10-21T04:12:07.229+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:07.229+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:07.236+05:30 [Info] switched currmeta from 790 -> 791 force true 2023-10-21T04:12:07.265+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:07.265+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:07.276+05:30 [Info] switched currmeta from 791 -> 792 force true 2023-10-21T04:12:07.304+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:07.304+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:07.314+05:30 [Info] switched currmeta from 792 -> 793 force true 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_secondary Progress: 100 InstId: 8899653570021014071 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 9246878583854294580 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 18051244886420988269 RealInstId: 18051244886420988269 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 14736339311583908837 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:idx_partitioned Progress: 100 InstId: 9010726441942672253 RealInstId: 9010726441942672253 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 3052172645459386575 RealInstId: 3052172645459386575 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c2%:#primary Progress: 100 InstId: 11675990026580235111 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 99.94366197183263 InstId: 360999437388707481 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:#primary Progress: 100 InstId: 13718648355797887482 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 16398781184179534607 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_partitioned Progress: 100 InstId: 6393061446487756912 RealInstId: 6393061446487756912 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:idx_secondary Progress: 100 InstId: 3513301448177380059 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 13420534524606213282 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 5299824419646640495 RealInstId: 5299824419646640495 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.334+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 8205593268191821683 RealInstId: 8205593268191821683 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.335+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 4362827867398787577 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.335+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 4894899676975150934 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.335+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c2%:#primary Progress: 100 InstId: 16983881020783103544 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.335+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 4503814398859864266 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9002 2023-10-21T04:12:07.335+05:30 [Info] ShardRebalancer::computeProgress 0.7029999777613049 2023-10-21T04:12:08.325+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:08.327+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:08.333+05:30 [Info] switched currmeta from 793 -> 795 force true 2023-10-21T04:12:08.346+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:08.346+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:08.353+05:30 [Info] switched currmeta from 795 -> 795 force true 2023-10-21T04:12:08.392+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:08.392+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:08.395+05:30 [Info] switched currmeta from 795 -> 796 force true 2023-10-21T04:12:08.546+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:08.546+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:08.549+05:30 [Info] switched currmeta from 796 -> 797 force true 2023-10-21T04:12:08.558+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:08.558+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:08.563+05:30 [Info] switched currmeta from 797 -> 798 force true 2023-10-21T04:12:08.609+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:08.609+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:08.622+05:30 [Info] switched currmeta from 798 -> 800 force true 2023-10-21T04:12:08.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:08.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:08.638+05:30 [Info] switched currmeta from 800 -> 800 force true 2023-10-21T04:12:09.005+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:09.005+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:09.009+05:30 [Info] switched currmeta from 800 -> 801 force true 2023-10-21T04:12:09.020+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:09.020+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:09.025+05:30 [Info] switched currmeta from 801 -> 802 force true 2023-10-21T04:12:09.048+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:09.048+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:09.053+05:30 [Info] switched currmeta from 802 -> 803 force true 2023-10-21T04:12:09.575+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:09.575+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:09.582+05:30 [Info] switched currmeta from 803 -> 804 force true 2023-10-21T04:12:09.996+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:09.996+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:10.001+05:30 [Info] switched currmeta from 804 -> 805 force true 2023-10-21T04:12:10.056+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:10.058+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:10.062+05:30 [Info] switched currmeta from 805 -> 806 force true 2023-10-21T04:12:10.080+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:10.080+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:10.086+05:30 [Info] switched currmeta from 806 -> 807 force true 2023-10-21T04:12:10.161+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:10.161+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:10.179+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:10.180+05:30 [Info] switched currmeta from 807 -> 808 force true 2023-10-21T04:12:10.191+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf7:10:2b:8a:50:ca:a0:55 MasterId: SourceId: DestId: RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokenff:15:2d:39:84:e6:37:31 SiblingTokenId: Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:10.250+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:40474 closed 2023-10-21T04:12:10.251+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 20] call. taskList: &{Rev:[0 0 0 0 0 0 0 21] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 6] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029999777613049 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:12:10.252+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 21] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:10.255+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 21] 2023-10-21T04:12:10.255+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 21] 2023-10-21T04:12:10.295+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43610 closed 2023-10-21T04:12:10.309+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43616 closed 2023-10-21T04:12:10.342+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43626 closed 2023-10-21T04:12:10.425+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43632 closed 2023-10-21T04:12:10.432+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43636 closed 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.435+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[360999437388707481:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:12:10.437+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43638 closed 2023-10-21T04:12:10.491+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43650 closed 2023-10-21T04:12:10.622+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:10.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:10.624+05:30 [Info] switched currmeta from 808 -> 811 force true 2023-10-21T04:12:10.638+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:10.638+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:10.649+05:30 [Info] switched currmeta from 811 -> 811 force true 2023-10-21T04:12:10.655+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:10.661+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken88:fb:25:5d:ca:b7:c6:8d MasterId: SourceId: DestId: RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken47:ec:19:47:fd:b7:cd:73 SiblingTokenId: Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:10.680+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43658 closed 2023-10-21T04:12:10.686+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43668 closed 2023-10-21T04:12:10.703+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43672 closed 2023-10-21T04:12:10.767+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43678 closed 2023-10-21T04:12:10.773+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43682 closed 2023-10-21T04:12:10.777+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43684 closed 2023-10-21T04:12:10.839+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43688 closed 2023-10-21T04:12:10.857+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:10.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:10.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:10.865+05:30 [Info] switched currmeta from 811 -> 811 force true 2023-10-21T04:12:10.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43690 closed 2023-10-21T04:12:10.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43694 closed 2023-10-21T04:12:10.910+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43696 closed 2023-10-21T04:12:10.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0] 13420534524606213282:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0] 13420534524606213282:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0] 13420534524606213282:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0] 13420534524606213282:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0] 13420534524606213282:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0] 13420534524606213282:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.939+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5299824419646640495:[5 6 7 8 1 2 3 4] 6796752657857722550:[0] 13420534524606213282:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:12:10.972+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43708 closed 2023-10-21T04:12:11.081+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43710 closed 2023-10-21T04:12:11.086+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43714 closed 2023-10-21T04:12:11.097+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43716 closed 2023-10-21T04:12:11.176+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43718 closed 2023-10-21T04:12:11.244+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.244+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.262+05:30 [Info] switched currmeta from 811 -> 812 force true 2023-10-21T04:12:11.297+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.297+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.301+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43736 closed 2023-10-21T04:12:11.305+05:30 [Info] switched currmeta from 812 -> 813 force true 2023-10-21T04:12:11.328+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.329+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.331+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43748 closed 2023-10-21T04:12:11.336+05:30 [Info] switched currmeta from 813 -> 814 force true 2023-10-21T04:12:11.343+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.344+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.348+05:30 [Info] switched currmeta from 814 -> 814 force true 2023-10-21T04:12:11.385+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43758 closed 2023-10-21T04:12:11.393+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43764 closed 2023-10-21T04:12:11.399+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.400+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.403+05:30 [Info] switched currmeta from 814 -> 815 force true 2023-10-21T04:12:11.422+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.422+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.424+05:30 [Info] switched currmeta from 815 -> 815 force true 2023-10-21T04:12:11.435+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.435+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.438+05:30 [Info] switched currmeta from 815 -> 816 force true 2023-10-21T04:12:11.461+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43768 closed 2023-10-21T04:12:11.465+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43774 closed 2023-10-21T04:12:11.498+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.518+05:30 [Info] switched currmeta from 816 -> 816 force true 2023-10-21T04:12:11.531+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.531+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.534+05:30 [Info] switched currmeta from 816 -> 817 force true 2023-10-21T04:12:11.541+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43776 closed 2023-10-21T04:12:11.557+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43790 closed 2023-10-21T04:12:11.577+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.578+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.586+05:30 [Info] switched currmeta from 817 -> 818 force true 2023-10-21T04:12:11.674+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.674+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.677+05:30 [Info] switched currmeta from 818 -> 818 force true 2023-10-21T04:12:11.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.698+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.704+05:30 [Info] switched currmeta from 818 -> 819 force true 2023-10-21T04:12:11.740+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43830 closed 2023-10-21T04:12:11.742+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.742+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.745+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43800 closed 2023-10-21T04:12:11.745+05:30 [Info] switched currmeta from 819 -> 820 force true 2023-10-21T04:12:11.785+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.785+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.787+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43840 closed 2023-10-21T04:12:11.799+05:30 [Info] switched currmeta from 820 -> 821 force true 2023-10-21T04:12:11.811+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43844 closed 2023-10-21T04:12:11.815+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.815+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.836+05:30 [Info] switched currmeta from 821 -> 822 force true 2023-10-21T04:12:11.929+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.929+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.946+05:30 [Info] switched currmeta from 822 -> 826 force true 2023-10-21T04:12:11.959+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43848 closed 2023-10-21T04:12:11.964+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:11.964+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:11.967+05:30 [Info] ShardRebalancer::computeProgress 1 2023-10-21T04:12:11.988+05:30 [Info] switched currmeta from 826 -> 827 force true 2023-10-21T04:12:12.009+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43860 closed 2023-10-21T04:12:12.020+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.020+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.023+05:30 [Info] switched currmeta from 827 -> 828 force true 2023-10-21T04:12:12.027+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.027+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.030+05:30 [Info] switched currmeta from 828 -> 828 force true 2023-10-21T04:12:12.063+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43862 closed 2023-10-21T04:12:12.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.074+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43868 closed 2023-10-21T04:12:12.074+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43866 closed 2023-10-21T04:12:12.080+05:30 [Info] switched currmeta from 828 -> 828 force true 2023-10-21T04:12:12.109+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.110+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.113+05:30 [Info] switched currmeta from 828 -> 829 force true 2023-10-21T04:12:12.125+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43874 closed 2023-10-21T04:12:12.144+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43878 closed 2023-10-21T04:12:12.160+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.160+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.167+05:30 [Info] switched currmeta from 829 -> 830 force true 2023-10-21T04:12:12.171+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43880 closed 2023-10-21T04:12:12.193+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.193+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.208+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43886 closed 2023-10-21T04:12:12.233+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43894 closed 2023-10-21T04:12:12.233+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43892 closed 2023-10-21T04:12:12.240+05:30 [Info] switched currmeta from 830 -> 832 force true 2023-10-21T04:12:12.249+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.249+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.256+05:30 [Info] switched currmeta from 832 -> 832 force true 2023-10-21T04:12:12.275+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43896 closed 2023-10-21T04:12:12.291+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43900 closed 2023-10-21T04:12:12.322+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.326+05:30 [Info] switched currmeta from 832 -> 832 force true 2023-10-21T04:12:12.346+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.346+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.349+05:30 [Info] switched currmeta from 832 -> 833 force true 2023-10-21T04:12:12.351+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43908 closed 2023-10-21T04:12:12.357+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43910 closed 2023-10-21T04:12:12.372+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.372+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.376+05:30 [Info] switched currmeta from 833 -> 834 force true 2023-10-21T04:12:12.467+05:30 [Info] janitor: running cleanup. 2023-10-21T04:12:12.467+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[14889401051926000176 4860549750398470299]} 2023-10-21T04:12:12.472+05:30 [Info] memstats {"Alloc":241387216, "TotalAlloc":3878106656, "Sys":388813144, "Lookups":0, "Mallocs":44301556,"Frees":43462994, "HeapAlloc":241387216, "HeapSys":350027776, "HeapIdle":94388224, "HeapInuse":255639552,"HeapReleased":1294336, "HeapObjects":838562,"MSpanInuse":1981440, "MSpanSys": 3100800, "StackInuse": 14876672,"GCSys":17277936, "LastGC":1697841671005642195,"PauseTotalNs":48134248, "PauseNs":[266860, 318782, 371196, 334976, 691601, 136207, 172802, 145410, 406939, 136250, 188850, 173429, 93013, 3648648, 101608], "NumGC":32} 2023-10-21T04:12:12.477+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.477+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.478+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:12:12.518+05:30 [Info] switched currmeta from 834 -> 836 force true 2023-10-21T04:12:12.803+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.803+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.813+05:30 [Info] switched currmeta from 836 -> 839 force true 2023-10-21T04:12:12.960+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:12.960+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:12.965+05:30 [Info] switched currmeta from 839 -> 840 force true 2023-10-21T04:12:13.158+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.159+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.167+05:30 [Info] switched currmeta from 840 -> 844 force true 2023-10-21T04:12:13.179+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.179+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.184+05:30 [Info] switched currmeta from 844 -> 844 force true 2023-10-21T04:12:13.225+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.225+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.230+05:30 [Info] switched currmeta from 844 -> 844 force true 2023-10-21T04:12:13.241+05:30 [Info] connected with 3 indexers 2023-10-21T04:12:13.241+05:30 [Info] client stats current counts: current: 76, not current: 0 2023-10-21T04:12:13.278+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.283+05:30 [Info] switched currmeta from 844 -> 845 force true 2023-10-21T04:12:13.308+05:30 [Info] num concurrent scans {0} 2023-10-21T04:12:13.308+05:30 [Info] average scan response {0 ms} 2023-10-21T04:12:13.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.363+05:30 [Info] switched currmeta from 845 -> 846 force true 2023-10-21T04:12:13.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9940540994472487, throttleDelayMs (new, old, change): (4, 0, 4) 2023-10-21T04:12:13.454+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:12:13.454+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:12:13.476+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.476+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.479+05:30 [Info] switched currmeta from 846 -> 846 force true 2023-10-21T04:12:13.527+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.527+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.535+05:30 [Info] switched currmeta from 846 -> 847 force true 2023-10-21T04:12:13.590+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.591+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.602+05:30 [Info] switched currmeta from 847 -> 848 force true 2023-10-21T04:12:13.764+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.764+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.774+05:30 [Info] switched currmeta from 848 -> 848 force true 2023-10-21T04:12:13.784+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.784+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.794+05:30 [Info] switched currmeta from 848 -> 849 force true 2023-10-21T04:12:13.820+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.820+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.826+05:30 [Info] switched currmeta from 849 -> 850 force true 2023-10-21T04:12:13.871+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.872+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.878+05:30 [Info] switched currmeta from 850 -> 851 force true 2023-10-21T04:12:13.887+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.889+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.894+05:30 [Info] switched currmeta from 851 -> 851 force true 2023-10-21T04:12:13.911+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.912+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.931+05:30 [Info] switched currmeta from 851 -> 852 force true 2023-10-21T04:12:13.959+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.959+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.965+05:30 [Info] switched currmeta from 852 -> 853 force true 2023-10-21T04:12:13.980+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:13.980+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:13.986+05:30 [Info] switched currmeta from 853 -> 853 force true 2023-10-21T04:12:14.029+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.030+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.040+05:30 [Info] switched currmeta from 853 -> 854 force true 2023-10-21T04:12:14.049+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.049+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.053+05:30 [Info] switched currmeta from 854 -> 854 force true 2023-10-21T04:12:14.063+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.063+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.068+05:30 [Info] switched currmeta from 854 -> 855 force true 2023-10-21T04:12:14.096+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.096+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.102+05:30 [Info] switched currmeta from 855 -> 856 force true 2023-10-21T04:12:14.143+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.143+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.149+05:30 [Info] switched currmeta from 856 -> 856 force true 2023-10-21T04:12:14.160+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.161+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.169+05:30 [Info] switched currmeta from 856 -> 857 force true 2023-10-21T04:12:14.204+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.204+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.211+05:30 [Info] switched currmeta from 857 -> 858 force true 2023-10-21T04:12:14.252+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:12:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17729593977321451597 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17729593977321451597 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.801µs SnapOpenDur 76.13µs 2023-10-21T04:12:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14607307599259773249 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14607307599259773249 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.049µs SnapOpenDur 48.383µs 2023-10-21T04:12:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 4 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:12:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:140 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.155µs SnapOpenDur 83.185µs 2023-10-21T04:12:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 5 Creating New Snapshot SnapshotInfo: count:116 committed:true 2023-10-21T04:12:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:116 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.086µs SnapOpenDur 33.16µs 2023-10-21T04:12:14.252+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:12:14.252+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.316µs SnapOpenDur 36.269µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.09µs SnapOpenDur 36.06µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 8 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:124 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.087µs SnapOpenDur 34.222µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 1 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.145µs SnapOpenDur 44.495µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 2 Creating New Snapshot SnapshotInfo: count:110 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:110 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.257µs SnapOpenDur 34.578µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2483169159760864439 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2483169159760864439 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.797µs SnapOpenDur 92.087µs 2023-10-21T04:12:14.253+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17128008689111072444 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17128008689111072444 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.196µs SnapOpenDur 50.271µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11400459767051699632 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11400459767051699632 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.08µs SnapOpenDur 46.125µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.139µs SnapOpenDur 44.531µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 4 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:12:14.253+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.466µs SnapOpenDur 37.326µs 2023-10-21T04:12:14.253+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 5 Creating New Snapshot SnapshotInfo: count:117 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:117 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.458µs SnapOpenDur 40.374µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 6 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:137 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.625µs SnapOpenDur 36.144µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 7 Creating New Snapshot SnapshotInfo: count:102 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:102 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.565µs SnapOpenDur 41.596µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 8 Creating New Snapshot SnapshotInfo: count:148 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:148 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.653µs SnapOpenDur 37.885µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 1 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.13µs SnapOpenDur 36.167µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.513µs SnapOpenDur 33.139µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 18361169415695152662 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 18361169415695152662 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.903µs SnapOpenDur 65.467µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14780363202971367008 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14780363202971367008 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:21 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.064µs SnapOpenDur 44.887µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 5 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.187µs SnapOpenDur 46.017µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 6 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:106 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.294µs SnapOpenDur 36.171µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 7 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.305µs SnapOpenDur 36.842µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 8 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:12:14.254+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.231µs SnapOpenDur 39.723µs 2023-10-21T04:12:14.254+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 1 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:12:14.255+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:120 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.438µs SnapOpenDur 35.265µs 2023-10-21T04:12:14.255+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 2 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:12:14.255+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.043µs SnapOpenDur 34.774µs 2023-10-21T04:12:14.255+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:12:14.255+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.467µs SnapOpenDur 37.118µs 2023-10-21T04:12:14.255+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 4 Creating New Snapshot SnapshotInfo: count:130 committed:true 2023-10-21T04:12:14.255+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:130 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.503µs SnapOpenDur 37.009µs 2023-10-21T04:12:14.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:12:14.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.257+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 3 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:12:14.257+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 5.325µs SnapOpenDur 3.773803ms 2023-10-21T04:12:14.257+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.260+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.262+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.262+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.262+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.262+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.262+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:12:14.291+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.291+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.294+05:30 [Info] switched currmeta from 858 -> 858 force true 2023-10-21T04:12:14.315+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.319+05:30 [Info] switched currmeta from 858 -> 859 force true 2023-10-21T04:12:14.327+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 SnapshotId 3 Created recovery point (took 72.150073ms) 2023-10-21T04:12:14.328+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.345+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.345+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.348+05:30 [Info] switched currmeta from 859 -> 860 force true 2023-10-21T04:12:14.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9952589422042002, throttleDelayMs (new, old, change): (9, 4, 5) 2023-10-21T04:12:14.457+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 SnapshotId 4 Created recovery point (took 200.986124ms) 2023-10-21T04:12:14.458+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:12:14.515+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.515+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.518+05:30 [Info] switched currmeta from 860 -> 860 force true 2023-10-21T04:12:14.523+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.523+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.527+05:30 [Info] switched currmeta from 860 -> 861 force true 2023-10-21T04:12:14.547+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.547+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.550+05:30 [Info] switched currmeta from 861 -> 862 force true 2023-10-21T04:12:14.575+05:30 [Info] Indexer::ReadMemstats Time Taken 446.34µs 2023-10-21T04:12:14.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 SnapshotId 3 Created recovery point (took 317.432669ms) 2023-10-21T04:12:14.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:12:14.580+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:12:14.705+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.705+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.735+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 SnapshotId 3 Created recovery point (took 473.227871ms) 2023-10-21T04:12:14.737+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.744+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 SnapshotId 3 Created recovery point (took 481.695643ms) 2023-10-21T04:12:14.747+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.759+05:30 [Info] switched currmeta from 862 -> 866 force true 2023-10-21T04:12:14.759+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 SnapshotId 3 Created recovery point (took 505.617978ms) 2023-10-21T04:12:14.760+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:12:14.760+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:12:14.766+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 SnapshotId 3 Created recovery point (took 511.523896ms) 2023-10-21T04:12:14.768+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.769+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.773+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 SnapshotId 3 Created recovery point (took 518.142555ms) 2023-10-21T04:12:14.775+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.775+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.791+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 SnapshotId 3 Created recovery point (took 536.230642ms) 2023-10-21T04:12:14.794+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.797+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.799+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 SnapshotId 3 Created recovery point (took 543.205557ms) 2023-10-21T04:12:14.800+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:12:14.800+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:12:14.804+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:12:14.806+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 SnapshotId 3 Created recovery point (took 550.265909ms) 2023-10-21T04:12:14.807+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:12:14.811+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:12:14.813+05:30 [Info] switched currmeta from 866 -> 866 force true 2023-10-21T04:12:14.818+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 SnapshotId 3 Created recovery point (took 562.460765ms) 2023-10-21T04:12:14.825+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.825+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.828+05:30 [Info] switched currmeta from 866 -> 867 force true 2023-10-21T04:12:14.833+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:12:14.836+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 SnapshotId 3 Created recovery point (took 580.796648ms) 2023-10-21T04:12:14.840+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:12:14.840+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:12:14.840+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:12:14.843+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 SnapshotId 3 Created recovery point (took 586.895457ms) 2023-10-21T04:12:14.844+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:12:14.844+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:12:14.844+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:12:14.858+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 SnapshotId 3 Created recovery point (took 602.185353ms) 2023-10-21T04:12:14.859+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:12:14.860+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:12:14.866+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 SnapshotId 3 Created recovery point (took 610.232165ms) 2023-10-21T04:12:14.871+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:12:14.871+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:12:14.875+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.875+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.875+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 SnapshotId 3 Created recovery point (took 619.128173ms) 2023-10-21T04:12:14.876+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.877+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:12:14.877+05:30 [Info] switched currmeta from 867 -> 868 force true 2023-10-21T04:12:14.891+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 SnapshotId 3 Created recovery point (took 634.938307ms) 2023-10-21T04:12:14.892+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:12:14.893+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:12:14.901+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.901+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.905+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 SnapshotId 3 Created recovery point (took 644.642979ms) 2023-10-21T04:12:14.906+05:30 [Info] switched currmeta from 868 -> 868 force true 2023-10-21T04:12:14.906+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:12:14.907+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:12:14.917+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 SnapshotId 3 Created recovery point (took 656.851292ms) 2023-10-21T04:12:14.919+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:12:14.919+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:12:14.924+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.924+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.925+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 SnapshotId 3 Created recovery point (took 664.711196ms) 2023-10-21T04:12:14.927+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:12:14.927+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:12:14.936+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 SnapshotId 3 Created recovery point (took 675.366494ms) 2023-10-21T04:12:14.937+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:12:14.938+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:12:14.938+05:30 [Info] switched currmeta from 868 -> 869 force true 2023-10-21T04:12:14.947+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 SnapshotId 3 Created recovery point (took 686.057408ms) 2023-10-21T04:12:14.948+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:12:14.949+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:12:14.963+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.963+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:14.974+05:30 [Info] switched currmeta from 869 -> 870 force true 2023-10-21T04:12:14.989+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:14.989+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.002+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 SnapshotId 3 Created recovery point (took 741.171631ms) 2023-10-21T04:12:15.004+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:12:15.004+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:12:15.009+05:30 [Info] switched currmeta from 870 -> 872 force true 2023-10-21T04:12:15.014+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 SnapshotId 3 Created recovery point (took 752.652572ms) 2023-10-21T04:12:15.015+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:12:15.015+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:12:15.027+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 SnapshotId 3 Created recovery point (took 765.722628ms) 2023-10-21T04:12:15.028+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:12:15.029+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:12:15.040+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 SnapshotId 3 Created recovery point (took 778.671509ms) 2023-10-21T04:12:15.043+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:12:15.043+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:12:15.076+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 SnapshotId 3 Created recovery point (took 814.669773ms) 2023-10-21T04:12:15.079+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.079+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.079+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:12:15.079+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:12:15.084+05:30 [Info] switched currmeta from 872 -> 874 force true 2023-10-21T04:12:15.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.121+05:30 [Info] switched currmeta from 874 -> 875 force true 2023-10-21T04:12:15.133+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.133+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.139+05:30 [Info] switched currmeta from 875 -> 875 force true 2023-10-21T04:12:15.141+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 SnapshotId 3 Created recovery point (took 879.809866ms) 2023-10-21T04:12:15.145+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:12:15.147+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:12:15.160+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 SnapshotId 3 Created recovery point (took 898.543929ms) 2023-10-21T04:12:15.161+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:12:15.162+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:12:15.163+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.166+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.171+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 SnapshotId 3 Created recovery point (took 909.349277ms) 2023-10-21T04:12:15.172+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:12:15.173+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:12:15.195+05:30 [Info] switched currmeta from 875 -> 877 force true 2023-10-21T04:12:15.220+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.222+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.234+05:30 [Info] switched currmeta from 877 -> 879 force true 2023-10-21T04:12:15.242+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.251+05:30 [Info] switched currmeta from 879 -> 879 force true 2023-10-21T04:12:15.252+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 22] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:15.254+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 22] 2023-10-21T04:12:15.252+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 21] call. taskList: &{Rev:[0 0 0 0 0 0 0 22] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 7] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.99 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:12:15.259+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 22] 2023-10-21T04:12:15.327+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.327+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.330+05:30 [Info] switched currmeta from 879 -> 880 force true 2023-10-21T04:12:15.342+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.342+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.357+05:30 [Info] switched currmeta from 880 -> 881 force true 2023-10-21T04:12:15.387+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.387+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.393+05:30 [Info] switched currmeta from 881 -> 882 force true 2023-10-21T04:12:15.402+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.402+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.407+05:30 [Info] switched currmeta from 882 -> 883 force true 2023-10-21T04:12:15.416+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.419+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.450+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.994451566531183, throttleDelayMs (new, old, change): (10, 9, 1) 2023-10-21T04:12:15.466+05:30 [Info] switched currmeta from 883 -> 884 force true 2023-10-21T04:12:15.479+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.481+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.488+05:30 [Info] switched currmeta from 884 -> 886 force true 2023-10-21T04:12:15.502+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.502+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.508+05:30 [Info] switched currmeta from 886 -> 886 force true 2023-10-21T04:12:15.546+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.546+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.565+05:30 [Info] switched currmeta from 886 -> 887 force true 2023-10-21T04:12:15.576+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.577+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.582+05:30 [Info] switched currmeta from 887 -> 887 force true 2023-10-21T04:12:15.682+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:15.763+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:ec:19:47:fd:b7:cd:73 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5653766467314045229 RealInstId: 18089525033132576654 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17533500296092519815 RealInstId: 6393061446487756912 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14991206072146508370 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 3513301448177380059 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13420534524606213282 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6492262833116702924 RealInstId: 5299824419646640495 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4138705102427425280 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13207161782265087251 RealInstId: 9358717944830195392 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 10455069697192970620 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 10819337149286705062 RealInstId: 8205593268191821683 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 5179062340836596587 RealInstId: 7291286939424384589 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 4362827867398787577 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 11158580445292211864 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 9134754361302452675 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4894899676975150934 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 16983881020783103544 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 11484519016652644658 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 6796752657857722550 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 4503814398859864266 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:15.795+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[85232c9ba3e4451b95e9e6fc8337e027:0 f2b29e6bf10e9349c06775f41b68f8ff:-1], batchSize: 2 2023-10-21T04:12:15.797+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardTokenff:15:2d:39:84:e6:37:31 is in state: ShardTokenReady 2023-10-21T04:12:15.802+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken47:ec:19:47:fd:b7:cd:73 2023-10-21T04:12:15.957+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:15.966+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:15.966+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:12:15.969+05:30 [Info] switched currmeta from 887 -> 887 force true 2023-10-21T04:12:16.276+05:30 [Info] ShardRebalancer::computeProgress 1 2023-10-21T04:12:16.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8246761459240426, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:12:16.557+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:16.588+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:15:2d:39:84:e6:37:31 MasterId: 31fd475f22d8ee0da906c31968c91305 SourceId: 85232c9ba3e4451b95e9e6fc8337e027 (127.0.0.1:9004) DestId: f2b29e6bf10e9349c06775f41b68f8ff (127.0.0.1:9002) RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6213797968262183765 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 8899653570021014071 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 9246878583854294580 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 17161083103924027797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 7170604883654040991 RealInstId: 18051244886420988269 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 14736339311583908837 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6137244366124413283 RealInstId: 9010726441942672253 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6054633421341911295 RealInstId: 10574086758347311295 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 16060708638314595734 RealInstId: 16044079815412989935 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 18184041269291378080 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 13518927892577684793 RealInstId: 1955535609704140162 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6452766258658064766 RealInstId: 3052172645459386575 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11675990026580235111 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 360999437388707481 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] InstId: 1938940178723796232 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [2] InstId: 13718648355797887482 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 16398781184179534607 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] InstId: 7287869093983910291 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [2] InstId: 15283305021611177781 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:16.594+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[85232c9ba3e4451b95e9e6fc8337e027:0 f2b29e6bf10e9349c06775f41b68f8ff:-2], batchSize: 2 2023-10-21T04:12:16.594+05:30 [Info] ShardRebalancer::processShardTransferTokenAsMaster No Tokens Found. Mark Done. 2023-10-21T04:12:16.595+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardTokenff:15:2d:39:84:e6:37:31 2023-10-21T04:12:16.595+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 22] call. taskList: &{Rev:[0 0 0 0 0 0 0 23] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a03babfad26a7c7d717571a6ae235db1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]} {Rev:[0 0 0 0 0 0 0 8] ID:rebalance/a03babfad26a7c7d717571a6ae235db1 Type:task-rebalance Status:task-running IsCancelable:true Progress:1 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a03babfad26a7c7d717571a6ae235db1]}]} 2023-10-21T04:12:16.595+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 23] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:16.595+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:12:16.603+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 23] 2023-10-21T04:12:16.603+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 23] 2023-10-21T04:12:16.603+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:12:16.603+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:12:16.603+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress done received. progress collector stopped 2023-10-21T04:12:16.603+05:30 [Info] ShardRebalancer::updateProgress Done Received 2023-10-21T04:12:16.603+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:12:16.604+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:12:16.613+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {31fd475f22d8ee0da906c31968c91305 a03babfad26a7c7d717571a6ae235db1 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:12:16.634+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:12:16.634+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken88:fb:25:5d:ca:b7:c6:8d MasterId: SourceId: DestId: RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken47:ec:19:47:fd:b7:cd:73 SiblingTokenId: Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:16.635+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenf7:10:2b:8a:50:ca:a0:55 MasterId: SourceId: DestId: RebalId: a03babfad26a7c7d717571a6ae235db1 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokenff:15:2d:39:84:e6:37:31 SiblingTokenId: Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:12:16.662+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:12:16.662+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:12:16.662+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.744µs 2023-10-21T04:12:16.662+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:12:16.662+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:12:16.662+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:12:16.666+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:12:16.666+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:12:16.666+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:12:16.666+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:12:16.666+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 29.227µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:12:16.672+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:12:16.672+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:12:16.672+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster true, forceUnbalanced false, err: 2023-10-21T04:12:16.672+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 23] call. taskList: &{Rev:[0 0 0 0 0 0 0 24] Tasks:[]} 2023-10-21T04:12:16.672+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 24] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:16.672+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:12:16.684+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 24] 2023-10-21T04:12:16.685+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 24] 2023-10-21T04:12:16.685+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:12:16.685+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 24] Tasks:[]} 2023-10-21T04:12:16.686+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:12:16.686+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 24] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:16.687+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 24] 2023-10-21T04:12:16.688+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 24] 2023-10-21T04:12:16.740+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9112 2023-10-21T04:12:16.740+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:12:16.819+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:12:16.937+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:12:17.064+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:12:17.064+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:12:17.064+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:12:17.122+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:17.122+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:17.122+05:30 [Info] UnwatchMetadata(): indexer 85232c9ba3e4451b95e9e6fc8337e027 2023-10-21T04:12:17.122+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:12:17.122+05:30 [Info] UnwatchMetadata(): finish for indexer 85232c9ba3e4451b95e9e6fc8337e027 2023-10-21T04:12:17.123+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:53520->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:12:17.163+05:30 [Info] switched currmeta from 887 -> 888 force true 2023-10-21T04:12:17.164+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:12:17.164+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] ... stopped 2023-10-21T04:12:17.209+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:17.230+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:37162 2023-10-21T04:12:17.231+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:12:17.440+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:12:17.440+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:17.440+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:17.444+05:30 [Info] switched currmeta from 888 -> 888 force true 2023-10-21T04:12:17.451+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7955106507615827, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:12:17.456+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Stopping releaseConnsRoutine 2023-10-21T04:12:17.469+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9004. 2023-10-21T04:12:17.469+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 4 != len(c.nodesvs): 3 2023-10-21T04:12:17.469+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:12:17.487+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9004. 2023-10-21T04:12:17.487+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 4 != len(c.nodesvs): 3 2023-10-21T04:12:17.487+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:12:17.565+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 317.629203ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:12:17.565+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:12:17.565+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:12:17.565+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:12:17.577+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:12:17.577+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:12:17.577+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:12:17.577+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:12:17.577+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:12:17.578+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:57068->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:12:17.578+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:37162->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:12:17.578+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:37162. Reason = EOF. Cleaning Pipe. 2023-10-21T04:12:17.578+05:30 [Info] messageListener.start(): message channel closed. Remove peer 7d:1:5e:5d:a9:c4:ff:b5 as follower. 2023-10-21T04:12:17.689+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:12:17.689+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:17.726+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:17.726+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:17.735+05:30 [Info] switched currmeta from 888 -> 888 force true 2023-10-21T04:12:17.766+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:40518 closed 2023-10-21T04:12:17.766+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:57160. Error = read tcp4 127.0.0.1:9118->127.0.0.1:57160: read: connection reset by peer. Kill Pipe. 2023-10-21T04:12:17.766+05:30 [Info] messageListener.start(): message channel closed. Remove peer 3e:aa:f0:3c:96:76:cf:ea as follower. 2023-10-21T04:12:17.890+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:12:17.918+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:17.918+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:17.929+05:30 [Info] switched currmeta from 888 -> 888 force true 2023-10-21T04:12:17.969+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:17.969+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:17.978+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:17.978+05:30 [Info] switched currmeta from 888 -> 888 force true 2023-10-21T04:12:18.001+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 182] updated cluster rebalancing status to false 2023-10-21T04:12:18.042+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:18.043+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:18.069+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:12:18.076+05:30 [Info] switched currmeta from 888 -> 888 force true 2023-10-21T04:12:19.385+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:43912 closed 2023-10-21T04:12:19.389+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44206 closed 2023-10-21T04:12:19.441+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44212 closed 2023-10-21T04:12:19.605+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44220 closed 2023-10-21T04:12:19.627+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44242 closed 2023-10-21T04:12:19.662+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44246 closed 2023-10-21T04:12:19.740+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44258 closed 2023-10-21T04:12:19.777+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44262 closed 2023-10-21T04:12:19.791+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44270 closed 2023-10-21T04:12:19.794+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44272 closed 2023-10-21T04:12:19.808+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44274 closed 2023-10-21T04:12:19.817+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44278 closed 2023-10-21T04:12:19.852+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44280 closed 2023-10-21T04:12:19.855+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44282 closed 2023-10-21T04:12:19.915+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44286 closed 2023-10-21T04:12:19.954+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44294 closed 2023-10-21T04:12:19.980+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44298 closed 2023-10-21T04:12:19.985+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44300 closed 2023-10-21T04:12:20.010+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44308 closed 2023-10-21T04:12:20.048+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44310 closed 2023-10-21T04:12:20.095+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44318 closed 2023-10-21T04:12:20.128+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44320 closed 2023-10-21T04:12:20.154+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44322 closed 2023-10-21T04:12:20.157+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44326 closed 2023-10-21T04:12:20.170+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44328 closed 2023-10-21T04:12:20.186+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44332 closed 2023-10-21T04:12:20.245+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44334 closed 2023-10-21T04:12:20.278+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44340 closed 2023-10-21T04:12:20.283+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44342 closed 2023-10-21T04:12:20.300+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44346 closed 2023-10-21T04:12:20.366+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44348 closed 2023-10-21T04:12:20.374+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44354 closed 2023-10-21T04:12:20.387+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44356 closed 2023-10-21T04:12:20.479+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44358 closed 2023-10-21T04:12:20.509+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44366 closed 2023-10-21T04:12:20.513+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44368 closed 2023-10-21T04:12:20.575+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44370 closed 2023-10-21T04:12:20.601+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44384 closed 2023-10-21T04:12:20.630+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44390 closed 2023-10-21T04:12:20.655+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44392 closed 2023-10-21T04:12:20.657+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44396 closed 2023-10-21T04:12:20.660+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44398 closed 2023-10-21T04:12:20.682+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44400 closed 2023-10-21T04:12:20.702+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44404 closed 2023-10-21T04:12:20.754+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44406 closed 2023-10-21T04:12:24.649+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44414 closed 2023-10-21T04:12:24.704+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44528 closed 2023-10-21T04:12:24.733+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44540 closed 2023-10-21T04:12:24.777+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44550 closed 2023-10-21T04:12:24.785+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44554 closed 2023-10-21T04:12:24.789+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44556 closed 2023-10-21T04:12:24.855+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44560 closed 2023-10-21T04:12:24.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44568 closed 2023-10-21T04:12:24.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44570 closed 2023-10-21T04:12:24.906+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44572 closed 2023-10-21T04:12:24.915+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44574 closed 2023-10-21T04:12:24.943+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44576 closed 2023-10-21T04:12:25.021+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44578 closed 2023-10-21T04:12:25.026+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44580 closed 2023-10-21T04:12:25.038+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44582 closed 2023-10-21T04:12:25.062+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44584 closed 2023-10-21T04:12:25.075+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44588 closed 2023-10-21T04:12:25.082+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44590 closed 2023-10-21T04:12:25.087+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44592 closed 2023-10-21T04:12:25.093+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44594 closed 2023-10-21T04:12:25.105+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44598 closed 2023-10-21T04:12:25.105+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44596 closed 2023-10-21T04:12:25.117+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44600 closed 2023-10-21T04:12:25.145+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44602 closed 2023-10-21T04:12:25.169+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44610 closed 2023-10-21T04:12:25.197+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44618 closed 2023-10-21T04:12:25.201+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44622 closed 2023-10-21T04:12:25.222+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44626 closed 2023-10-21T04:12:25.229+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44630 closed 2023-10-21T04:12:25.258+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44632 closed 2023-10-21T04:12:25.264+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44642 closed 2023-10-21T04:12:25.267+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44644 closed 2023-10-21T04:12:25.295+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44648 closed 2023-10-21T04:12:25.335+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44652 closed 2023-10-21T04:12:25.352+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44662 closed 2023-10-21T04:12:25.355+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44664 closed 2023-10-21T04:12:25.358+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44666 closed 2023-10-21T04:12:25.366+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44670 closed 2023-10-21T04:12:25.376+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44674 closed 2023-10-21T04:12:25.398+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44682 closed 2023-10-21T04:12:25.457+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44684 closed 2023-10-21T04:12:25.467+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44690 closed 2023-10-21T04:12:25.492+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44692 closed 2023-10-21T04:12:25.502+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44696 closed 2023-10-21T04:12:25.564+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44698 closed 2023-10-21T04:12:25.568+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44712 closed 2023-10-21T04:12:25.577+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44714 closed 2023-10-21T04:12:25.580+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44716 closed 2023-10-21T04:12:25.636+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44728 closed 2023-10-21T04:12:25.642+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44732 closed 2023-10-21T04:12:25.645+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44734 closed 2023-10-21T04:12:25.706+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44744 closed 2023-10-21T04:12:25.710+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44746 closed 2023-10-21T04:12:25.713+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44748 closed 2023-10-21T04:12:25.755+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44752 closed 2023-10-21T04:12:27.504+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49054" doAuth() ... 2023-10-21T04:12:27.504+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49054" doReceive() ... 2023-10-21T04:12:27.518+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49056" doAuth() ... 2023-10-21T04:12:27.519+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49056" doReceive() ... 2023-10-21T04:12:27.521+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49058" doAuth() ... 2023-10-21T04:12:27.522+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49058" doReceive() ... 2023-10-21T04:12:27.532+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49062" doAuth() ... 2023-10-21T04:12:27.533+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49062" doReceive() ... 2023-10-21T04:12:27.556+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49064" doAuth() ... 2023-10-21T04:12:27.557+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49064" doReceive() ... 2023-10-21T04:12:27.571+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49066" doAuth() ... 2023-10-21T04:12:27.573+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49066" doReceive() ... 2023-10-21T04:12:27.582+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49068" doAuth() ... 2023-10-21T04:12:27.583+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49068" doReceive() ... 2023-10-21T04:12:27.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49072" doAuth() ... 2023-10-21T04:12:27.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49072" doReceive() ... 2023-10-21T04:12:27.600+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49074" doAuth() ... 2023-10-21T04:12:27.601+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49074" doReceive() ... 2023-10-21T04:12:27.637+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49082" doAuth() ... 2023-10-21T04:12:27.638+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49082" doReceive() ... 2023-10-21T04:12:27.650+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49086" doAuth() ... 2023-10-21T04:12:27.650+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49086" doReceive() ... 2023-10-21T04:12:27.686+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49090" doAuth() ... 2023-10-21T04:12:27.687+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49090" doReceive() ... 2023-10-21T04:12:27.689+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49092" doAuth() ... 2023-10-21T04:12:27.689+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49092" doReceive() ... 2023-10-21T04:12:27.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49096" doAuth() ... 2023-10-21T04:12:27.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49096" doReceive() ... 2023-10-21T04:12:27.738+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49098" doAuth() ... 2023-10-21T04:12:27.738+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49098" doReceive() ... 2023-10-21T04:12:27.784+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49102" doAuth() ... 2023-10-21T04:12:27.785+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49102" doReceive() ... 2023-10-21T04:12:27.787+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49104" doAuth() ... 2023-10-21T04:12:27.788+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49104" doReceive() ... 2023-10-21T04:12:27.810+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49110" doAuth() ... 2023-10-21T04:12:27.811+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49110" doReceive() ... 2023-10-21T04:12:27.814+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49112" doAuth() ... 2023-10-21T04:12:27.814+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49112" doReceive() ... 2023-10-21T04:12:27.856+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49118" doAuth() ... 2023-10-21T04:12:27.857+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49118" doReceive() ... 2023-10-21T04:12:27.896+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49124" doAuth() ... 2023-10-21T04:12:27.897+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49124" doReceive() ... 2023-10-21T04:12:27.991+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49134" doAuth() ... 2023-10-21T04:12:27.993+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49134" doReceive() ... 2023-10-21T04:12:27.999+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49138" doAuth() ... 2023-10-21T04:12:28.000+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49138" doReceive() ... 2023-10-21T04:12:28.011+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49140" doAuth() ... 2023-10-21T04:12:28.012+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49140" doReceive() ... 2023-10-21T04:12:28.061+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49148" doAuth() ... 2023-10-21T04:12:28.062+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49148" doReceive() ... 2023-10-21T04:12:28.066+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49150" doAuth() ... 2023-10-21T04:12:28.066+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49150" doReceive() ... 2023-10-21T04:12:28.068+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49152" doAuth() ... 2023-10-21T04:12:28.069+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49152" doReceive() ... 2023-10-21T04:12:28.071+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49154" doAuth() ... 2023-10-21T04:12:28.072+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49154" doReceive() ... 2023-10-21T04:12:28.074+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49156" doAuth() ... 2023-10-21T04:12:28.076+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49156" doReceive() ... 2023-10-21T04:12:28.079+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49158" doAuth() ... 2023-10-21T04:12:28.079+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49158" doReceive() ... 2023-10-21T04:12:28.109+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49160" doAuth() ... 2023-10-21T04:12:28.110+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49160" doReceive() ... 2023-10-21T04:12:28.124+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49162" doAuth() ... 2023-10-21T04:12:28.126+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49162" doReceive() ... 2023-10-21T04:12:28.189+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49178" doAuth() ... 2023-10-21T04:12:28.190+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49178" doReceive() ... 2023-10-21T04:12:28.193+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49180" doAuth() ... 2023-10-21T04:12:28.194+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49180" doReceive() ... 2023-10-21T04:12:28.197+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49182" doAuth() ... 2023-10-21T04:12:28.199+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49182" doReceive() ... 2023-10-21T04:12:28.214+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49186" doAuth() ... 2023-10-21T04:12:28.215+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49186" doReceive() ... 2023-10-21T04:12:28.261+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49196" doAuth() ... 2023-10-21T04:12:28.266+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49196" doReceive() ... 2023-10-21T04:12:28.308+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49200" doAuth() ... 2023-10-21T04:12:28.310+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49200" doReceive() ... 2023-10-21T04:12:28.363+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49210" doAuth() ... 2023-10-21T04:12:28.364+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49210" doReceive() ... 2023-10-21T04:12:28.367+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49212" doAuth() ... 2023-10-21T04:12:28.368+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49212" doReceive() ... 2023-10-21T04:12:28.380+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49214" doAuth() ... 2023-10-21T04:12:28.380+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49214" doReceive() ... 2023-10-21T04:12:28.386+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49216" doAuth() ... 2023-10-21T04:12:28.386+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49216" doReceive() ... 2023-10-21T04:12:28.476+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49224" doAuth() ... 2023-10-21T04:12:28.477+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49224" doReceive() ... 2023-10-21T04:12:28.483+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49226" doAuth() ... 2023-10-21T04:12:28.483+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49226" doReceive() ... 2023-10-21T04:12:28.497+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49228" doAuth() ... 2023-10-21T04:12:28.498+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49228" doReceive() ... 2023-10-21T04:12:28.513+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49232" doAuth() ... 2023-10-21T04:12:28.514+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49232" doReceive() ... 2023-10-21T04:12:28.525+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49234" doAuth() ... 2023-10-21T04:12:28.526+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49234" doReceive() ... 2023-10-21T04:12:28.561+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49242" doAuth() ... 2023-10-21T04:12:28.563+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49242" doReceive() ... 2023-10-21T04:12:28.565+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49244" doAuth() ... 2023-10-21T04:12:28.566+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49244" doReceive() ... 2023-10-21T04:12:28.594+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49246" doAuth() ... 2023-10-21T04:12:28.595+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49246" doReceive() ... 2023-10-21T04:12:28.652+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49252" doAuth() ... 2023-10-21T04:12:28.666+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49252" doReceive() ... 2023-10-21T04:12:28.694+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49256" doAuth() ... 2023-10-21T04:12:28.696+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49256" doReceive() ... 2023-10-21T04:12:28.750+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49266" doAuth() ... 2023-10-21T04:12:28.751+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49266" doReceive() ... 2023-10-21T04:12:28.769+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49272" doAuth() ... 2023-10-21T04:12:28.770+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49272" doReceive() ... 2023-10-21T04:12:29.321+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44754 closed 2023-10-21T04:12:29.327+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44806 closed 2023-10-21T04:12:29.330+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44808 closed 2023-10-21T04:12:29.364+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44816 closed 2023-10-21T04:12:29.398+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44822 closed 2023-10-21T04:12:29.401+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44828 closed 2023-10-21T04:12:29.404+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44830 closed 2023-10-21T04:12:29.437+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44834 closed 2023-10-21T04:12:29.446+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44838 closed 2023-10-21T04:12:29.489+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44840 closed 2023-10-21T04:12:29.494+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44848 closed 2023-10-21T04:12:29.512+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44852 closed 2023-10-21T04:12:29.533+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44854 closed 2023-10-21T04:12:29.539+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44858 closed 2023-10-21T04:12:29.539+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44856 closed 2023-10-21T04:12:29.541+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44860 closed 2023-10-21T04:12:29.550+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44862 closed 2023-10-21T04:12:29.565+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44864 closed 2023-10-21T04:12:29.591+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44866 closed 2023-10-21T04:12:29.630+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44868 closed 2023-10-21T04:12:29.634+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44882 closed 2023-10-21T04:12:29.667+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44884 closed 2023-10-21T04:12:29.671+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44890 closed 2023-10-21T04:12:29.691+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44894 closed 2023-10-21T04:12:29.699+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44896 closed 2023-10-21T04:12:29.704+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44898 closed 2023-10-21T04:12:29.731+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44900 closed 2023-10-21T04:12:29.734+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44906 closed 2023-10-21T04:12:29.770+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44910 closed 2023-10-21T04:12:29.784+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44914 closed 2023-10-21T04:12:29.800+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44918 closed 2023-10-21T04:12:29.805+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44920 closed 2023-10-21T04:12:29.814+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44924 closed 2023-10-21T04:12:29.819+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44926 closed 2023-10-21T04:12:29.826+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44928 closed 2023-10-21T04:12:29.827+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44930 closed 2023-10-21T04:12:29.837+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44932 closed 2023-10-21T04:12:29.853+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44938 closed 2023-10-21T04:12:29.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44942 closed 2023-10-21T04:12:29.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44948 closed 2023-10-21T04:12:29.905+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44952 closed 2023-10-21T04:12:29.905+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44950 closed 2023-10-21T04:12:29.916+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44954 closed 2023-10-21T04:12:29.917+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44956 closed 2023-10-21T04:12:29.932+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44960 closed 2023-10-21T04:12:29.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44962 closed 2023-10-21T04:12:29.961+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44966 closed 2023-10-21T04:12:29.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44968 closed 2023-10-21T04:12:30.016+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44972 closed 2023-10-21T04:12:30.018+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44978 closed 2023-10-21T04:12:30.044+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44982 closed 2023-10-21T04:12:30.076+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44984 closed 2023-10-21T04:12:30.171+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:44986 closed 2023-10-21T04:12:30.198+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45000 closed 2023-10-21T04:12:30.331+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45002 closed 2023-10-21T04:12:33.449+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49410" doAuth() ... 2023-10-21T04:12:33.450+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49410" doReceive() ... 2023-10-21T04:12:33.452+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49412" doAuth() ... 2023-10-21T04:12:33.454+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49412" doReceive() ... 2023-10-21T04:12:33.462+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49414" doAuth() ... 2023-10-21T04:12:33.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49414" doReceive() ... 2023-10-21T04:12:33.477+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49418" doAuth() ... 2023-10-21T04:12:33.477+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49418" doReceive() ... 2023-10-21T04:12:33.499+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49422" doAuth() ... 2023-10-21T04:12:33.501+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49422" doReceive() ... 2023-10-21T04:12:33.509+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49424" doAuth() ... 2023-10-21T04:12:33.511+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49424" doReceive() ... 2023-10-21T04:12:33.519+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49426" doAuth() ... 2023-10-21T04:12:33.522+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49426" doReceive() ... 2023-10-21T04:12:33.547+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49430" doAuth() ... 2023-10-21T04:12:33.548+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49430" doReceive() ... 2023-10-21T04:12:33.593+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49444" doAuth() ... 2023-10-21T04:12:33.594+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49444" doReceive() ... 2023-10-21T04:12:33.618+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49452" doAuth() ... 2023-10-21T04:12:33.621+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49452" doReceive() ... 2023-10-21T04:12:33.627+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49454" doAuth() ... 2023-10-21T04:12:33.628+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49454" doReceive() ... 2023-10-21T04:12:33.630+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49456" doAuth() ... 2023-10-21T04:12:33.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49456" doReceive() ... 2023-10-21T04:12:33.663+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49466" doAuth() ... 2023-10-21T04:12:33.664+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49466" doReceive() ... 2023-10-21T04:12:33.700+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49476" doAuth() ... 2023-10-21T04:12:33.702+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49476" doReceive() ... 2023-10-21T04:12:33.743+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49486" doAuth() ... 2023-10-21T04:12:33.745+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49486" doReceive() ... 2023-10-21T04:12:33.801+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49498" doAuth() ... 2023-10-21T04:12:33.803+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49498" doReceive() ... 2023-10-21T04:12:33.805+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49500" doAuth() ... 2023-10-21T04:12:33.808+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49500" doReceive() ... 2023-10-21T04:12:33.824+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49506" doAuth() ... 2023-10-21T04:12:33.826+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49506" doReceive() ... 2023-10-21T04:12:33.893+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49518" doAuth() ... 2023-10-21T04:12:33.895+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49518" doReceive() ... 2023-10-21T04:12:33.898+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49520" doAuth() ... 2023-10-21T04:12:33.900+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49520" doReceive() ... 2023-10-21T04:12:33.909+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49522" doAuth() ... 2023-10-21T04:12:33.911+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49522" doReceive() ... 2023-10-21T04:12:33.944+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49526" doAuth() ... 2023-10-21T04:12:33.944+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49526" doReceive() ... 2023-10-21T04:12:33.998+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49532" doAuth() ... 2023-10-21T04:12:33.999+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49532" doReceive() ... 2023-10-21T04:12:34.018+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49536" doAuth() ... 2023-10-21T04:12:34.020+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49536" doReceive() ... 2023-10-21T04:12:34.023+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49538" doAuth() ... 2023-10-21T04:12:34.024+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49538" doReceive() ... 2023-10-21T04:12:34.062+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49544" doAuth() ... 2023-10-21T04:12:34.065+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49544" doReceive() ... 2023-10-21T04:12:34.068+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49546" doAuth() ... 2023-10-21T04:12:34.070+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49546" doReceive() ... 2023-10-21T04:12:34.078+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49548" doAuth() ... 2023-10-21T04:12:34.079+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49548" doReceive() ... 2023-10-21T04:12:34.095+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49556" doAuth() ... 2023-10-21T04:12:34.097+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49556" doReceive() ... 2023-10-21T04:12:34.159+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49562" doAuth() ... 2023-10-21T04:12:34.161+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49562" doReceive() ... 2023-10-21T04:12:34.191+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49564" doAuth() ... 2023-10-21T04:12:34.192+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49564" doReceive() ... 2023-10-21T04:12:34.211+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49566" doAuth() ... 2023-10-21T04:12:34.212+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49566" doReceive() ... 2023-10-21T04:12:34.221+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45026 closed 2023-10-21T04:12:34.225+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45122 closed 2023-10-21T04:12:34.232+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49570" doAuth() ... 2023-10-21T04:12:34.233+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49570" doReceive() ... 2023-10-21T04:12:34.238+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45124 closed 2023-10-21T04:12:34.244+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45128 closed 2023-10-21T04:12:34.244+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45126 closed 2023-10-21T04:12:34.253+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45132 closed 2023-10-21T04:12:34.253+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45130 closed 2023-10-21T04:12:34.272+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49576" doAuth() ... 2023-10-21T04:12:34.273+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49576" doReceive() ... 2023-10-21T04:12:34.276+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49578" doAuth() ... 2023-10-21T04:12:34.278+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49578" doReceive() ... 2023-10-21T04:12:34.281+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49580" doAuth() ... 2023-10-21T04:12:34.284+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49580" doReceive() ... 2023-10-21T04:12:34.291+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45136 closed 2023-10-21T04:12:34.297+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45146 closed 2023-10-21T04:12:34.297+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45148 closed 2023-10-21T04:12:34.299+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49586" doAuth() ... 2023-10-21T04:12:34.299+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49586" doReceive() ... 2023-10-21T04:12:34.300+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45150 closed 2023-10-21T04:12:34.302+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45152 closed 2023-10-21T04:12:34.308+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49590" doAuth() ... 2023-10-21T04:12:34.309+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49590" doReceive() ... 2023-10-21T04:12:34.332+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45154 closed 2023-10-21T04:12:34.337+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49594" doAuth() ... 2023-10-21T04:12:34.337+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49594" doReceive() ... 2023-10-21T04:12:34.340+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49596" doAuth() ... 2023-10-21T04:12:34.342+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49596" doReceive() ... 2023-10-21T04:12:34.343+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45164 closed 2023-10-21T04:12:34.351+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49598" doAuth() ... 2023-10-21T04:12:34.353+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49598" doReceive() ... 2023-10-21T04:12:34.377+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45168 closed 2023-10-21T04:12:34.378+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45170 closed 2023-10-21T04:12:34.402+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49600" doAuth() ... 2023-10-21T04:12:34.403+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49600" doReceive() ... 2023-10-21T04:12:34.425+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45182 closed 2023-10-21T04:12:34.428+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45184 closed 2023-10-21T04:12:34.468+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49610" doAuth() ... 2023-10-21T04:12:34.468+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49610" doReceive() ... 2023-10-21T04:12:34.471+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49612" doAuth() ... 2023-10-21T04:12:34.471+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49612" doReceive() ... 2023-10-21T04:12:34.475+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45188 closed 2023-10-21T04:12:34.479+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45190 closed 2023-10-21T04:12:34.484+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49614" doAuth() ... 2023-10-21T04:12:34.484+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49614" doReceive() ... 2023-10-21T04:12:34.524+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49620" doAuth() ... 2023-10-21T04:12:34.524+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49620" doReceive() ... 2023-10-21T04:12:34.533+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49622" doAuth() ... 2023-10-21T04:12:34.534+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49622" doReceive() ... 2023-10-21T04:12:34.536+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45200 closed 2023-10-21T04:12:34.536+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49624" doAuth() ... 2023-10-21T04:12:34.537+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49624" doReceive() ... 2023-10-21T04:12:34.540+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49626" doAuth() ... 2023-10-21T04:12:34.540+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45208 closed 2023-10-21T04:12:34.540+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49626" doReceive() ... 2023-10-21T04:12:34.552+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45212 closed 2023-10-21T04:12:34.562+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49632" doAuth() ... 2023-10-21T04:12:34.563+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49632" doReceive() ... 2023-10-21T04:12:34.575+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45214 closed 2023-10-21T04:12:34.583+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49638" doAuth() ... 2023-10-21T04:12:34.584+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49638" doReceive() ... 2023-10-21T04:12:34.596+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49642" doAuth() ... 2023-10-21T04:12:34.596+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49642" doReceive() ... 2023-10-21T04:12:34.599+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49644" doAuth() ... 2023-10-21T04:12:34.599+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49644" doReceive() ... 2023-10-21T04:12:34.602+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49646" doAuth() ... 2023-10-21T04:12:34.603+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49646" doReceive() ... 2023-10-21T04:12:34.611+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49648" doAuth() ... 2023-10-21T04:12:34.612+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49648" doReceive() ... 2023-10-21T04:12:34.629+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49652" doAuth() ... 2023-10-21T04:12:34.629+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49652" doReceive() ... 2023-10-21T04:12:34.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49654" doAuth() ... 2023-10-21T04:12:34.633+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49654" doReceive() ... 2023-10-21T04:12:34.646+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49656" doAuth() ... 2023-10-21T04:12:34.646+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49656" doReceive() ... 2023-10-21T04:12:34.670+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49660" doAuth() ... 2023-10-21T04:12:34.671+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49660" doReceive() ... 2023-10-21T04:12:34.690+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45218 closed 2023-10-21T04:12:34.700+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45238 closed 2023-10-21T04:12:34.704+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45240 closed 2023-10-21T04:12:34.712+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49672" doAuth() ... 2023-10-21T04:12:34.713+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49672" doReceive() ... 2023-10-21T04:12:34.737+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45248 closed 2023-10-21T04:12:34.737+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45246 closed 2023-10-21T04:12:34.750+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45250 closed 2023-10-21T04:12:34.764+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45252 closed 2023-10-21T04:12:34.777+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45254 closed 2023-10-21T04:12:34.780+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45256 closed 2023-10-21T04:12:34.795+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45260 closed 2023-10-21T04:12:34.809+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45262 closed 2023-10-21T04:12:34.820+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45264 closed 2023-10-21T04:12:34.831+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45268 closed 2023-10-21T04:12:34.856+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45270 closed 2023-10-21T04:12:34.859+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45272 closed 2023-10-21T04:12:34.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45276 closed 2023-10-21T04:12:34.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45278 closed 2023-10-21T04:12:34.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45280 closed 2023-10-21T04:12:34.894+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45288 closed 2023-10-21T04:12:34.913+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45290 closed 2023-10-21T04:12:34.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45294 closed 2023-10-21T04:12:34.975+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45296 closed 2023-10-21T04:12:34.984+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45302 closed 2023-10-21T04:12:35.028+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45312 closed 2023-10-21T04:12:35.092+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45314 closed 2023-10-21T04:12:35.115+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45318 closed 2023-10-21T04:12:35.129+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45320 closed 2023-10-21T04:12:35.132+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45322 closed 2023-10-21T04:12:35.152+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45324 closed 2023-10-21T04:12:35.164+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45328 closed 2023-10-21T04:12:35.198+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45332 closed 2023-10-21T04:12:35.455+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49708" doAuth() ... 2023-10-21T04:12:35.456+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49708" doReceive() ... 2023-10-21T04:12:38.283+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49744" doAuth() ... 2023-10-21T04:12:38.283+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49744" doReceive() ... 2023-10-21T04:12:38.298+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49748" doAuth() ... 2023-10-21T04:12:38.300+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49748" doReceive() ... 2023-10-21T04:12:38.327+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49752" doAuth() ... 2023-10-21T04:12:38.329+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49752" doReceive() ... 2023-10-21T04:12:38.366+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49760" doAuth() ... 2023-10-21T04:12:38.369+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49760" doReceive() ... 2023-10-21T04:12:38.377+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49764" doAuth() ... 2023-10-21T04:12:38.379+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49764" doReceive() ... 2023-10-21T04:12:38.564+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49782" doAuth() ... 2023-10-21T04:12:38.565+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49782" doReceive() ... 2023-10-21T04:12:38.571+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49786" doAuth() ... 2023-10-21T04:12:38.571+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49786" doReceive() ... 2023-10-21T04:12:38.582+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49790" doAuth() ... 2023-10-21T04:12:38.583+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49790" doReceive() ... 2023-10-21T04:12:38.596+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49794" doAuth() ... 2023-10-21T04:12:38.597+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49794" doReceive() ... 2023-10-21T04:12:38.629+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49806" doAuth() ... 2023-10-21T04:12:38.630+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49806" doReceive() ... 2023-10-21T04:12:38.640+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49810" doAuth() ... 2023-10-21T04:12:38.642+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49810" doReceive() ... 2023-10-21T04:12:38.644+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49812" doAuth() ... 2023-10-21T04:12:38.645+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49812" doReceive() ... 2023-10-21T04:12:38.746+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49828" doAuth() ... 2023-10-21T04:12:38.747+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49828" doReceive() ... 2023-10-21T04:12:38.758+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49832" doAuth() ... 2023-10-21T04:12:38.759+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49832" doReceive() ... 2023-10-21T04:12:38.764+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49836" doAuth() ... 2023-10-21T04:12:38.765+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49836" doReceive() ... 2023-10-21T04:12:38.771+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49840" doAuth() ... 2023-10-21T04:12:38.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49840" doReceive() ... 2023-10-21T04:12:38.797+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49848" doAuth() ... 2023-10-21T04:12:38.797+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49848" doReceive() ... 2023-10-21T04:12:38.823+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49852" doAuth() ... 2023-10-21T04:12:38.824+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49852" doReceive() ... 2023-10-21T04:12:38.830+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49854" doAuth() ... 2023-10-21T04:12:38.831+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49854" doReceive() ... 2023-10-21T04:12:38.848+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49860" doAuth() ... 2023-10-21T04:12:38.849+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49860" doReceive() ... 2023-10-21T04:12:38.853+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45334 closed 2023-10-21T04:12:38.855+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49864" doAuth() ... 2023-10-21T04:12:38.856+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49864" doReceive() ... 2023-10-21T04:12:38.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45382 closed 2023-10-21T04:12:38.884+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49870" doAuth() ... 2023-10-21T04:12:38.886+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49870" doReceive() ... 2023-10-21T04:12:38.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45390 closed 2023-10-21T04:12:38.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45392 closed 2023-10-21T04:12:38.905+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49874" doAuth() ... 2023-10-21T04:12:38.907+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49874" doReceive() ... 2023-10-21T04:12:38.927+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49878" doAuth() ... 2023-10-21T04:12:38.928+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49878" doReceive() ... 2023-10-21T04:12:38.944+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45394 closed 2023-10-21T04:12:38.962+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45402 closed 2023-10-21T04:12:38.970+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49882" doAuth() ... 2023-10-21T04:12:38.971+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49882" doReceive() ... 2023-10-21T04:12:39.007+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45408 closed 2023-10-21T04:12:39.019+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45416 closed 2023-10-21T04:12:39.024+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49886" doAuth() ... 2023-10-21T04:12:39.026+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49886" doReceive() ... 2023-10-21T04:12:39.029+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49888" doAuth() ... 2023-10-21T04:12:39.029+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49888" doReceive() ... 2023-10-21T04:12:39.030+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45420 closed 2023-10-21T04:12:39.032+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49890" doAuth() ... 2023-10-21T04:12:39.032+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49890" doReceive() ... 2023-10-21T04:12:39.035+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49892" doAuth() ... 2023-10-21T04:12:39.036+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49892" doReceive() ... 2023-10-21T04:12:39.047+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49896" doAuth() ... 2023-10-21T04:12:39.047+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49896" doReceive() ... 2023-10-21T04:12:39.050+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49898" doAuth() ... 2023-10-21T04:12:39.050+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49898" doReceive() ... 2023-10-21T04:12:39.059+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49900" doAuth() ... 2023-10-21T04:12:39.060+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49900" doReceive() ... 2023-10-21T04:12:39.065+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45426 closed 2023-10-21T04:12:39.083+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49906" doAuth() ... 2023-10-21T04:12:39.084+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49906" doReceive() ... 2023-10-21T04:12:39.087+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49908" doAuth() ... 2023-10-21T04:12:39.087+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49908" doReceive() ... 2023-10-21T04:12:39.107+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49912" doAuth() ... 2023-10-21T04:12:39.108+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49912" doReceive() ... 2023-10-21T04:12:39.116+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45428 closed 2023-10-21T04:12:39.126+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45432 closed 2023-10-21T04:12:39.128+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45434 closed 2023-10-21T04:12:39.133+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45438 closed 2023-10-21T04:12:39.140+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45440 closed 2023-10-21T04:12:39.149+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45446 closed 2023-10-21T04:12:39.150+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45448 closed 2023-10-21T04:12:39.196+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45450 closed 2023-10-21T04:12:39.203+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45454 closed 2023-10-21T04:12:39.206+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49924" doAuth() ... 2023-10-21T04:12:39.207+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49924" doReceive() ... 2023-10-21T04:12:39.239+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49928" doAuth() ... 2023-10-21T04:12:39.239+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49928" doReceive() ... 2023-10-21T04:12:39.258+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45458 closed 2023-10-21T04:12:39.268+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49932" doAuth() ... 2023-10-21T04:12:39.269+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49932" doReceive() ... 2023-10-21T04:12:39.288+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45468 closed 2023-10-21T04:12:39.296+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45472 closed 2023-10-21T04:12:39.298+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45474 closed 2023-10-21T04:12:39.306+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49938" doAuth() ... 2023-10-21T04:12:39.307+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49938" doReceive() ... 2023-10-21T04:12:39.313+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45480 closed 2023-10-21T04:12:39.324+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49940" doAuth() ... 2023-10-21T04:12:39.325+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49940" doReceive() ... 2023-10-21T04:12:39.340+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49942" doAuth() ... 2023-10-21T04:12:39.340+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45486 closed 2023-10-21T04:12:39.341+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49942" doReceive() ... 2023-10-21T04:12:39.358+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49946" doAuth() ... 2023-10-21T04:12:39.358+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49946" doReceive() ... 2023-10-21T04:12:39.360+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49948" doAuth() ... 2023-10-21T04:12:39.360+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45488 closed 2023-10-21T04:12:39.361+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49948" doReceive() ... 2023-10-21T04:12:39.369+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45496 closed 2023-10-21T04:12:39.376+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49950" doAuth() ... 2023-10-21T04:12:39.377+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49950" doReceive() ... 2023-10-21T04:12:39.393+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45500 closed 2023-10-21T04:12:39.399+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49954" doAuth() ... 2023-10-21T04:12:39.399+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49954" doReceive() ... 2023-10-21T04:12:39.415+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45502 closed 2023-10-21T04:12:39.481+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45504 closed 2023-10-21T04:12:39.502+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45520 closed 2023-10-21T04:12:39.518+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45526 closed 2023-10-21T04:12:39.532+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45528 closed 2023-10-21T04:12:39.590+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45530 closed 2023-10-21T04:12:39.618+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45536 closed 2023-10-21T04:12:39.630+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45540 closed 2023-10-21T04:12:39.660+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45542 closed 2023-10-21T04:12:39.677+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45546 closed 2023-10-21T04:12:39.678+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45544 closed 2023-10-21T04:12:39.693+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45548 closed 2023-10-21T04:12:39.715+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45552 closed 2023-10-21T04:12:39.730+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45558 closed 2023-10-21T04:12:40.066+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:12:40.081+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49962" doAuth() ... 2023-10-21T04:12:40.082+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49962" doReceive() ... 2023-10-21T04:12:42.478+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:12:42.998+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50008" doAuth() ... 2023-10-21T04:12:42.998+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50008" doReceive() ... 2023-10-21T04:12:43.017+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50012" doAuth() ... 2023-10-21T04:12:43.017+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50012" doReceive() ... 2023-10-21T04:12:43.019+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50014" doAuth() ... 2023-10-21T04:12:43.020+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50014" doReceive() ... 2023-10-21T04:12:43.037+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50022" doAuth() ... 2023-10-21T04:12:43.038+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50022" doReceive() ... 2023-10-21T04:12:43.040+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50024" doAuth() ... 2023-10-21T04:12:43.041+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50024" doReceive() ... 2023-10-21T04:12:43.072+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50026" doAuth() ... 2023-10-21T04:12:43.072+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50026" doReceive() ... 2023-10-21T04:12:43.083+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50032" doAuth() ... 2023-10-21T04:12:43.083+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50032" doReceive() ... 2023-10-21T04:12:43.085+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50034" doAuth() ... 2023-10-21T04:12:43.086+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50034" doReceive() ... 2023-10-21T04:12:43.126+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50042" doAuth() ... 2023-10-21T04:12:43.127+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50042" doReceive() ... 2023-10-21T04:12:43.144+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50048" doAuth() ... 2023-10-21T04:12:43.144+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50048" doReceive() ... 2023-10-21T04:12:43.169+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50054" doAuth() ... 2023-10-21T04:12:43.170+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50054" doReceive() ... 2023-10-21T04:12:43.215+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50056" doAuth() ... 2023-10-21T04:12:43.216+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50056" doReceive() ... 2023-10-21T04:12:43.251+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50064" doAuth() ... 2023-10-21T04:12:43.252+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50064" doReceive() ... 2023-10-21T04:12:43.320+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50068" doAuth() ... 2023-10-21T04:12:43.321+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50068" doReceive() ... 2023-10-21T04:12:43.335+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50072" doAuth() ... 2023-10-21T04:12:43.335+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50072" doReceive() ... 2023-10-21T04:12:43.341+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50076" doAuth() ... 2023-10-21T04:12:43.343+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50076" doReceive() ... 2023-10-21T04:12:43.365+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50082" doAuth() ... 2023-10-21T04:12:43.365+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50082" doReceive() ... 2023-10-21T04:12:43.367+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50084" doAuth() ... 2023-10-21T04:12:43.369+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50084" doReceive() ... 2023-10-21T04:12:43.374+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50088" doAuth() ... 2023-10-21T04:12:43.375+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50088" doReceive() ... 2023-10-21T04:12:43.406+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50098" doAuth() ... 2023-10-21T04:12:43.407+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50098" doReceive() ... 2023-10-21T04:12:43.420+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50102" doAuth() ... 2023-10-21T04:12:43.422+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50102" doReceive() ... 2023-10-21T04:12:43.462+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50110" doAuth() ... 2023-10-21T04:12:43.463+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50110" doReceive() ... 2023-10-21T04:12:43.494+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50114" doAuth() ... 2023-10-21T04:12:43.497+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50114" doReceive() ... 2023-10-21T04:12:43.539+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50122" doAuth() ... 2023-10-21T04:12:43.541+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50122" doReceive() ... 2023-10-21T04:12:43.580+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50126" doAuth() ... 2023-10-21T04:12:43.581+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50126" doReceive() ... 2023-10-21T04:12:43.606+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50136" doAuth() ... 2023-10-21T04:12:43.607+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50136" doReceive() ... 2023-10-21T04:12:43.715+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50154" doAuth() ... 2023-10-21T04:12:43.716+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50154" doReceive() ... 2023-10-21T04:12:43.730+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50158" doAuth() ... 2023-10-21T04:12:43.730+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50158" doReceive() ... 2023-10-21T04:12:43.769+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50172" doAuth() ... 2023-10-21T04:12:43.770+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50172" doReceive() ... 2023-10-21T04:12:43.829+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50184" doAuth() ... 2023-10-21T04:12:43.829+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50184" doReceive() ... 2023-10-21T04:12:43.855+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50188" doAuth() ... 2023-10-21T04:12:43.858+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50188" doReceive() ... 2023-10-21T04:12:43.881+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50192" doAuth() ... 2023-10-21T04:12:43.881+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50192" doReceive() ... 2023-10-21T04:12:43.950+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50202" doAuth() ... 2023-10-21T04:12:43.951+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50202" doReceive() ... 2023-10-21T04:12:43.965+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50206" doAuth() ... 2023-10-21T04:12:43.965+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50206" doReceive() ... 2023-10-21T04:12:43.975+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50208" doAuth() ... 2023-10-21T04:12:43.977+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50208" doReceive() ... 2023-10-21T04:12:44.022+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50214" doAuth() ... 2023-10-21T04:12:44.024+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50214" doReceive() ... 2023-10-21T04:12:44.038+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50216" doAuth() ... 2023-10-21T04:12:44.040+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50216" doReceive() ... 2023-10-21T04:12:44.062+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50220" doAuth() ... 2023-10-21T04:12:44.062+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50220" doReceive() ... 2023-10-21T04:12:46.685+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 24] call. err: operation_canceled 2023-10-21T04:12:46.685+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:12:46.685+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 24] Tasks:[]} 2023-10-21T04:12:46.686+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:12:46.686+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 24] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:46.688+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 24] call. err: operation_canceled 2023-10-21T04:12:46.688+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:12:46.688+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 24] Tasks:[]} 2023-10-21T04:12:46.689+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:12:46.689+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 24] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:12:46.689+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 24] 2023-10-21T04:12:46.690+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 24] 2023-10-21T04:12:47.562+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50284" doAuth() ... 2023-10-21T04:12:47.562+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50284" doReceive() ... 2023-10-21T04:12:47.582+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50286" doAuth() ... 2023-10-21T04:12:47.583+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50286" doReceive() ... 2023-10-21T04:12:47.603+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50292" doAuth() ... 2023-10-21T04:12:47.603+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50292" doReceive() ... 2023-10-21T04:12:47.614+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50296" doAuth() ... 2023-10-21T04:12:47.614+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50296" doReceive() ... 2023-10-21T04:12:47.636+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50300" doAuth() ... 2023-10-21T04:12:47.636+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50300" doReceive() ... 2023-10-21T04:12:47.638+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50302" doAuth() ... 2023-10-21T04:12:47.640+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50302" doReceive() ... 2023-10-21T04:12:47.686+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50312" doAuth() ... 2023-10-21T04:12:47.687+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50312" doReceive() ... 2023-10-21T04:12:47.774+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50320" doAuth() ... 2023-10-21T04:12:47.775+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50320" doReceive() ... 2023-10-21T04:12:47.790+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50322" doAuth() ... 2023-10-21T04:12:47.791+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50322" doReceive() ... 2023-10-21T04:12:47.804+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50328" doAuth() ... 2023-10-21T04:12:47.805+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50328" doReceive() ... 2023-10-21T04:12:47.807+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50330" doAuth() ... 2023-10-21T04:12:47.808+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50330" doReceive() ... 2023-10-21T04:12:47.816+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50332" doAuth() ... 2023-10-21T04:12:47.816+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50332" doReceive() ... 2023-10-21T04:12:47.843+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50338" doAuth() ... 2023-10-21T04:12:47.844+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50338" doReceive() ... 2023-10-21T04:12:47.846+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50340" doAuth() ... 2023-10-21T04:12:47.846+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50340" doReceive() ... 2023-10-21T04:12:47.849+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50342" doAuth() ... 2023-10-21T04:12:47.850+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50342" doReceive() ... 2023-10-21T04:12:47.866+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50350" doAuth() ... 2023-10-21T04:12:47.867+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50350" doReceive() ... 2023-10-21T04:12:47.900+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50352" doAuth() ... 2023-10-21T04:12:47.901+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50352" doReceive() ... 2023-10-21T04:12:47.975+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50362" doAuth() ... 2023-10-21T04:12:47.976+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50362" doReceive() ... 2023-10-21T04:12:48.009+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50366" doAuth() ... 2023-10-21T04:12:48.010+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50366" doReceive() ... 2023-10-21T04:12:48.040+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50368" doAuth() ... 2023-10-21T04:12:48.040+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50368" doReceive() ... 2023-10-21T04:12:48.043+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50370" doAuth() ... 2023-10-21T04:12:48.044+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50370" doReceive() ... 2023-10-21T04:12:48.053+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50374" doAuth() ... 2023-10-21T04:12:48.054+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50374" doReceive() ... 2023-10-21T04:12:48.056+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50376" doAuth() ... 2023-10-21T04:12:48.057+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50376" doReceive() ... 2023-10-21T04:12:48.059+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50378" doAuth() ... 2023-10-21T04:12:48.059+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50378" doReceive() ... 2023-10-21T04:12:48.140+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50394" doAuth() ... 2023-10-21T04:12:48.141+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50394" doReceive() ... 2023-10-21T04:12:48.147+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50398" doAuth() ... 2023-10-21T04:12:48.148+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50398" doReceive() ... 2023-10-21T04:12:48.153+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50402" doAuth() ... 2023-10-21T04:12:48.154+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50402" doReceive() ... 2023-10-21T04:12:48.183+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50408" doAuth() ... 2023-10-21T04:12:48.183+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50408" doReceive() ... 2023-10-21T04:12:48.186+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50410" doAuth() ... 2023-10-21T04:12:48.186+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50410" doReceive() ... 2023-10-21T04:12:48.189+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50412" doAuth() ... 2023-10-21T04:12:48.189+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50412" doReceive() ... 2023-10-21T04:12:48.198+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50416" doAuth() ... 2023-10-21T04:12:48.199+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50416" doReceive() ... 2023-10-21T04:12:48.230+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50420" doAuth() ... 2023-10-21T04:12:48.230+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50420" doReceive() ... 2023-10-21T04:12:48.234+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50422" doAuth() ... 2023-10-21T04:12:48.234+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50422" doReceive() ... 2023-10-21T04:12:48.236+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50424" doAuth() ... 2023-10-21T04:12:48.237+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50424" doReceive() ... 2023-10-21T04:12:48.262+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50434" doAuth() ... 2023-10-21T04:12:48.262+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50434" doReceive() ... 2023-10-21T04:12:48.264+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50436" doAuth() ... 2023-10-21T04:12:48.265+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50436" doReceive() ... 2023-10-21T04:12:48.268+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50438" doAuth() ... 2023-10-21T04:12:48.268+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50438" doReceive() ... 2023-10-21T04:12:48.279+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50442" doAuth() ... 2023-10-21T04:12:48.279+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50442" doReceive() ... 2023-10-21T04:12:48.287+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50446" doAuth() ... 2023-10-21T04:12:48.289+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50446" doReceive() ... 2023-10-21T04:12:48.396+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50458" doAuth() ... 2023-10-21T04:12:48.398+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50458" doReceive() ... 2023-10-21T04:12:48.411+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50460" doAuth() ... 2023-10-21T04:12:48.413+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50460" doReceive() ... 2023-10-21T04:12:48.422+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50464" doAuth() ... 2023-10-21T04:12:48.424+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50464" doReceive() ... 2023-10-21T04:12:48.427+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50466" doAuth() ... 2023-10-21T04:12:48.429+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50466" doReceive() ... 2023-10-21T04:12:52.098+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50516" doAuth() ... 2023-10-21T04:12:52.100+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50516" doReceive() ... 2023-10-21T04:12:52.102+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50518" doAuth() ... 2023-10-21T04:12:52.105+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50518" doReceive() ... 2023-10-21T04:12:52.111+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50522" doAuth() ... 2023-10-21T04:12:52.113+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50522" doReceive() ... 2023-10-21T04:12:52.129+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50528" doAuth() ... 2023-10-21T04:12:52.130+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50528" doReceive() ... 2023-10-21T04:12:52.137+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50532" doAuth() ... 2023-10-21T04:12:52.138+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50532" doReceive() ... 2023-10-21T04:12:52.161+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50538" doAuth() ... 2023-10-21T04:12:52.163+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50538" doReceive() ... 2023-10-21T04:12:52.165+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50540" doAuth() ... 2023-10-21T04:12:52.167+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50540" doReceive() ... 2023-10-21T04:12:52.175+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50542" doAuth() ... 2023-10-21T04:12:52.177+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50542" doReceive() ... 2023-10-21T04:12:52.366+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50554" doAuth() ... 2023-10-21T04:12:52.366+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50554" doReceive() ... 2023-10-21T04:12:52.368+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50556" doAuth() ... 2023-10-21T04:12:52.369+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50556" doReceive() ... 2023-10-21T04:12:52.394+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50558" doAuth() ... 2023-10-21T04:12:52.395+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50558" doReceive() ... 2023-10-21T04:12:52.414+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50564" doAuth() ... 2023-10-21T04:12:52.415+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50564" doReceive() ... 2023-10-21T04:12:52.449+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50574" doAuth() ... 2023-10-21T04:12:52.450+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50574" doReceive() ... 2023-10-21T04:12:52.460+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50576" doAuth() ... 2023-10-21T04:12:52.461+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50576" doReceive() ... 2023-10-21T04:12:52.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50578" doAuth() ... 2023-10-21T04:12:52.465+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50578" doReceive() ... 2023-10-21T04:12:52.467+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50580" doAuth() ... 2023-10-21T04:12:52.468+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50580" doReceive() ... 2023-10-21T04:12:52.479+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50582" doAuth() ... 2023-10-21T04:12:52.480+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50582" doReceive() ... 2023-10-21T04:12:52.503+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50592" doAuth() ... 2023-10-21T04:12:52.504+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50592" doReceive() ... 2023-10-21T04:12:52.511+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50596" doAuth() ... 2023-10-21T04:12:52.512+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50596" doReceive() ... 2023-10-21T04:12:52.548+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50600" doAuth() ... 2023-10-21T04:12:52.549+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50600" doReceive() ... 2023-10-21T04:12:52.557+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50602" doAuth() ... 2023-10-21T04:12:52.557+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50602" doReceive() ... 2023-10-21T04:12:52.732+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50628" doAuth() ... 2023-10-21T04:12:52.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50628" doReceive() ... 2023-10-21T04:12:52.778+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50632" doAuth() ... 2023-10-21T04:12:52.778+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50632" doReceive() ... 2023-10-21T04:12:52.793+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50634" doAuth() ... 2023-10-21T04:12:52.793+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50634" doReceive() ... 2023-10-21T04:12:52.795+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50636" doAuth() ... 2023-10-21T04:12:52.796+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50636" doReceive() ... 2023-10-21T04:12:52.803+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50638" doAuth() ... 2023-10-21T04:12:52.804+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50638" doReceive() ... 2023-10-21T04:12:52.816+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50640" doAuth() ... 2023-10-21T04:12:52.817+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50640" doReceive() ... 2023-10-21T04:12:52.828+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50644" doAuth() ... 2023-10-21T04:12:52.829+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50644" doReceive() ... 2023-10-21T04:12:52.835+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50648" doAuth() ... 2023-10-21T04:12:52.835+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50648" doReceive() ... 2023-10-21T04:12:52.870+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50656" doAuth() ... 2023-10-21T04:12:52.870+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50656" doReceive() ... 2023-10-21T04:12:52.883+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50664" doAuth() ... 2023-10-21T04:12:52.884+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50664" doReceive() ... 2023-10-21T04:12:52.897+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50670" doAuth() ... 2023-10-21T04:12:52.897+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50670" doReceive() ... 2023-10-21T04:12:52.908+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50672" doAuth() ... 2023-10-21T04:12:52.909+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50672" doReceive() ... 2023-10-21T04:12:52.928+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50676" doAuth() ... 2023-10-21T04:12:52.929+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50676" doReceive() ... 2023-10-21T04:12:52.933+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50678" doAuth() ... 2023-10-21T04:12:52.935+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50678" doReceive() ... 2023-10-21T04:12:52.959+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50686" doAuth() ... 2023-10-21T04:12:52.960+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50686" doReceive() ... 2023-10-21T04:12:52.993+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50692" doAuth() ... 2023-10-21T04:12:52.994+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50692" doReceive() ... 2023-10-21T04:12:53.000+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50696" doAuth() ... 2023-10-21T04:12:53.001+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50696" doReceive() ... 2023-10-21T04:12:53.003+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50698" doAuth() ... 2023-10-21T04:12:53.004+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50698" doReceive() ... 2023-10-21T04:12:53.015+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50702" doAuth() ... 2023-10-21T04:12:53.016+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50702" doReceive() ... 2023-10-21T04:12:53.032+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50706" doAuth() ... 2023-10-21T04:12:53.033+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50706" doReceive() ... 2023-10-21T04:12:53.035+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50708" doAuth() ... 2023-10-21T04:12:53.036+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50708" doReceive() ... 2023-10-21T04:12:53.038+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50710" doAuth() ... 2023-10-21T04:12:53.038+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50710" doReceive() ... 2023-10-21T04:12:53.087+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50716" doAuth() ... 2023-10-21T04:12:53.088+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50716" doReceive() ... 2023-10-21T04:12:53.090+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50718" doAuth() ... 2023-10-21T04:12:53.090+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50718" doReceive() ... 2023-10-21T04:12:53.107+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50724" doAuth() ... 2023-10-21T04:12:53.108+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50724" doReceive() ... 2023-10-21T04:12:53.125+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50728" doAuth() ... 2023-10-21T04:12:53.126+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50728" doReceive() ... 2023-10-21T04:12:53.153+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50736" doAuth() ... 2023-10-21T04:12:53.155+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50736" doReceive() ... 2023-10-21T04:12:53.167+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50740" doAuth() ... 2023-10-21T04:12:53.168+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50740" doReceive() ... 2023-10-21T04:12:53.215+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50744" doAuth() ... 2023-10-21T04:12:53.216+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50744" doReceive() ... 2023-10-21T04:12:53.227+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50748" doAuth() ... 2023-10-21T04:12:53.227+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50748" doReceive() ... 2023-10-21T04:12:53.230+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50750" doAuth() ... 2023-10-21T04:12:53.230+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50750" doReceive() ... 2023-10-21T04:12:55.185+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 16338040286194952893 &{0} 2023-10-21T04:12:55.185+05:30 [Info] Indexer::handleDropIndex - IndexInstId 16338040286194952893 2023-10-21T04:12:55.186+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:12:55.186+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:12:55.186+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16338040286194952893 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:12:55.187+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [16338040286194952893] 2023-10-21T04:12:55.187+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [16338040286194952893] 2023-10-21T04:12:55.187+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:12:55.188+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [16338040286194952893] 2023-10-21T04:12:55.191+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0, IndexDefnId 5465835793930010953. 2023-10-21T04:12:55.191+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:12:55.192+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:12:55.192+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:12:55.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex] 2023-10-21T04:12:55.192+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex 2023-10-21T04:12:55.192+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex] 2023-10-21T04:12:55.192+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex] 2023-10-21T04:12:55.192+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex 2023-10-21T04:12:55.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex stopped 2023-10-21T04:12:55.192+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:12:55.192+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:12:55.192+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:12:55.192+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:12:55.188+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 62064 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:12:55.193+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 16338040286194952893 Defn: DefnId: 5465835793930010953 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 62064 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:12:55.193+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 8.060633ms 2023-10-21T04:12:55.193+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 16338040286194952893 2023-10-21T04:12:55.194+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId Instances [16338040286194952893] 2023-10-21T04:12:55.201+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId 2023-10-21T04:12:55.201+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 62064 2023-10-21T04:12:55.201+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 7.886357ms 2023-10-21T04:12:55.201+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 16.030534ms 2023-10-21T04:12:55.206+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:12:55.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex] 2023-10-21T04:12:55.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex closed 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Mainstore#16338040286194952893:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:12:55.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex] 2023-10-21T04:12:55.207+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex 2023-10-21T04:12:55.207+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex] 2023-10-21T04:12:55.207+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex] 2023-10-21T04:12:55.207+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex 2023-10-21T04:12:55.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex stopped 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:12:55.207+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:12:55.215+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:55.215+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:55.218+05:30 [Info] switched currmeta from 888 -> 891 force true 2023-10-21T04:12:55.225+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 43.289497ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:12:55.233+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:12:55.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex] 2023-10-21T04:12:55.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex closed 2023-10-21T04:12:55.234+05:30 [Info] bucket_1/idx_secondary/Backstore#16338040286194952893:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:12:55.234+05:30 [Info] Indexer::cleanupIndexData IndexInst 16338040286194952893 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:12:55.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex ... 2023-10-21T04:12:55.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : removed plasmaId 92 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex ... 2023-10-21T04:12:55.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:12:55.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/mainIndex sucessfully destroyed 2023-10-21T04:12:55.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex ... 2023-10-21T04:12:55.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : removed plasmaId 82 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex ... 2023-10-21T04:12:55.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:12:55.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_16338040286194952893_0.index destroyed 2023-10-21T04:12:55.240+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16338040286194952893, PartitionId 0, IndexDefnId 5465835793930010953. 2023-10-21T04:12:55.240+05:30 [Info] Indexer::cleanupIndexData IndexInst 16338040286194952893 Partition 0 Destroy Done 2023-10-21T04:12:55.266+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:55.266+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:55.286+05:30 [Info] switched currmeta from 891 -> 892 force true 2023-10-21T04:12:55.345+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 18361169415695152662 &{0} 2023-10-21T04:12:55.345+05:30 [Info] Indexer::handleDropIndex - IndexInstId 18361169415695152662 2023-10-21T04:12:55.348+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:12:55.349+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:12:55.349+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18361169415695152662 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:12:55.350+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [18361169415695152662] 2023-10-21T04:12:55.350+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [18361169415695152662] 2023-10-21T04:12:55.351+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:12:55.351+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [18361169415695152662] 2023-10-21T04:12:55.352+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 9997 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:12:55.352+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 18361169415695152662 Defn: DefnId: 12195715103909237684 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 9997 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:12:55.353+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 7.79225ms 2023-10-21T04:12:55.353+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0, IndexDefnId 12195715103909237684. 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:12:55.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex] 2023-10-21T04:12:55.353+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex] 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex] 2023-10-21T04:12:55.353+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex 2023-10-21T04:12:55.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex stopped 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:12:55.353+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:12:55.353+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 18361169415695152662 2023-10-21T04:12:55.354+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId Instances [18361169415695152662] 2023-10-21T04:12:55.359+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId 2023-10-21T04:12:55.359+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 9997 2023-10-21T04:12:55.359+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.015215ms 2023-10-21T04:12:55.359+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 13.886562ms 2023-10-21T04:12:55.362+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:12:55.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex] 2023-10-21T04:12:55.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex closed 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Mainstore#18361169415695152662:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:12:55.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex] 2023-10-21T04:12:55.363+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex] 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex] 2023-10-21T04:12:55.363+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex 2023-10-21T04:12:55.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex stopped 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:12:55.363+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:12:55.397+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 60.581286ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:12:55.401+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:12:55.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex] 2023-10-21T04:12:55.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex closed 2023-10-21T04:12:55.402+05:30 [Info] bucket_%2/idx_secondary/Backstore#18361169415695152662:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:12:55.403+05:30 [Info] Indexer::cleanupIndexData IndexInst 18361169415695152662 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:12:55.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex ... 2023-10-21T04:12:55.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : removed plasmaId 82 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex ... 2023-10-21T04:12:55.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:12:55.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/docIndex sucessfully destroyed 2023-10-21T04:12:55.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex ... 2023-10-21T04:12:55.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : removed plasmaId 92 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex ... 2023-10-21T04:12:55.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:12:55.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_18361169415695152662_0.index destroyed 2023-10-21T04:12:55.420+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 18361169415695152662, PartitionId 0, IndexDefnId 12195715103909237684. 2023-10-21T04:12:55.420+05:30 [Info] Indexer::cleanupIndexData IndexInst 18361169415695152662 Partition 0 Destroy Done 2023-10-21T04:12:55.479+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:55.479+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:55.483+05:30 [Info] switched currmeta from 892 -> 896 force true 2023-10-21T04:12:55.519+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:55.519+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:55.522+05:30 [Info] switched currmeta from 896 -> 896 force true 2023-10-21T04:12:56.233+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:12:56.238+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:12:56.239+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:12:56.241+05:30 [Info] switched currmeta from 896 -> 896 force true 2023-10-21T04:13:12.467+05:30 [Info] janitor: running cleanup. 2023-10-21T04:13:12.467+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[14889401051926000176 4860549750398470299 5465835793930010953 12195715103909237684]} 2023-10-21T04:13:12.473+05:30 [Info] memstats {"Alloc":261768688, "TotalAlloc":5367377224, "Sys":393007448, "Lookups":0, "Mallocs":60789839,"Frees":59748520, "HeapAlloc":261768688, "HeapSys":354156544, "HeapIdle":77840384, "HeapInuse":276316160,"HeapReleased":3145728, "HeapObjects":1041319,"MSpanInuse":2276160, "MSpanSys": 3117120, "StackInuse": 14942208,"GCSys":17286128, "LastGC":1697841730810888503,"PauseTotalNs":52292625, "PauseNs":[132814, 406504, 177746, 231817, 2463821, 133601, 168375, 307682, 136017], "NumGC":41} 2023-10-21T04:13:12.486+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:13:13.241+05:30 [Info] connected with 2 indexers 2023-10-21T04:13:13.241+05:30 [Info] client stats current counts: current: 72, not current: 0 2023-10-21T04:13:13.309+05:30 [Info] num concurrent scans {0} 2023-10-21T04:13:13.309+05:30 [Info] average scan response {0 ms} 2023-10-21T04:13:13.498+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:13:14.658+05:30 [Info] Indexer::ReadMemstats Time Taken 238.809µs 2023-10-21T04:13:16.285+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 4535197604682045916 2023-10-21T04:13:16.285+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 1.049654ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:16.322+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52712 2023-10-21T04:13:16.325+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52712 2023-10-21T04:13:16.332+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52712 2023-10-21T04:13:16.462+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 4535197604682045916, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:13:16.462+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 4535197604682045916, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:13:16.507+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 3234500417931906398, indexDefn DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:13:16.508+05:30 [Info] Indexer::handleCreateIndex InstId: 3234500417931906398 Defn: DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:13:16.508+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3234500417931906398 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:13:16.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : metadata saved successfully 2023-10-21T04:13:16.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data/recovery) 2023-10-21T04:13:16.519+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:13:16.519+05:30 [Info] bucket_1/idx_secondary/Backstore#3234500417931906398:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:13:16.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Assign plasmaId 83 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/docIndex 2023-10-21T04:13:16.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529 2023-10-21T04:13:16.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529/data 2023-10-21T04:13:16.529+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/docIndex 2023-10-21T04:13:16.529+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/docIndex 2023-10-21T04:13:16.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard3268862397300335529(3268862397300335529)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/docIndex started 2023-10-21T04:13:16.530+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:16.530+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:16.530+05:30 [Info] bucket_1/idx_secondary/Backstore#3234500417931906398:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:13:16.531+05:30 [Info] bucket_1/idx_secondary/Backstore#3234500417931906398:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:13:16.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : metadata saved successfully 2023-10-21T04:13:16.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data/recovery) 2023-10-21T04:13:16.534+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:13:16.534+05:30 [Info] switched currmeta from 896 -> 899 force true 2023-10-21T04:13:16.534+05:30 [Info] bucket_1/idx_secondary/Mainstore#3234500417931906398:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:13:16.543+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:16.543+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:16.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Assign plasmaId 93 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/mainIndex 2023-10-21T04:13:16.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314 2023-10-21T04:13:16.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314/data 2023-10-21T04:13:16.545+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/mainIndex 2023-10-21T04:13:16.545+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/mainIndex 2023-10-21T04:13:16.547+05:30 [Info] switched currmeta from 899 -> 899 force true 2023-10-21T04:13:16.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10563635432419104314(10563635432419104314)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index/mainIndex started 2023-10-21T04:13:16.547+05:30 [Info] bucket_1/idx_secondary/Mainstore#3234500417931906398:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:13:16.547+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3234500417931906398 partitionId 0 WriterThreads 4 2023-10-21T04:13:16.550+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3234500417931906398 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3234500417931906398_0.index Index: 3234500417931906398 Partition: 0 2023-10-21T04:13:16.550+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3234500417931906398 2023-10-21T04:13:16.550+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:16.548+05:30 [Info] bucket_1/idx_secondary/Mainstore#3234500417931906398:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:13:16.551+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:13:16.551+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:16.551+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:16.552+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 44.154615ms 2023-10-21T04:13:16.552+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 44.205298ms 2023-10-21T04:13:16.552+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 3234500417931906398, partnShardIdMap: map[0:[10563635432419104314 3268862397300335529]], indexDefn DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:13:16.572+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:16.572+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:16.577+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [3234500417931906398] &{0} 2023-10-21T04:13:16.578+05:30 [Info] Indexer::handleBuildIndex [3234500417931906398] 2023-10-21T04:13:16.578+05:30 [Info] switched currmeta from 899 -> 901 force true 2023-10-21T04:13:16.578+05:30 [Info] bulkUpdateRState: Index instance 3234500417931906398 rstate moved to ACTIVE 2023-10-21T04:13:16.578+05:30 [Info] Indexer::handleBuildIndex Added Index: [3234500417931906398] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:13:16.582+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:16.583+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:16.583+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:16.584+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:13:16.584+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:16.584+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:16.584+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57461 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:16.584+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:13:16.584+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:13:16.585+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:13:16.585+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57461 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:16.585+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:13:16.585+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:16.585+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:16.585+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:13:16.585+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:13:16.585+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:16.585+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:16.585+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:13:16.585+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:11] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:13:16.585+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3234500417931906398 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:13:16.585+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:13:16.585+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [ InstId: 3234500417931906398 Defn: DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 57462 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:13:16.586+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:13:16.586+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:13:16.587+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3234500417931906398 Defn: DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:13:16.592+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 14.871133ms 2023-10-21T04:13:16.593+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [3234500417931906398] 2023-10-21T04:13:16.593+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 15.150946ms 2023-10-21T04:13:16.595+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:13:16.595+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:13:16.604+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 4535197604682045916 2023-10-21T04:13:16.604+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 142.630381ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:16.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:16.614+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:16.616+05:30 [Info] switched currmeta from 901 -> 903 force true 2023-10-21T04:13:16.625+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:16.625+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:16.627+05:30 [Info] switched currmeta from 903 -> 904 force true 2023-10-21T04:13:16.619+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.131496ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:16.637+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:16.647+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:16.657+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40400" doAuth() ... 2023-10-21T04:13:16.657+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40400" auth successful 2023-10-21T04:13:16.657+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:40400" +1 2023-10-21T04:13:16.662+05:30 [Info] switched currmeta from 904 -> 905 force true 2023-10-21T04:13:16.670+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:16.670+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:16.673+05:30 [Info] switched currmeta from 905 -> 905 force true 2023-10-21T04:13:16.691+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:13:16.691+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 24] Nodes:[f2b29e6bf10e9349c06775f41b68f8ff 31fd475f22d8ee0da906c31968c91305] IsBalanced:true Messages:[]} 2023-10-21T04:13:16.691+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 24] call. err: operation_canceled 2023-10-21T04:13:16.691+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:13:16.692+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 24] Tasks:[]} 2023-10-21T04:13:16.693+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 24] 2023-10-21T04:13:16.694+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 24] 2023-10-21T04:13:16.964+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 178840478062605 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.966+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 186517626268899 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.970+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 215374859564337 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.971+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 39552632604610 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.972+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 148075163102199 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.976+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 242327631330054 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.978+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 28605411869539 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.978+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 191903564840858 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.979+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 50961953886801 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.979+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 225280996447309 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.982+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 183148377680315 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.982+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 66881023717063 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.982+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 274410200699189 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.982+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 4429827396609 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.985+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 127185710837163 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.986+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 226814108379280 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.988+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 257031483891250 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.989+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 242578104468179 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.990+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 55290066231400 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.991+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 210306088158507 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.994+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 209771427122473 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.995+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 230239359684242 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:16.996+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 260036488774725 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.001+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 160961818008256 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.002+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 216462437971699 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.003+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 280874278672777 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.003+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 219132834127088 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.005+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 102578907139247 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.005+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 184056281482149 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.010+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 99433130332778 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.010+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 45957426777107 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.010+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 217381278987664 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.011+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 82469662608686 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.014+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 136958029391999 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.019+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 70213628416850 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.020+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 28417669987133 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.022+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 253845164956592 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.024+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 253494845579900 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.029+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 222555662236724 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.030+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 5915864305121 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.033+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 188107826286720 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.036+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 259381390638966 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.036+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 252931432434966 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.037+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 158545118983293 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.045+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 146267938201125 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.045+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 12301831552869 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.046+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 242994859770444 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.047+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 207367350302991 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.047+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 271074273345339 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.047+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 227378777358533 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.048+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 215467668600630 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.048+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 200436537417141 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.068+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 204867593700407 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.068+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 188718367400378 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.068+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 279362078165491 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.078+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 264835102027079 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.078+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 170056388266420 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.088+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 129411666769864 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.089+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 16044440367455 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.089+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 125009072873642 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.089+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 75566139929862 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 133748675726909 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 10527297481171 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 106932309485550 0 57462 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:17.109+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_1:_default:c1 InstanceIds [3234500417931906398] 2023-10-21T04:13:17.109+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 57462 2023-10-21T04:13:17.109+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:13:17.109+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:13:17.109+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:13:17.109+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:13:17.109+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:13:17.109+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:13:17.110+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:17.445+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:13:17.451+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3234500417931906398 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:17.451+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3234500417931906398 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.358255ms SnapOpenDur 115.791µs 2023-10-21T04:13:17.451+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 3234500417931906398 INDEX_STATE_CATCHUP 2023-10-21T04:13:17.451+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 3234500417931906398 true 2023-10-21T04:13:17.451+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 3234500417931906398 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 57462 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] 2023-10-21T04:13:17.451+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3234500417931906398, PartitionId 0 SnapshotId 1 Creating recovery point ... 2023-10-21T04:13:17.451+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 57462 2023-10-21T04:13:17.451+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 57462 2023-10-21T04:13:17.453+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:17.461+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:17.461+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:17.462+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3234500417931906398 Defn: DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:13:17.462+05:30 [Info] send progress stats to clients 2023-10-21T04:13:17.462+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:13:17.462+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:13:17.462+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:12] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:13:17.462+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 3234500417931906398 Defn: DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 57462 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:13:17.465+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:13:17.466+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:13:17.466+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:13:17.478+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:17.478+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:17.482+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:13:17.483+05:30 [Info] switched currmeta from 905 -> 906 force true 2023-10-21T04:13:17.483+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_1 2023-10-21T04:13:17.484+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57462 2023-10-21T04:13:17.484+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57462 2023-10-21T04:13:17.484+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:13:17.484+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:13:17.484+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 3234500417931906398 false 2023-10-21T04:13:17.484+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 3234500417931906398 2023-10-21T04:13:17.484+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:13:17.484+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 57462 2023-10-21T04:13:17.491+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 29.101149ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:17.512+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:17.512+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:17.516+05:30 [Info] switched currmeta from 906 -> 907 force true 2023-10-21T04:13:17.525+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3234500417931906398, PartitionId 0 SnapshotId 1 Created recovery point (took 73.787348ms) 2023-10-21T04:13:17.530+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:13:17.530+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:13:17.531+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:13:17.536+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39656 2023-10-21T04:13:17.537+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:13:17.557+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 10.213031ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:17.558+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:13:17.558+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:13:17.558+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:13:17.558+05:30 [Info] DDLServiceMgr: processing create index token /indexing/ddl/commandToken/create/4535197604682045916/0 2023-10-21T04:13:17.563+05:30 [Info] DDLServiceMgr: Update Replica Count. Index Defn 4535197604682045916 replica Count {true 1 0 0} 2023-10-21T04:13:17.564+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 24.017µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:17.571+05:30 [Info] DDLServiceMgr: Remove create index token 4535197604682045916. 2023-10-21T04:13:17.571+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:13:17.571+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:13:17.571+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:13:17.571+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:59596->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:13:17.572+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:39656->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:13:17.572+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39656. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:17.572+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5f:7c:a4:b0:82:cc:cb:df as follower. 2023-10-21T04:13:17.664+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39664 2023-10-21T04:13:17.685+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 14.043145ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:17.690+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39664. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:17.692+05:30 [Info] messageListener.start(): message channel closed. Remove peer d6:13:fb:db:27:97:d3:55 as follower. 2023-10-21T04:13:18.059+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:13:18.110+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:13:18.110+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:13:18.110+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 3234500417931906398 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 57462, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 55 0 55} { 1 a2a78b2a980d 2 55 0 55} { 2 86ac6f7f2bf7 2 54 0 54} { 3 c3e1ddff7531 2 35 0 35} { 4 1a043769c363 2 58 0 58} { 5 23f9109dc7c2 2 52 0 52} { 6 dc654c627b06 2 62 0 62} { 7 ae8907d03f9a 2 63 0 63} { 8 2e5981289251 2 45 0 45} { 9 cce451c4944d 2 53 0 53} { 10 f9931a891135 2 54 0 54} { 11 3cd3f3a92ec7 2 51 0 51} { 12 4076649d401 2 42 0 42} { 13 a6928dfe29bb 2 46 0 46} { 14 ce49465c2890 2 72 0 72} { 15 73acbb1d6dab 2 50 0 50} { 16 e9c4ce4b1e32 2 56 0 56} { 17 324938c1e068 2 51 0 51} { 18 dc9f9dbf42d3 2 63 0 63} { 19 bf45b3adc52b 2 48 0 48} { 20 d166c73f4e92 2 50 0 50} { 21 bec93765cd29 2 48 0 48} { 22 ec80768ac445 2 51 0 51} { 23 c4df16b8f6f3 2 57 0 57} { 24 9264d7fc1ac0 2 59 0 59} { 25 ff74239b5989 2 71 0 71} { 26 c74cd6bc9cf0 2 52 0 52} { 27 a765f146c3a5 2 49 0 49} { 28 5d4b83524caf 2 58 0 58} { 29 29cc4c190c13 2 57 0 57} { 30 5a6f1469926a 2 58 0 58} { 31 c5b505ea3590 2 67 0 67} { 32 4b0176f1cd2e 2 60 0 60} { 33 7c9006b0e87f 2 65 0 65} { 34 19d8811ffb3d 2 58 0 58} { 35 3fdbe2615352 2 47 0 47} { 36 e6deeee29bb0 2 58 0 58} { 37 e68d5e39227c 2 54 0 54} { 38 ab15442a5080 2 60 0 60} { 39 ca69c730bc34 2 67 0 67} { 40 56164fd8de1 2 60 0 60} { 41 ebe7efa67f76 2 58 0 58} { 42 e60a302ecd16 2 46 0 46} { 43 8507a87f9e25 2 55 0 55} { 44 9032297f747d 2 67 0 67} { 45 b303e4cbf65 2 64 0 64} { 46 dd00a64c824c 2 54 0 54} { 47 f68a65c53f3b 2 53 0 53} { 48 ceccbf4100c5 2 58 0 58} { 49 c3f779d8f736 2 34 0 34} { 50 bc997940b90f 2 64 0 64} { 51 b64bc49cddb5 2 54 0 54} { 52 aba36b4061ba 2 56 0 56} { 53 ba5374260437 2 56 0 56} { 54 fe140d6ec9f3 2 55 0 55} { 55 9aaa569d49b4 2 70 0 70} { 56 f0ddba2e2d47 2 52 0 52} { 57 75b3006d77c8 2 55 0 55} { 58 e97a32b315f 2 53 0 53} { 59 71b1f161e0aa 2 53 0 53} { 60 44ba1ced7506 2 56 0 56} { 61 61411e5267ee 2 53 0 53} { 62 99313d3a9d3 2 57 0 57} { 63 79a4ca4e4e3d 2 56 0 56} 2023-10-21T04:13:18.110+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 3234500417931906398 INDEX_STATE_ACTIVE 2023-10-21T04:13:18.110+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:13:18.110+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:13:18.110+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:13:18.110+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 3 CurrMaint 1 New 0 2023-10-21T04:13:18.110+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 57462 2023-10-21T04:13:18.111+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:18.111+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:18.111+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3234500417931906398 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:13:18.112+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57462 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:18.112+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3234500417931906398 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:13:18.112+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:13:18.112+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:13:18.112+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:13:18.112+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 57462 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:18.112+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:13:18.112+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:18 1:14 2:15 3:7 4:19 5:8 6:18 7:18 8:9 9:11 10:17 11:12 12:14 13:15 14:21 15:12 16:10 17:12 18:18 19:8 20:15 21:8 22:14 23:13 24:11 25:23 26:11 27:16 28:17 29:18 30:14 31:19 32:17 33:17 34:17 35:15 36:14 37:15 38:16 39:25 40:22 41:17 42:11 43:13 44:33 45:16 46:17 47:14 48:17 49:14 50:22 51:23 52:17 53:13 54:23 55:22 56:10 57:13 58:14 59:13 60:16 61:14 62:19 63:16] 2023-10-21T04:13:18.113+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:36 1:34 2:36 3:21 4:38 5:38 6:46 7:46 8:34 9:31 10:42 11:34 12:34 13:31 14:48 15:36 16:35 17:34 18:45 19:29 20:35 21:32 22:32 23:34 24:36 25:50 26:29 27:35 28:36 29:38 30:44 31:44 32:41 33:43 34:41 35:37 36:37 37:41 38:38 39:46 40:43 41:38 42:34 43:34 44:55 45:41 46:42 47:37 48:42 49:28 50:47 51:43 52:40 53:38 54:45 55:46 56:39 57:38 58:33 59:41 60:41 61:36 62:41 63:40] 2023-10-21T04:13:18.113+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:13:18.113+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:40400: use of closed network connection 2023-10-21T04:13:18.113+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:40400" exit: read tcp4 127.0.0.1:9121->127.0.0.1:40400: use of closed network connection 2023-10-21T04:13:18.113+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40400" closed ! 2023-10-21T04:13:18.113+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:13:18.113+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:18.113+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:18.113+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:18.113+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:18.113+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:13:18.113+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3234500417931906398 Defn: DefnId: 4535197604682045916 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:13:18.114+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 57462 2023-10-21T04:13:18.114+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:13:18.129+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.278903ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:18.136+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:18.136+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:18.138+05:30 [Info] switched currmeta from 907 -> 908 force true 2023-10-21T04:13:18.157+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:13:18.157+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:13:18.157+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:13:18.157+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57462 2023-10-21T04:13:18.157+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 57462 2023-10-21T04:13:18.157+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:13:18.185+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:18.185+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:18.187+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:13:18.203+05:30 [Info] switched currmeta from 908 -> 909 force true 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.591+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3234500417931906398:[0]]. New rollbacktime: 1697841557312484841, Old rollbacktime: 0 2023-10-21T04:13:18.935+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.936+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.937+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:18.938+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14894967726271717657:[0]]. New rollbacktime: 1697841701991805708, Old rollbacktime: 0 2023-10-21T04:13:19.576+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:13:19.576+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17148287981272169168 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:19.576+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:132 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.97µs SnapOpenDur 121.338µs 2023-10-21T04:13:19.576+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4004220452657849790 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:19.576+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17148287981272169168 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.908µs SnapOpenDur 100.792µs 2023-10-21T04:13:19.576+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4004220452657849790 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.691µs SnapOpenDur 73.128µs 2023-10-21T04:13:19.576+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 4 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:13:19.576+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:124 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.475µs SnapOpenDur 38.191µs 2023-10-21T04:13:19.576+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 5 Creating New Snapshot SnapshotInfo: count:111 committed:true 2023-10-21T04:13:19.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.576+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:111 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.15µs SnapOpenDur 31.158µs 2023-10-21T04:13:19.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 6 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:140 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.238µs SnapOpenDur 33.505µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:125 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.054µs SnapOpenDur 38.694µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3810060733504350542 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 8 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3810060733504350542 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.972µs SnapOpenDur 54.947µs 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.464µs SnapOpenDur 37.696µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:13:19.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.25µs SnapOpenDur 36.921µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1532172649864318584 PartitionId 2 Creating New Snapshot SnapshotInfo: count:113 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1532172649864318584 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:113 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.006µs SnapOpenDur 36.723µs 2023-10-21T04:13:19.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.576+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.899µs SnapOpenDur 648.64µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11160011617777336020 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 2 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11160011617777336020 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.403µs SnapOpenDur 58.66µs 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:120 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.831µs SnapOpenDur 27.133µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 3 Creating New Snapshot SnapshotInfo: count:139 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:139 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.569µs SnapOpenDur 33.85µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 5 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:131 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.232µs SnapOpenDur 54.41µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 4 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:112 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.19µs SnapOpenDur 40.875µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 6 Creating New Snapshot SnapshotInfo: count:115 committed:true 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 5 Creating New Snapshot SnapshotInfo: count:133 committed:true 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:115 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.182µs SnapOpenDur 43.912µs 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:133 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.713µs SnapOpenDur 29.812µs 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 7 Creating New Snapshot SnapshotInfo: count:109 committed:true 2023-10-21T04:13:19.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.577+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:109 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.51µs SnapOpenDur 48.611µs 2023-10-21T04:13:19.578+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 8 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:13:19.578+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:128 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.35µs SnapOpenDur 37.969µs 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 1 Creating New Snapshot SnapshotInfo: count:135 committed:true 2023-10-21T04:13:19.578+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:135 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.236µs SnapOpenDur 34.993µs 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 2 Creating New Snapshot SnapshotInfo: count:107 committed:true 2023-10-21T04:13:19.578+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:107 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.437µs SnapOpenDur 42.633µs 2023-10-21T04:13:19.578+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 3 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:138 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.22µs SnapOpenDur 30.634µs 2023-10-21T04:13:19.578+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2170849387178940761 PartitionId 4 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:13:19.578+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2170849387178940761 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:137 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.895µs SnapOpenDur 28.011µs 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3234500417931906398 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:19.578+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3234500417931906398 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.616µs SnapOpenDur 56.785µs 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3292093078969249751 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:19.578+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3292093078969249751 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.288µs SnapOpenDur 1.267712ms 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.577+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 6 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:13:19.579+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.223µs SnapOpenDur 1.137856ms 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.579+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 7 Creating New Snapshot SnapshotInfo: count:119 committed:true 2023-10-21T04:13:19.579+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:119 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.004µs SnapOpenDur 44.07µs 2023-10-21T04:13:19.579+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16726320877690809560 PartitionId 8 Creating New Snapshot SnapshotInfo: count:122 committed:true 2023-10-21T04:13:19.579+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16726320877690809560 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:122 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.336µs SnapOpenDur 47.193µs 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3234500417931906398, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:13:19.579+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13258659765240018889 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:13:19.579+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13258659765240018889 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:21 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.949µs SnapOpenDur 53.479µs 2023-10-21T04:13:19.579+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 SnapshotId 5 Creating recovery point ... 2023-10-21T04:13:19.593+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 SnapshotId 4 Created recovery point (took 16.567192ms) 2023-10-21T04:13:19.594+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17148287981272169168, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:19.637+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 SnapshotId 4 Created recovery point (took 59.756556ms) 2023-10-21T04:13:19.638+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3810060733504350542, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:19.672+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 SnapshotId 4 Created recovery point (took 94.946365ms) 2023-10-21T04:13:19.673+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:13:19.674+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:13:19.778+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 SnapshotId 4 Created recovery point (took 200.302037ms) 2023-10-21T04:13:19.779+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:13:19.779+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:13:19.858+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 SnapshotId 4 Created recovery point (took 279.253785ms) 2023-10-21T04:13:19.859+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11160011617777336020, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:20.073+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 SnapshotId 5 Created recovery point (took 493.909836ms) 2023-10-21T04:13:20.074+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13258659765240018889, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:13:20.094+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 SnapshotId 4 Created recovery point (took 517.64072ms) 2023-10-21T04:13:20.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:13:20.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:13:20.119+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 SnapshotId 4 Created recovery point (took 542.372469ms) 2023-10-21T04:13:20.120+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:20.121+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4004220452657849790, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:20.139+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 SnapshotId 4 Created recovery point (took 561.557893ms) 2023-10-21T04:13:20.140+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:20.140+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3292093078969249751, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:20.156+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 SnapshotId 4 Created recovery point (took 578.08901ms) 2023-10-21T04:13:20.157+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:13:20.157+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:13:20.170+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 SnapshotId 4 Created recovery point (took 592.176266ms) 2023-10-21T04:13:20.171+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:13:20.171+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:13:20.190+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 SnapshotId 4 Created recovery point (took 612.159761ms) 2023-10-21T04:13:20.193+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:13:20.193+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:13:20.212+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 SnapshotId 4 Created recovery point (took 634.093421ms) 2023-10-21T04:13:20.213+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:13:20.213+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:13:20.244+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 SnapshotId 4 Created recovery point (took 665.98168ms) 2023-10-21T04:13:20.245+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:13:20.245+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:13:20.267+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 SnapshotId 4 Created recovery point (took 689.214744ms) 2023-10-21T04:13:20.268+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:13:20.268+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:13:20.280+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 SnapshotId 4 Created recovery point (took 701.835462ms) 2023-10-21T04:13:20.281+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:13:20.282+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:13:20.289+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 SnapshotId 4 Created recovery point (took 710.398949ms) 2023-10-21T04:13:20.289+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:13:20.290+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1532172649864318584, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:13:20.300+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 SnapshotId 4 Created recovery point (took 722.058956ms) 2023-10-21T04:13:20.301+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:13:20.302+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:13:20.314+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 SnapshotId 4 Created recovery point (took 735.882276ms) 2023-10-21T04:13:20.315+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:13:20.316+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:13:20.331+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 SnapshotId 4 Created recovery point (took 752.331684ms) 2023-10-21T04:13:20.332+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:13:20.332+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:13:20.352+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 SnapshotId 4 Created recovery point (took 773.417274ms) 2023-10-21T04:13:20.353+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:13:20.353+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:13:20.377+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 SnapshotId 4 Created recovery point (took 798.577168ms) 2023-10-21T04:13:20.378+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:13:20.378+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:13:20.392+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 SnapshotId 4 Created recovery point (took 813.805148ms) 2023-10-21T04:13:20.393+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:13:20.394+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:13:20.402+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 SnapshotId 4 Created recovery point (took 823.673257ms) 2023-10-21T04:13:20.403+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:13:20.404+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:13:20.415+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 SnapshotId 4 Created recovery point (took 836.848019ms) 2023-10-21T04:13:20.416+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:13:20.417+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:13:20.425+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 SnapshotId 4 Created recovery point (took 846.838304ms) 2023-10-21T04:13:20.427+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:13:20.427+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:13:20.450+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 SnapshotId 4 Created recovery point (took 871.487429ms) 2023-10-21T04:13:20.452+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:13:20.452+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:13:20.473+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 SnapshotId 4 Created recovery point (took 893.903371ms) 2023-10-21T04:13:20.474+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:13:20.475+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:13:20.493+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 SnapshotId 4 Created recovery point (took 914.040465ms) 2023-10-21T04:13:20.494+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:13:20.495+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2170849387178940761, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:13:20.513+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 SnapshotId 4 Created recovery point (took 933.738109ms) 2023-10-21T04:13:20.514+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:13:20.514+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16726320877690809560, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:13:20.523+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3234500417931906398, PartitionId 0 SnapshotId 2 Created recovery point (took 943.993951ms) 2023-10-21T04:13:21.374+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:13:21.400+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:21.400+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:21.403+05:30 [Info] switched currmeta from 909 -> 909 force true 2023-10-21T04:13:25.593+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 4138047918192025493 2023-10-21T04:13:25.594+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 1.292974ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:25.633+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52712 2023-10-21T04:13:25.639+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52712 2023-10-21T04:13:25.714+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52712 2023-10-21T04:13:25.838+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 4138047918192025493, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:13:25.838+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 4138047918192025493, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:13:25.864+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 13895028351589205356, indexDefn DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:13:25.865+05:30 [Info] Indexer::handleCreateIndex InstId: 13895028351589205356 Defn: DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:13:25.866+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13895028351589205356 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:13:25.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : metadata saved successfully 2023-10-21T04:13:25.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data/recovery) 2023-10-21T04:13:25.877+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:13:25.877+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13895028351589205356:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:13:25.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : metadata saved successfully 2023-10-21T04:13:25.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data/recovery) 2023-10-21T04:13:25.888+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:13:25.889+05:30 [Info] bucket_%2/idx_secondary/Backstore#13895028351589205356:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:13:25.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Assign plasmaId 93 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/mainIndex 2023-10-21T04:13:25.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837 2023-10-21T04:13:25.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837/data 2023-10-21T04:13:25.893+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/mainIndex 2023-10-21T04:13:25.893+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/mainIndex 2023-10-21T04:13:25.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6611724767191156837(6611724767191156837)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/mainIndex started 2023-10-21T04:13:25.893+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13895028351589205356:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:13:25.894+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13895028351589205356:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:13:25.904+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:25.904+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:25.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Assign plasmaId 83 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/docIndex 2023-10-21T04:13:25.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617 2023-10-21T04:13:25.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617/data 2023-10-21T04:13:25.905+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/docIndex 2023-10-21T04:13:25.905+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/docIndex 2023-10-21T04:13:25.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15964300471522801617(15964300471522801617)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index/docIndex started 2023-10-21T04:13:25.907+05:30 [Info] switched currmeta from 909 -> 912 force true 2023-10-21T04:13:25.908+05:30 [Info] bucket_%2/idx_secondary/Backstore#13895028351589205356:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:13:25.908+05:30 [Info] bucket_%2/idx_secondary/Backstore#13895028351589205356:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:13:25.908+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13895028351589205356 partitionId 0 WriterThreads 4 2023-10-21T04:13:25.908+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13895028351589205356 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13895028351589205356_0.index Index: 13895028351589205356 Partition: 0 2023-10-21T04:13:25.909+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13895028351589205356 2023-10-21T04:13:25.909+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:25.912+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:13:25.913+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:25.913+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:25.913+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 48.698387ms 2023-10-21T04:13:25.913+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 48.752658ms 2023-10-21T04:13:25.913+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 13895028351589205356, partnShardIdMap: map[0:[6611724767191156837 15964300471522801617]], indexDefn DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:13:25.914+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:25.915+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:25.919+05:30 [Info] switched currmeta from 912 -> 913 force true 2023-10-21T04:13:25.928+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:25.930+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:25.933+05:30 [Info] switched currmeta from 913 -> 914 force true 2023-10-21T04:13:25.938+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [13895028351589205356] &{0} 2023-10-21T04:13:25.938+05:30 [Info] Indexer::handleBuildIndex [13895028351589205356] 2023-10-21T04:13:25.940+05:30 [Info] bulkUpdateRState: Index instance 13895028351589205356 rstate moved to ACTIVE 2023-10-21T04:13:25.940+05:30 [Info] Indexer::handleBuildIndex Added Index: [13895028351589205356] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:13:25.941+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:25.941+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:25.941+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:25.941+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:13:25.941+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:25.941+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:25.942+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11733 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:25.942+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:13:25.942+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:13:25.942+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:13:25.942+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11733 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:25.942+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:13:25.942+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:25.942+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:25.942+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:13:25.942+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:13:25.942+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:25.942+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:25.942+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:13:25.942+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:12] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:13:25.942+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 13895028351589205356 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:13:25.942+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:13:25.942+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [ InstId: 13895028351589205356 Defn: DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 11734 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:13:25.943+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:13:25.943+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:13:25.944+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13895028351589205356 Defn: DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:13:25.944+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 6.29441ms 2023-10-21T04:13:25.945+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 6.341821ms 2023-10-21T04:13:25.945+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [13895028351589205356] 2023-10-21T04:13:25.946+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:25.946+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:25.955+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:13:25.955+05:30 [Info] switched currmeta from 914 -> 916 force true 2023-10-21T04:13:25.957+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:13:25.977+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 4138047918192025493 2023-10-21T04:13:25.977+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 139.403527ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:25.991+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40610" doAuth() ... 2023-10-21T04:13:25.993+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.340934ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:25.998+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40610" auth successful 2023-10-21T04:13:26.003+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:40610" +1 2023-10-21T04:13:26.018+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:26.018+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:26.021+05:30 [Info] switched currmeta from 916 -> 918 force true 2023-10-21T04:13:26.034+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:26.035+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:26.037+05:30 [Info] switched currmeta from 918 -> 918 force true 2023-10-21T04:13:26.447+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:13:26.448+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:13:26.454+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:13:26.455+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:13:26.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 35403626585217 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 263268055970459 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.477+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 215749515705641 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.480+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 3874519919278 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.480+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 159113185399719 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.480+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 176639880977228 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.482+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 261493429196201 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.484+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 170288557775498 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.487+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 264975169139761 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.487+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 236398693961229 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.487+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:26.487+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:26.489+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 234758907181738 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.489+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 206533515052028 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.490+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 116104661063478 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.490+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:13:26.490+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:13:26.490+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:13:26.493+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 140236896721046 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.493+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 102350060311343 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.496+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 162158720181435 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.497+05:30 [Info] switched currmeta from 918 -> 918 force true 2023-10-21T04:13:26.498+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 200752780087280 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.500+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 98820809009123 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.500+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 207134287529256 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.500+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 208836052671833 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.502+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 27979933591532 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.502+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 92956357968546 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.509+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 192215088742243 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.510+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 154934968850942 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.510+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 271667630317209 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.512+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 120989102157727 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.512+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 168740244208559 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.513+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39882 2023-10-21T04:13:26.513+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:13:26.521+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 236953356346174 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.521+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 130231725986856 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.528+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 278426329668882 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.528+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 67362620867705 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.536+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 43594649874772 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.536+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 83647976044922 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.543+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5715487545729 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.547+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 24368176313355 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.547+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 100659545510465 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.547+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 60545336618717 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.547+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 181267443431019 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.547+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 153478150002150 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.547+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 73432771436458 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.549+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 45479338375128 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.549+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 116785907755771 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.549+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 51776324008509 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.550+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 249919763471169 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.550+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 166675095997741 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.552+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 49796766757346 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.556+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 124194574848641 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.556+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 14.730973ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:26.556+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:13:26.557+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:13:26.557+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:13:26.557+05:30 [Info] DDLServiceMgr: processing create index token /indexing/ddl/commandToken/create/4138047918192025493/0 2023-10-21T04:13:26.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 19274425379955 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 149022976193937 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.561+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 184487117398647 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.562+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 199728167534433 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 158076890855634 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 196777743741606 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.569+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 138935694816848 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.569+05:30 [Info] DDLServiceMgr: Update Replica Count. Index Defn 4138047918192025493 replica Count {true 1 0 0} 2023-10-21T04:13:26.570+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 24856128127173 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.570+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 29.489µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:26.572+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 61798873988101 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 191719959781498 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 29071415952852 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.576+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 140942236111856 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.576+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 103336286678362 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.580+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 119871837460216 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.580+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 56626109228904 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.580+05:30 [Info] DDLServiceMgr: Remove create index token 4138047918192025493. 2023-10-21T04:13:26.580+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:13:26.580+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:13:26.580+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:13:26.580+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:59822->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:13:26.580+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:39882->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:13:26.580+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39882. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:26.581+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4e:d9:f1:47:40:a6:69:a7 as follower. 2023-10-21T04:13:26.583+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 177105814270799 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.592+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 252537041863196 0 11734 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:13:26.611+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 bucket_%2:_default:c1 InstanceIds [13895028351589205356] 2023-10-21T04:13:26.611+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 11734 2023-10-21T04:13:26.611+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:13:26.611+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:13:26.612+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_%2:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:13:26.612+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 34 0 52} { 1 a0a72d5bdb4c 2 26 0 59} { 2 20330c961c81 0 0 0 50} { 3 c439193e7d29 2 45 0 45} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 0 0 0 0} { 6 edd3aef555a9 0 0 0 0} { 7 9ae064feda8a 0 0 0 52} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 47 87891d8f3591 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 49 a7ca41192277 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 51 b2f7e383f2a6 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} { 53 7e5c7cbc7650 0 0 0 0} { 54 169b44b000c5 0 0 0 0} { 55 3834abee7c05 0 0 0 0} { 56 1a70b76bc5d4 0 0 0 0} { 57 ae5e4819607a 0 0 0 0} { 58 5dfbdaa8ed5a 0 0 0 0} { 59 802fabead3f0 0 0 0 0} { 60 33804b202768 0 0 0 0} { 61 6d05d64a56f8 0 0 0 0} { 62 a113a925b34f 0 0 0 0} { 63 e5ae5cac421c 0 0 0 0} HWT bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 34 0 52} { 1 a0a72d5bdb4c 2 26 0 59} { 2 20330c961c81 0 0 0 50} { 3 c439193e7d29 2 45 0 45} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 0 0 0 0} { 6 edd3aef555a9 0 0 0 0} { 7 9ae064feda8a 0 0 0 52} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 47 87891d8f3591 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 49 a7ca41192277 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 51 b2f7e383f2a6 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} { 53 7e5c7cbc7650 0 0 0 0} { 54 169b44b000c5 0 0 0 0} { 55 3834abee7c05 0 0 0 0} { 56 1a70b76bc5d4 0 0 0 0} { 57 ae5e4819607a 0 0 0 0} { 58 5dfbdaa8ed5a 0 0 0 0} { 59 802fabead3f0 0 0 0 0} { 60 33804b202768 0 0 0 0} { 61 6d05d64a56f8 0 0 0 0} { 62 a113a925b34f 0 0 0 0} { 63 e5ae5cac421c 0 0 0 0} 2023-10-21T04:13:26.612+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:13:26.612+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:13:26.612+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:13:26.612+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:13:26.613+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:26.749+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39888 2023-10-21T04:13:26.799+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.435914ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:26.800+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39888. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:26.801+05:30 [Info] messageListener.start(): message channel closed. Remove peer af:3f:0:2:4b:73:40:d9 as follower. 2023-10-21T04:13:27.058+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:13:27.093+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:13:27.098+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13895028351589205356 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:27.098+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13895028351589205356 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.815µs SnapOpenDur 92.55µs 2023-10-21T04:13:27.098+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 13895028351589205356 INDEX_STATE_CATCHUP 2023-10-21T04:13:27.098+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 13895028351589205356 true 2023-10-21T04:13:27.099+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13895028351589205356, PartitionId 0 SnapshotId 1 Creating recovery point ... 2023-10-21T04:13:27.099+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 13895028351589205356 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 11734 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] 2023-10-21T04:13:27.099+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 11734 2023-10-21T04:13:27.099+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 11734 2023-10-21T04:13:27.101+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:27.101+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:27.101+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:27.102+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13895028351589205356 Defn: DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:13:27.102+05:30 [Info] send progress stats to clients 2023-10-21T04:13:27.102+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:13:27.102+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:13:27.102+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:12 bucket_1:12] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:13:27.102+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 13895028351589205356 Defn: DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 11734 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:13:27.106+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:13:27.109+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.402408ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:27.117+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceId bucket_%2 2023-10-21T04:13:27.117+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11734 2023-10-21T04:13:27.117+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11734 2023-10-21T04:13:27.117+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:13:27.117+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:13:27.117+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 13895028351589205356 false 2023-10-21T04:13:27.117+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 13895028351589205356 2023-10-21T04:13:27.117+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:13:27.117+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 11734 2023-10-21T04:13:27.127+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:27.127+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:27.131+05:30 [Info] switched currmeta from 918 -> 920 force true 2023-10-21T04:13:27.146+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:27.147+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:27.151+05:30 [Info] switched currmeta from 920 -> 920 force true 2023-10-21T04:13:27.184+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13895028351589205356, PartitionId 0 SnapshotId 1 Created recovery point (took 84.92692ms) 2023-10-21T04:13:27.300+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:13:27.613+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:13:27.613+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:13:27.613+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 13895028351589205356 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 11734, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 52 0 52} { 1 a0a72d5bdb4c 2 59 0 59} { 2 20330c961c81 2 50 0 50} { 3 c439193e7d29 2 45 0 45} { 4 90b66ce513a7 2 58 0 58} { 5 3861b622aae 2 54 0 54} { 6 edd3aef555a9 2 61 0 61} { 7 9ae064feda8a 2 52 0 52} { 8 f0fe56d4b431 2 56 0 56} { 9 d700dc2dce0d 2 62 0 62} { 10 bbd754cc83fc 2 50 0 50} { 11 d58311453aaa 2 56 0 56} { 12 6998b9145736 2 55 0 55} { 13 5d163afcd32f 2 53 0 53} { 14 7f8b7269d896 2 60 0 60} { 15 937b84e9e4bb 2 67 0 67} { 16 b69566250ff0 2 52 0 52} { 17 59e08337a7e3 2 44 0 44} { 18 bc6335a10528 2 46 0 46} { 19 bdef6ebc5d59 2 45 0 45} { 20 548b16b242a2 2 50 0 50} { 21 19729600a7ec 2 54 0 54} { 22 aed19015bf63 2 55 0 55} { 23 8ce99bc819fe 2 51 0 51} { 24 f7148c9a1e99 2 52 0 52} { 25 9977e650bbaf 2 59 0 59} { 26 6e09f874af9f 2 59 0 59} { 27 532bd9ad981 2 54 0 54} { 28 d78200a2233e 2 54 0 54} { 29 7671efc41428 2 49 0 49} { 30 3710cf482edd 2 46 0 46} { 31 42c966419faa 2 60 0 60} { 32 3d4415168479 2 67 0 67} { 33 fd3a2e79dd12 2 54 0 54} { 34 2f171d667a3d 2 60 0 60} { 35 1629a87e340b 2 50 0 50} { 36 27a62b9eb154 2 52 0 52} { 37 4c13cfe5f97a 2 59 0 59} { 38 a4dc9d93366b 2 53 0 53} { 39 5b8ca0741241 2 49 0 49} { 40 8b966a9c95e6 2 68 0 68} { 41 6a37568beafb 2 55 0 55} { 42 295cfbcef7d8 2 55 0 55} { 43 e34cfab95b41 2 65 0 65} { 44 979711e3cd2d 2 57 0 57} { 45 2d4a3697f1e2 2 59 0 59} { 46 70f44d845281 2 54 0 54} { 47 87891d8f3591 2 55 0 55} { 48 1187ad472c73 2 55 0 55} { 49 a7ca41192277 2 55 0 55} { 50 b5a6d67a3f61 2 66 0 66} { 51 b2f7e383f2a6 2 51 0 51} { 52 8fc524ed38d2 2 54 0 54} { 53 7e5c7cbc7650 2 61 0 61} { 54 169b44b000c5 2 60 0 60} { 55 3834abee7c05 2 67 0 67} { 56 1a70b76bc5d4 2 61 0 61} { 57 ae5e4819607a 2 63 0 63} { 58 5dfbdaa8ed5a 2 46 0 46} { 59 802fabead3f0 2 63 0 63} { 60 33804b202768 2 46 0 46} { 61 6d05d64a56f8 2 39 0 39} { 62 a113a925b34f 2 58 0 58} { 63 e5ae5cac421c 2 73 0 73} 2023-10-21T04:13:27.613+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 13895028351589205356 INDEX_STATE_ACTIVE 2023-10-21T04:13:27.613+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:13:27.613+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:13:27.613+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:13:27.614+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 4 CurrMaint 1 New 0 2023-10-21T04:13:27.614+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 11734 2023-10-21T04:13:27.614+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:27.614+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841557392848703 bucket_1:1697841557312484841] 2023-10-21T04:13:27.615+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13895028351589205356 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:13:27.615+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11734 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:27.615+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 13895028351589205356 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:13:27.615+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:13:27.615+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:13:27.615+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:13:27.615+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 11734 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:13:27.615+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:13:27.616+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:17 1:18 2:15 3:14 4:21 5:14 6:21 7:13 8:16 9:14 10:14 11:13 12:12 13:14 14:10 15:20 16:18 17:14 18:12 19:13 20:12 21:16 22:13 23:17 24:16 25:10 26:15 27:17 28:20 29:16 30:11 31:17 32:20 33:12 34:18 35:20 36:11 37:12 38:12 39:9 40:17 41:15 42:19 43:14 44:15 45:15 46:16 47:17 48:15 49:14 50:18 51:15 52:17 53:24 54:16 55:18 56:14 57:16 58:11 59:19 60:17 61:14 62:19 63:28] 2023-10-21T04:13:27.616+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:37 1:38 2:35 3:33 4:41 5:34 6:47 7:33 8:42 9:41 10:34 11:37 12:35 13:37 14:38 15:47 16:39 17:27 18:34 19:32 20:33 21:36 22:37 23:40 24:33 25:33 26:38 27:47 28:39 29:36 30:32 31:40 32:47 33:38 34:45 35:39 36:33 37:33 38:39 39:29 40:45 41:41 42:46 43:41 44:34 45:34 46:42 47:38 48:38 49:42 50:44 51:36 52:37 53:50 54:41 55:47 56:36 57:41 58:32 59:46 60:37 61:30 62:43 63:50] 2023-10-21T04:13:27.616+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:13:27.616+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:40610: use of closed network connection 2023-10-21T04:13:27.616+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:40610" exit: read tcp4 127.0.0.1:9121->127.0.0.1:40610: use of closed network connection 2023-10-21T04:13:27.617+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40610" closed ! 2023-10-21T04:13:27.617+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:13:27.617+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:27.617+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:27.617+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:13:27.617+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:13:27.618+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:13:27.618+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13895028351589205356 Defn: DefnId: 4138047918192025493 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:13:27.618+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 11734 2023-10-21T04:13:27.618+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:13:27.633+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_31fd475f22d8ee0da906c31968c91305 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:13:27.633+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:13:27.633+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:13:27.634+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11734 2023-10-21T04:13:27.634+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 11734 2023-10-21T04:13:27.634+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:13:27.642+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.289145ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:13:27.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:27.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:27.667+05:30 [Info] switched currmeta from 920 -> 921 force true 2023-10-21T04:13:27.723+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:27.725+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:27.744+05:30 [Info] switched currmeta from 921 -> 922 force true 2023-10-21T04:13:28.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.091+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.092+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13895028351589205356:[0]]. New rollbacktime: 1697841557392848703, Old rollbacktime: 0 2023-10-21T04:13:28.934+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[204369152472931536:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:13:28.934+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[204369152472931536:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:13:28.934+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[204369152472931536:[0]]. New rollbacktime: 1697841701703351034, Old rollbacktime: 0 2023-10-21T04:13:29.094+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11400459767051699632 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:29.094+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11400459767051699632 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3µs SnapOpenDur 92.848µs 2023-10-21T04:13:29.094+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2483169159760864439 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:29.094+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2483169159760864439 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.292µs SnapOpenDur 61.334µs 2023-10-21T04:13:29.094+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17128008689111072444 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:29.094+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17128008689111072444 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.834µs SnapOpenDur 66.383µs 2023-10-21T04:13:29.094+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.094+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 6 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:106 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.768µs SnapOpenDur 53.713µs 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 7 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.207µs SnapOpenDur 41.567µs 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 8 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.423µs SnapOpenDur 40.142µs 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 1 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:120 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.918µs SnapOpenDur 32.734µs 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 2 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.323µs SnapOpenDur 74.48µs 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.567µs SnapOpenDur 41.27µs 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 4 Creating New Snapshot SnapshotInfo: count:130 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:130 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.65µs SnapOpenDur 37.812µs 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9664682240473767939 PartitionId 5 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14607307599259773249 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9664682240473767939 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:128 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 6.283µs SnapOpenDur 40.885µs 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14607307599259773249 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.901µs SnapOpenDur 51.426µs 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 8 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:124 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.499µs SnapOpenDur 54.356µs 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14780363202971367008 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 1 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14780363202971367008 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:21 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.697µs SnapOpenDur 50.525µs 2023-10-21T04:13:29.095+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.589µs SnapOpenDur 205.649µs 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 2 Creating New Snapshot SnapshotInfo: count:110 committed:true 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17729593977321451597 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:29.095+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:110 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.308µs SnapOpenDur 36.735µs 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17729593977321451597 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.031µs SnapOpenDur 44.536µs 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13895028351589205356 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13895028351589205356 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.364µs SnapOpenDur 47.839µs 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 3 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13895028351589205356, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:13:29.095+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 6 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:137 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.052µs SnapOpenDur 1.016484ms 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 SnapshotId 5 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 7 Creating New Snapshot SnapshotInfo: count:102 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:102 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.014µs SnapOpenDur 35.024µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 8 Creating New Snapshot SnapshotInfo: count:148 committed:true 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:148 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.246µs SnapOpenDur 37.684µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 1 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:112 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.93µs SnapOpenDur 36.668µs 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.248µs SnapOpenDur 37.255µs 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.34µs SnapOpenDur 315.138µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.073µs SnapOpenDur 33.127µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 4 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:140 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.115µs SnapOpenDur 70.038µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 4 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:138 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.405µs SnapOpenDur 36.22µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 5 Creating New Snapshot SnapshotInfo: count:116 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:116 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.215µs SnapOpenDur 33.764µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7410154703733445404 PartitionId 5 Creating New Snapshot SnapshotInfo: count:117 committed:true 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7410154703733445404 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:117 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.493µs SnapOpenDur 43.205µs 2023-10-21T04:13:29.096+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:13:29.096+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:127 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.117µs SnapOpenDur 32.903µs 2023-10-21T04:13:29.096+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.097+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 80911002600697866 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:13:29.097+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.097+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 80911002600697866 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:125 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.4µs SnapOpenDur 34.915µs 2023-10-21T04:13:29.097+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.097+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.097+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.097+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.097+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.097+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:13:29.125+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 SnapshotId 4 Created recovery point (took 29.754727ms) 2023-10-21T04:13:29.127+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17128008689111072444, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:29.167+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 SnapshotId 4 Created recovery point (took 71.916401ms) 2023-10-21T04:13:29.169+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:13:29.169+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:13:29.219+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 SnapshotId 4 Created recovery point (took 124.010815ms) 2023-10-21T04:13:29.221+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14607307599259773249, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:29.248+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 SnapshotId 4 Created recovery point (took 152.635597ms) 2023-10-21T04:13:29.249+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17729593977321451597, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:29.351+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 SnapshotId 5 Created recovery point (took 255.477749ms) 2023-10-21T04:13:29.354+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14780363202971367008, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:13:29.446+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 SnapshotId 4 Created recovery point (took 348.924992ms) 2023-10-21T04:13:29.447+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:13:29.448+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:13:29.486+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 SnapshotId 4 Created recovery point (took 391.528659ms) 2023-10-21T04:13:29.487+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:29.488+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11400459767051699632, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:29.505+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 SnapshotId 4 Created recovery point (took 410.376196ms) 2023-10-21T04:13:29.507+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:29.507+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2483169159760864439, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:13:29.518+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 SnapshotId 4 Created recovery point (took 423.398971ms) 2023-10-21T04:13:29.519+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:13:29.520+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:13:29.528+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 SnapshotId 4 Created recovery point (took 432.856337ms) 2023-10-21T04:13:29.529+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:13:29.529+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:13:29.539+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 SnapshotId 4 Created recovery point (took 443.363247ms) 2023-10-21T04:13:29.540+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:13:29.540+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:13:29.545+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 SnapshotId 4 Created recovery point (took 449.615869ms) 2023-10-21T04:13:29.546+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:13:29.546+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:13:29.553+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 SnapshotId 4 Created recovery point (took 457.499344ms) 2023-10-21T04:13:29.554+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:13:29.555+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:13:29.562+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 SnapshotId 4 Created recovery point (took 466.213366ms) 2023-10-21T04:13:29.563+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:13:29.563+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:13:29.570+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 SnapshotId 4 Created recovery point (took 474.848584ms) 2023-10-21T04:13:29.571+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:13:29.574+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:13:29.577+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 SnapshotId 4 Created recovery point (took 480.956661ms) 2023-10-21T04:13:29.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:13:29.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:13:29.589+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 SnapshotId 4 Created recovery point (took 493.745308ms) 2023-10-21T04:13:29.590+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:13:29.591+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:13:29.613+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 SnapshotId 4 Created recovery point (took 517.167747ms) 2023-10-21T04:13:29.614+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:13:29.615+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:13:29.635+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 SnapshotId 4 Created recovery point (took 538.726932ms) 2023-10-21T04:13:29.635+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:13:29.636+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:13:29.655+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 SnapshotId 4 Created recovery point (took 559.279323ms) 2023-10-21T04:13:29.656+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:13:29.657+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9664682240473767939, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:13:29.667+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13895028351589205356, PartitionId 0 SnapshotId 2 Created recovery point (took 571.145105ms) 2023-10-21T04:13:29.682+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 SnapshotId 4 Created recovery point (took 585.708993ms) 2023-10-21T04:13:29.683+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:13:29.683+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:13:29.693+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 SnapshotId 4 Created recovery point (took 597.215669ms) 2023-10-21T04:13:29.694+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:13:29.695+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:13:29.700+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 SnapshotId 4 Created recovery point (took 603.431504ms) 2023-10-21T04:13:29.701+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:13:29.702+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:13:29.706+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 SnapshotId 4 Created recovery point (took 609.947723ms) 2023-10-21T04:13:29.707+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:13:29.708+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:13:29.714+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 SnapshotId 4 Created recovery point (took 617.499436ms) 2023-10-21T04:13:29.715+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:13:29.715+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:13:29.722+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 SnapshotId 4 Created recovery point (took 625.508432ms) 2023-10-21T04:13:29.723+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:13:29.724+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:13:29.729+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 SnapshotId 4 Created recovery point (took 632.121422ms) 2023-10-21T04:13:29.730+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:13:29.730+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:13:29.736+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 SnapshotId 4 Created recovery point (took 638.849422ms) 2023-10-21T04:13:29.737+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:13:29.737+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 80911002600697866, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:13:29.751+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 SnapshotId 4 Created recovery point (took 654.556114ms) 2023-10-21T04:13:29.752+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:13:29.753+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:13:29.769+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 SnapshotId 4 Created recovery point (took 672.380134ms) 2023-10-21T04:13:29.770+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:13:29.770+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7410154703733445404, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:13:31.487+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:13:31.493+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:31.493+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112 127.0.0.1:9118] 2023-10-21T04:13:31.497+05:30 [Info] switched currmeta from 922 -> 922 force true 2023-10-21T04:13:40.108+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:13:41.672+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:13:41.768+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:41.768+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112] 2023-10-21T04:13:41.768+05:30 [Info] UnwatchMetadata(): indexer 31fd475f22d8ee0da906c31968c91305 2023-10-21T04:13:41.768+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:13:41.768+05:30 [Info] UnwatchMetadata(): finish for indexer 31fd475f22d8ee0da906c31968c91305 2023-10-21T04:13:41.769+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:57118->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:13:41.769+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35506. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:41.769+05:30 [Info] messageListener.start(): message channel closed. Remove peer 88:23:86:51:5a:e:de:ca as follower. 2023-10-21T04:13:41.769+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57118. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:41.769+05:30 [Info] messageListener.start(): message channel closed. Remove peer 13:82:1b:22:84:c1:e5:f4 as follower. 2023-10-21T04:13:41.774+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:47102 closed 2023-10-21T04:13:41.775+05:30 [Info] switched currmeta from 922 -> 923 force true 2023-10-21T04:13:41.775+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:13:41.775+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] ... stopped 2023-10-21T04:13:41.775+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:40504 closed 2023-10-21T04:13:41.805+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57040. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:41.805+05:30 [Info] messageListener.start(): message channel closed. Remove peer 28:17:e4:a8:a6:18:3a:74 as follower. 2023-10-21T04:13:41.836+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:40490 closed 2023-10-21T04:13:41.839+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57036. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:41.839+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2b:8c:e7:c7:10:2c:1b:19 as follower. 2023-10-21T04:13:41.840+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:13:41.846+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57032. Reason = EOF. Cleaning Pipe. 2023-10-21T04:13:41.846+05:30 [Info] messageListener.start(): message channel closed. Remove peer c6:5a:9d:0:dd:d4:1f:d4 as follower. 2023-10-21T04:13:41.848+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:41.848+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112] 2023-10-21T04:13:41.854+05:30 [Info] switched currmeta from 923 -> 923 force true 2023-10-21T04:13:41.858+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:40472 closed 2023-10-21T04:13:41.864+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50670 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50192 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50640 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49590 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50582 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49912 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49570 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49522 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49444 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49186 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49646 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50338 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49246 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50332 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49840 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50110 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50522 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49244 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49898 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50564 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49156 closed 2023-10-21T04:13:41.865+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50750 closed 2023-10-21T04:13:41.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49810 closed 2023-10-21T04:13:41.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49942 closed 2023-10-21T04:13:41.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50416 closed 2023-10-21T04:13:41.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50708 closed 2023-10-21T04:13:41.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49498 closed 2023-10-21T04:13:41.868+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50342 closed 2023-10-21T04:13:41.868+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50202 closed 2023-10-21T04:13:41.868+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50628 closed 2023-10-21T04:13:41.868+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50592 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50330 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50436 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50362 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49110 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49892 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49882 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50518 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49764 closed 2023-10-21T04:13:41.869+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50072 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50692 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49124 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50600 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49924 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49062 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50320 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50064 closed 2023-10-21T04:13:41.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49178 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50664 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49452 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49148 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50466 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49600 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49644 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49180 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49414 closed 2023-10-21T04:13:41.871+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49412 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50434 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49864 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50068 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50048 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49562 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49744 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49782 closed 2023-10-21T04:13:41.872+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49610 closed 2023-10-21T04:13:41.873+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50702 closed 2023-10-21T04:13:41.873+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50460 closed 2023-10-21T04:13:41.873+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49112 closed 2023-10-21T04:13:41.873+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49626 closed 2023-10-21T04:13:41.874+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49794 closed 2023-10-21T04:13:41.875+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49500 closed 2023-10-21T04:13:41.875+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49612 closed 2023-10-21T04:13:41.875+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50012 closed 2023-10-21T04:13:41.876+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49672 closed 2023-10-21T04:13:41.876+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49140 closed 2023-10-21T04:13:41.877+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49058 closed 2023-10-21T04:13:41.877+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50328 closed 2023-10-21T04:13:41.877+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50676 closed 2023-10-21T04:13:41.882+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50696 closed 2023-10-21T04:13:41.882+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49888 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50540 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49950 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50698 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49624 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49580 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50602 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50054 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49576 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49256 closed 2023-10-21T04:13:41.883+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50368 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50098 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49418 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49092 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50322 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50632 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49066 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49154 closed 2023-10-21T04:13:41.884+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49520 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50154 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50424 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50438 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50516 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50088 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49426 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49430 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49932 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49210 closed 2023-10-21T04:13:41.885+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49638 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49518 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50724 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50042 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50026 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50446 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49752 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49152 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50206 closed 2023-10-21T04:13:41.886+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49860 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49074 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49072 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50596 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50008 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49532 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49242 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49424 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50410 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49138 closed 2023-10-21T04:13:41.887+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49760 closed 2023-10-21T04:13:41.888+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49544 closed 2023-10-21T04:13:41.888+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49548 closed 2023-10-21T04:13:41.888+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49476 closed 2023-10-21T04:13:41.888+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50710 closed 2023-10-21T04:13:41.888+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50554 closed 2023-10-21T04:13:41.888+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50538 closed 2023-10-21T04:13:41.888+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:45562 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50740 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49654 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49232 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49652 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50580 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49098 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49252 closed 2023-10-21T04:13:41.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49660 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50458 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50302 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49648 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50408 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50292 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50126 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49900 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49852 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49096 closed 2023-10-21T04:13:41.890+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49870 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49422 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49656 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49486 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49212 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50638 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49836 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50442 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50576 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49596 closed 2023-10-21T04:13:41.891+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49848 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49928 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49410 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49104 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50394 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49118 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50574 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50634 closed 2023-10-21T04:13:41.892+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49266 closed 2023-10-21T04:13:41.893+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50370 closed 2023-10-21T04:13:41.893+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49890 closed 2023-10-21T04:13:41.893+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49622 closed 2023-10-21T04:13:41.893+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49196 closed 2023-10-21T04:13:41.893+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50340 closed 2023-10-21T04:13:41.893+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49642 closed 2023-10-21T04:13:41.894+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49556 closed 2023-10-21T04:13:41.894+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49854 closed 2023-10-21T04:13:41.894+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50184 closed 2023-10-21T04:13:41.894+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49790 closed 2023-10-21T04:13:41.894+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49090 closed 2023-10-21T04:13:41.894+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50172 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49064 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49200 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50398 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50678 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49536 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49578 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49216 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49546 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49878 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50402 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50312 closed 2023-10-21T04:13:41.895+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50672 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50076 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50220 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49160 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50748 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50084 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50300 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50706 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49068 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50014 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49832 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49620 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50082 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50528 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49938 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49182 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49226 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49102 closed 2023-10-21T04:13:41.896+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50296 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50578 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50718 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49228 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50636 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49082 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49272 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49586 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50114 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49748 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49874 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49786 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49806 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50542 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50122 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49886 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49908 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50686 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49162 closed 2023-10-21T04:13:41.897+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50556 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49896 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49632 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50736 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49086 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50420 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49812 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50644 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49234 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50032 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50136 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49946 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49134 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50558 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50284 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50352 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49506 closed 2023-10-21T04:13:41.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49708 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50056 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49526 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49566 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50412 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49158 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49828 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50464 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50744 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50656 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50728 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50378 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49906 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49454 closed 2023-10-21T04:13:41.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50422 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49224 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50374 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49214 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49954 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50158 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50716 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49538 closed 2023-10-21T04:13:41.900+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50216 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49150 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50376 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50648 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50366 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49962 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50286 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50102 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50532 closed 2023-10-21T04:13:41.901+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50188 closed 2023-10-21T04:13:41.902+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50214 closed 2023-10-21T04:13:41.902+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49056 closed 2023-10-21T04:13:41.902+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49594 closed 2023-10-21T04:13:41.902+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49054 closed 2023-10-21T04:13:41.902+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49456 closed 2023-10-21T04:13:41.903+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50208 closed 2023-10-21T04:13:41.903+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50034 closed 2023-10-21T04:13:41.903+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49940 closed 2023-10-21T04:13:41.905+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49598 closed 2023-10-21T04:13:41.905+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50350 closed 2023-10-21T04:13:41.905+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49564 closed 2023-10-21T04:13:41.905+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50024 closed 2023-10-21T04:13:41.905+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49948 closed 2023-10-21T04:13:41.906+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49466 closed 2023-10-21T04:13:41.906+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49614 closed 2023-10-21T04:13:41.906+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50022 closed 2023-10-21T04:13:41.958+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:13:42.034+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:13:42.034+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9112] 2023-10-21T04:13:42.048+05:30 [Info] switched currmeta from 923 -> 923 force true 2023-10-21T04:13:42.103+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:13:42.137+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 2 2023-10-21T04:13:42.137+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(1) 2023-10-21T04:16:11.172+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=4c6e4b64d0c01f9d6431a65ac36fc024 -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:16:11.173+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:16:11.173+05:30 [Info] Setting ipv6=false 2023-10-21T04:16:11.227+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:16:11.267+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:16:11.272+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:16:11.272+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:16:11.272+05:30 [Info] Setting log level to Info 2023-10-21T04:16:11.272+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:16:11.272+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:16:11.272+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:16:11.272+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:16:11.293+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:16:11 audit: created new audit service 2023-10-21T04:16:11.488+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:16:11.488+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:16:11.488+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:16:11.490+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:16:11.490+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:16:11.490+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:16:11.522+05:30 [Info] security context initialized 2023-10-21T04:16:11.522+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9120:19120 9499:19499 9999:9999] 2023-10-21T04:16:11.522+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:16:11.523+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:16:11.523+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:16:11.523+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:16:11.528+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:16:11.528+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:16:11.528+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:16:11.555+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:16:11.568+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:16:11.602+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:16:11.623+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:16:11.624+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:16:11.632+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:16:11.640+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:16:11.715+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:16:11.717+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:16:11.718+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:11.738+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:16:11.738+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:16:11.739+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:16:11.753+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:16:11.753+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:16:11.756+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:16:11.756+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:16:11.756+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:16:11.758+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:16:11.758+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:16:11.759+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 342 us 2023-10-21T04:16:11.761+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:16:11.764+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:16:11.777+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:16:11.802+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:16:11.804+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:16:11.807+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:16:11.808+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:16:11.809+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:16:11.811+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerId Value 4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:16:11.815+05:30 [Info] Indexer Id 4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:16:11.817+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:16:11.817+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:16:11.832+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:16:11.833+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:16:11.856+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 177] updating cluster rev epoch 0 -> 1 2023-10-21T04:16:11.856+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 177] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:16:11.884+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 226] bucketNames has been updated (was map[], now map[bucket_%2:7007f0ab442ac00e52d057a30cd22644 bucket_1:db19c9050b4ab1232e6bc6654011618f]) 2023-10-21T04:16:11.884+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 227] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:16:11.884+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 228] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:16:11.900+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:16:11.900+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:16:11.903+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:16:11.906+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:16:11.906+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:16:11.913+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 226] updated cluster rebalancing status to true 2023-10-21T04:16:11.913+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:16:11.913+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:16:11.913+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:16:11.915+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 224] updating rebalancing status to true 2023-10-21T04:16:11.921+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:16:11.921+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:16:11.921+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:16:11.921+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning false RebalanceToken 2023-10-21T04:16:11.921+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:16:11.921+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:16:11.922+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:16:11.920+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:16:11.924+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:16:11.941+05:30 DEBU REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:323 [id 221] disregarding & deleting stale (age: 2m9.931706903s) storage report from node [index f2b29e6bf10e9349c06775f41b68f8ff] (delete err: ) 2023-10-21T04:16:11.942+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 221] /regulator/report/s/index/f2b29e6bf10e9349c06775f41b68f8ff has 0-length value 2023-10-21T04:16:11.945+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap , elapsed: 21.510666ms 2023-10-21T04:16:11.945+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:16:11.945+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:16:11.945+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:16:11.946+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager false 2023-10-21T04:16:11.946+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:16:11.946+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:16:11.946+05:30 [Error] RebalanceServiceManager::getCachedIndexerNodeUUIDs: 127.0.0.1:9120/getCachedIndexerNodeUUIDs returned err: Unable to find given hostport in cbauth database: `127.0.0.1:9120' 2023-10-21T04:16:11.946+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:16:11.947+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:16:11.951+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024. Err projector.topicMissing 2023-10-21T04:16:11.951+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:16:11.951+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:16:11.951+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:16:11.952+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024. Err projector.topicMissing 2023-10-21T04:16:11.952+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:16:11.952+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:16:11.952+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:11.953+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:16:11.989+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:16:11.989+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:16:11.992+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:16:11.996+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:16:11.996+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024] IsBalanced:true Messages:[]} 2023-10-21T04:16:11.997+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:16:11.998+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:16:11.999+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:16:11.999+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:16:12.002+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:16:12.002+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/stats: no such file or directory 2023-10-21T04:16:12.002+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:16:12.002+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:12.003+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:16:12.004+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:16:12.004+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{4c6e4b64d0c01f9d6431a65ac36fc024 7060000 } 2023-10-21T04:16:12.005+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:16:12.010+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:16:12.010+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:16:12.010+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:16:12.010+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:16:12.010+05:30 [Info] progress stats collection done. 2023-10-21T04:16:12.010+05:30 [Info] send progress stats to clients 2023-10-21T04:16:12.020+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:16:12.021+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:16:12.022+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:16:12.025+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:16:12.025+05:30 [Info] Setting log level to Info 2023-10-21T04:16:12.025+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:16:12.026+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:16:12.041+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:16:12.041+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:16:12.041+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:16:12.041+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:16:12.041+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:16:12.064+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:16:12.096+05:30 [Info] security context initialized 2023-10-21T04:16:12.096+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9499:19499 9999:9999] 2023-10-21T04:16:12.103+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:16:12.142+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:16:12.142+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:16:12.145+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:16:12.145+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 1 unhealthy node 0 add node 0 2023-10-21T04:16:12.145+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:12.145+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106] 2023-10-21T04:16:12.145+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:16:12.149+05:30 [Warn] watcher:ClientAuth cluster Ver (9)/Internal Version () not yet initialised 2023-10-21T04:16:12.151+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:16:12.153+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:16:12.153+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:16:12.180+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 67] will report aggregate recorder stats every 5m0s 2023-10-21T04:16:12.240+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {cf56ae547f07d182689428d60f3e89c9 [] topology-change-rebalance [{{4c6e4b64d0c01f9d6431a65ac36fc024 7060000 } recovery-full} {{458c2b8d760618b2a2f6a998f4c93e74 7060000 } recovery-full}] [{217f03d04907d0ef82070fe4f41363c1 7060000 }]} 2023-10-21T04:16:12.266+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:46298 2023-10-21T04:16:12.278+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:16:12.306+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:16:12.307+05:30 [Info] initialized currmeta 47 force true 2023-10-21T04:16:12.344+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:46302 2023-10-21T04:16:12.353+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:46306 2023-10-21T04:16:12.359+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:46308 2023-10-21T04:16:12.402+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:16:12.424+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:12.430+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:16:12.430+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:16:12.430+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:16:12.431+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:16:12.434+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:16:12.434+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:16:12.434+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:16:12.434+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:16:12.434+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:16:12.434+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:16:12.434+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:16:12.436+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 6.285µs 2023-10-21T04:16:12.436+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:16:12.436+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:16:12.436+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:16:12.436+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:16:12.439+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:16:12.439+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:12.440+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 4.0612ms 2023-10-21T04:16:12.440+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:16:12.443+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:16:12.442+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 5.579838ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 1 error 2023-10-21T04:16:12.443+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 8.683µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.443+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:16:12.445+05:30 [Info] Indexer::ReadMemstats Time Taken 1.535946ms 2023-10-21T04:16:12.445+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/cf56ae547f07d182689428d60f3e89c9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:cf56ae547f07d182689428d60f3e89c9]}]} 2023-10-21T04:16:12.465+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:16:12.465+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:16:12.470+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:12.470+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:12.470+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:16:12.473+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 125.719367ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.478+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 20.507µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.487+05:30 [Warn] watcher:ClientAuth cluster Ver (9)/Internal Version () not yet initialised 2023-10-21T04:16:12.488+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:46346 2023-10-21T04:16:12.488+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:16:12.495+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 225.79696ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.498+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 10.96µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.503+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 140.504464ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.505+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 11.027µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.518+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 160.091029ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.521+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.128µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.531+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 40.293784ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.531+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:16:12.531+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:16:12.531+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 7.496µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.544+05:30 [Warn] watcher:ClientAuth cluster Ver (9)/Internal Version () not yet initialised 2023-10-21T04:16:12.546+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:16:12.569+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:16:12.573+05:30 [Info] switched currmeta from 47 -> 49 force true 2023-10-21T04:16:12.589+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:16:12.590+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:16:12.591+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:16:12.597+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:16:12.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57954" doAuth() ... 2023-10-21T04:16:12.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57954" doReceive() ... 2023-10-21T04:16:12.611+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57960" doAuth() ... 2023-10-21T04:16:12.613+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57962" doAuth() ... 2023-10-21T04:16:12.615+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57962" doReceive() ... 2023-10-21T04:16:12.621+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57960" doReceive() ... 2023-10-21T04:16:12.623+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57966" doAuth() ... 2023-10-21T04:16:12.626+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57966" doReceive() ... 2023-10-21T04:16:12.641+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:12.641+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:12.646+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:16:12.647+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:16:12.648+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:16:12.646+05:30 [Info] switched currmeta from 49 -> 49 force true 2023-10-21T04:16:12.650+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 2023-10-21T04:16:12.652+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.651+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:16:12.654+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57976" doAuth() ... 2023-10-21T04:16:12.653+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.654+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[13739166377675070800:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.655+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 13739166377675070800:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.656+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 13739166377675070800:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.660+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57976" doReceive() ... 2023-10-21T04:16:12.664+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.664+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 5004576769623010783:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.664+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.664+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.664+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.664+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.664+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.665+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 9912003609070511414:[6 7 8 1 2 3 4 5] 14213410630307008054:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.665+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 9912003609070511414:[6 7 8 1 2 3 4 5] 14213410630307008054:[0] 15667952562388529486:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.665+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 7494249569853685720:[0] 9912003609070511414:[6 7 8 1 2 3 4 5] 14213410630307008054:[0] 15667952562388529486:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.665+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 7494249569853685720:[0] 7582778950442222621:[0] 9912003609070511414:[6 7 8 1 2 3 4 5] 14213410630307008054:[0] 15667952562388529486:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[4992762551574016313:[0] 9720186692804226822:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.665+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 4992762551574016313:[0] 9720186692804226822:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 7494249569853685720:[0] 7582778950442222621:[0] 9912003609070511414:[6 7 8 1 2 3 4 5] 14213410630307008054:[0] 15667952562388529486:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 2023-10-21T04:16:12.665+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 7494249569853685720:[0] 7582778950442222621:[0] 9912003609070511414:[6 7 8 1 2 3 4 5] 14213410630307008054:[0] 15667952562388529486:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[2296526085582506469:[0] 4992762551574016313:[0] 9720186692804226822:[0] 11050396066566371125:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.665+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 3100906924534408756:[0] 4440544141961093904:[4 5 6 7 8 1 2 3] 5004576769623010783:[0] 5744899830895973442:[2 3 4 5 6 7 8 1] 7494249569853685720:[0] 7582778950442222621:[0] 9912003609070511414:[6 7 8 1 2 3 4 5] 14213410630307008054:[0] 15667952562388529486:[0]]. New rollbacktime: 1697841845713082402, Old rollbacktime: 0 {Index inst : [partitions]}: map[1265729768899070816:[0] 2296526085582506469:[0] 4992762551574016313:[0] 9720186692804226822:[0] 11050396066566371125:[0] 11460826864060385936:[2 3 4 5 6 7 8 1] 13739166377675070800:[0] 14579235200590321485:[5 6 7 8 1 2 3 4] 14975131556821987100:[0] 17357232428819395534:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697841845874870817, Old rollbacktime: 0 2023-10-21T04:16:12.674+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:46400 2023-10-21T04:16:12.696+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.599198ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.698+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.825µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:12.757+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57988" doAuth() ... 2023-10-21T04:16:12.757+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:57988" doReceive() ... 2023-10-21T04:16:12.942+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 220] aggregate storage info: map[bucket_%2:map[kv:5716160 index:23500507] bucket_1:map[kv:6441152 index:21169449]] 2023-10-21T04:16:12.948+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 0 NodeUUIDs: [] 2023-10-21T04:16:12.949+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 1 NodeUUIDs: [4c6e4b64d0c01f9d6431a65ac36fc024] 2023-10-21T04:16:12.949+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 1 NodeUUIDs: [4c6e4b64d0c01f9d6431a65ac36fc024] 2023-10-21T04:16:12.949+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:16:12.949+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/cf56ae547f07d182689428d60f3e89c9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:cf56ae547f07d182689428d60f3e89c9]}]} 2023-10-21T04:16:12.951+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:16:12.951+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:16:12.979+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:16:12.996+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:16:13.004+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:16:13.019+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:16:13.031+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:16:13.197+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:16:13.827+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {458c2b8d760618b2a2f6a998f4c93e74 cf56ae547f07d182689428d60f3e89c9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:16:13.829+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {458c2b8d760618b2a2f6a998f4c93e74 cf56ae547f07d182689428d60f3e89c9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:16:13.830+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"458c2b8d760618b2a2f6a998f4c93e74","RebalId":"cf56ae547f07d182689428d60f3e89c9","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:16:13.831+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{458c2b8d760618b2a2f6a998f4c93e74 cf56ae547f07d182689428d60f3e89c9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:16:13.831+05:30 [Info] NewShardRebalancer nodeId 4c6e4b64d0c01f9d6431a65ac36fc024 rebalToken &{458c2b8d760618b2a2f6a998f4c93e74 cf56ae547f07d182689428d60f3e89c9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:16:13.832+05:30 [Info] ShardRebalancer::observeRebalance &{458c2b8d760618b2a2f6a998f4c93e74 cf56ae547f07d182689428d60f3e89c9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:16:13.843+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"458c2b8d760618b2a2f6a998f4c93e74","RebalId":"cf56ae547f07d182689428d60f3e89c9","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:16:13.843+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:16:14.017+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:59430 2023-10-21T04:16:14.021+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:59430 2023-10-21T04:16:14.033+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:59430 2023-10-21T04:16:14.229+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:14.398+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:14.447+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{458c2b8d760618b2a2f6a998f4c93e74 cf56ae547f07d182689428d60f3e89c9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:16:14.550+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:14.668+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:14.983+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:14.988+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:14.990+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:16:14.991+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:16:14.992+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:16:14.994+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 68.621µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:15.008+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.010+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:16:15.010+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:16:15.010+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:16:15.010+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 29.077µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:15.060+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.063+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.067+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.141+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"458c2b8d760618b2a2f6a998f4c93e74","RebalId":"cf56ae547f07d182689428d60f3e89c9","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:16:15.141+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:16:15.141+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:16:15.141+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:16:15.141+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:16:15.141+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:16:15.141+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 18.035µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:15.179+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.292+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.588+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.588+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:16:15.588+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:16:15.588+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:16:15.588+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 28.035µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:15.588+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2864853750198994630:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/2864853750198994630/shards/shard2864853750198994630 12177293282243154215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/12177293282243154215/shards/shard12177293282243154215] cf56ae547f07d182689428d60f3e89c9 ShardTokena7:37:e9:2a:6a:83:12:d0 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc005f330e0 0xc005f33140 map[] 0xc005b5d020 0xc005b7f8c0 false} 2023-10-21T04:16:15.589+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:16:15.589+05:30 [Info] Directory /opt/build/ns_server/data/n_3/data/@2i/shards does not exist. Skip shard recovery. 2023-10-21T04:16:15.590+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/12177293282243154215/shards/shard12177293282243154215 2023-10-21T04:16:15.591+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/2864853750198994630/shards/shard2864853750198994630 2023-10-21T04:16:15.638+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.641+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/12177293282243154215/shards/shard12177293282243154215/data: super block snapshot head offset :0 tail offset:4894720 sbgen :858 2023-10-21T04:16:15.642+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/12177293282243154215/shards/shard12177293282243154215/data/recovery: super block snapshot head offset :0 tail offset:4714496 sbgen :858 2023-10-21T04:16:15.644+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/2864853750198994630/shards/shard2864853750198994630/data: super block snapshot head offset :0 tail offset:5853184 sbgen :974 2023-10-21T04:16:15.644+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/2864853750198994630/shards/shard2864853750198994630/data/recovery: super block snapshot head offset :0 tail offset:5648384 sbgen :974 2023-10-21T04:16:15.648+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/12177293282243154215/shards/shard12177293282243154215: metadata saved successfully 2023-10-21T04:16:15.650+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 (12177293282243154215) added 2023-10-21T04:16:15.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 from metadata completed, tenant :bucket_1_db19c9050b4ab1232e6bc6654011618f 2023-10-21T04:16:15.650+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 160559267 totalBytes: 9633556 numShards: 2 duration: 60.364152ms 2023-10-21T04:16:15.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/12177293282243154215/shards/shard12177293282243154215, newshardId 12177293282243154215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215, shardsize 9633556, duration 60.125238ms 2023-10-21T04:16:15.650+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 12177293282243154215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215, err: , elapsed(sec): 0.061657182 2023-10-21T04:16:15.650+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 12177293282243154215 bytesWritten: 9633556, totalBytes: 9633556, transferRate: 1.6055926666666666e+08 2023-10-21T04:16:15.651+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/2864853750198994630/shards/shard2864853750198994630: metadata saved successfully 2023-10-21T04:16:15.652+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 (2864853750198994630) added 2023-10-21T04:16:15.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 from metadata completed, tenant :bucket_1_db19c9050b4ab1232e6bc6654011618f 2023-10-21T04:16:15.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardTokena7_37_e9_2a_6a_83_12_d0/2864853750198994630/shards/shard2864853750198994630, newshardId 2864853750198994630, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630, shardsize 11526931, duration 60.81335ms 2023-10-21T04:16:15.652+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2864853750198994630, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630, err: , elapsed(sec): 0.063613285 2023-10-21T04:16:15.652+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: cf56ae547f07d182689428d60f3e89c9, transferId: ShardTokena7:37:e9:2a:6a:83:12:d0 2023-10-21T04:16:15.652+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: cf56ae547f07d182689428d60f3e89c9, transferId: ShardTokena7:37:e9:2a:6a:83:12:d0, elapsed(sec): 0.000293677 2023-10-21T04:16:15.652+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2864853750198994630: 12177293282243154215:], shardPaths: map[2864853750198994630:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 12177293282243154215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.063972446 2023-10-21T04:16:15.652+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2864853750198994630 bytesWritten: 11526931, totalBytes: 11526931, transferRate: 1.9211551666666666e+08 2023-10-21T04:16:15.652+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [2864853750198994630 12177293282243154215], ttid: ShardTokena7:37:e9:2a:6a:83:12:d0, elapsed(sec): 0.064596013 2023-10-21T04:16:15.685+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.685+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardTokena7:37:e9:2a:6a:83:12:d0 2023-10-21T04:16:15.685+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2864853750198994630:true 12177293282243154215:true] 2023-10-21T04:16:15.685+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.587µs 2023-10-21T04:16:15.685+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [2864853750198994630 12177293282243154215], isLockedForRecovery: true 2023-10-21T04:16:15.685+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [2864853750198994630 12177293282243154215], errMap: map[2864853750198994630: 12177293282243154215:], elapsed: 10.541µs 2023-10-21T04:16:15.690+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 573521267164877540, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:15.690+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 573521267164877540, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:15.891+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.906+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:15.998+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 9912003609070511414,realInstId: 0, indexDefn DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:15.998+05:30 [Info] Indexer::handleRecoverIndex InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:15.999+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 639.178µs 2023-10-21T04:16:15.999+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 681.919µs 2023-10-21T04:16:15.999+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:15.999+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:15.999+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': added 2023-10-21T04:16:16.000+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:16:15.999+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 9912003609070511414, indexDefn DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:16.013+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 323.142914ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:16.014+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner initialized 2023-10-21T04:16:16.014+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:16.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.019+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCleaner initialized for recovery 2023-10-21T04:16:16.019+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:16.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery 2023-10-21T04:16:16.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data], Shared [true] 2023-10-21T04:16:16.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[4714496] Data log Head [0] Tail [4894720] 2023-10-21T04:16:16.019+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [4714496] 2023-10-21T04:16:16.021+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner initialized 2023-10-21T04:16:16.021+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:16.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.022+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCleaner initialized for recovery 2023-10-21T04:16:16.022+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:16.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery 2023-10-21T04:16:16.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data], Shared [true] 2023-10-21T04:16:16.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[5648384] Data log Head [0] Tail [5853184] 2023-10-21T04:16:16.023+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [5648384] 2023-10-21T04:16:16.025+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.028+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex 2023-10-21T04:16:16.028+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is disabled 2023-10-21T04:16:16.030+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.037+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:16:16.033+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.037+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex 2023-10-21T04:16:16.038+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.037+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.040+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:16:16.041+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:16:16.043+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.043+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:16:16.043+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.043+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:16:16.043+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.044+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:16:16.044+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.044+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:16:16.044+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.045+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:16:16.045+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.046+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:16:16.046+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.046+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:16:16.052+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.052+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex 2023-10-21T04:16:16.052+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.053+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.053+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:16:16.053+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex 2023-10-21T04:16:16.054+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex 2023-10-21T04:16:16.054+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex 2023-10-21T04:16:16.054+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex 2023-10-21T04:16:16.054+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex 2023-10-21T04:16:16.054+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.055+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex 2023-10-21T04:16:16.055+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.055+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex 2023-10-21T04:16:16.056+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.056+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex 2023-10-21T04:16:16.057+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.057+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex 2023-10-21T04:16:16.057+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.058+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex 2023-10-21T04:16:16.059+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.059+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex 2023-10-21T04:16:16.059+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.059+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex 2023-10-21T04:16:16.059+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.059+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex 2023-10-21T04:16:16.060+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.060+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex 2023-10-21T04:16:16.060+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.060+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex 2023-10-21T04:16:16.060+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.060+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex 2023-10-21T04:16:16.060+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:16:16.061+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.061+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:16:16.061+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.061+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:16:16.061+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.061+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:16:16.061+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.061+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:16:16.062+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.062+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:16:16.062+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.062+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:16:16.065+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.065+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex 2023-10-21T04:16:16.070+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.072+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex 2023-10-21T04:16:16.074+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.078+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:16.084+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:16.084+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex 2023-10-21T04:16:16.092+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.094+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex 2023-10-21T04:16:16.094+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.094+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex 2023-10-21T04:16:16.094+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.095+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex 2023-10-21T04:16:16.095+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.095+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex 2023-10-21T04:16:16.095+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.096+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex 2023-10-21T04:16:16.097+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.097+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex 2023-10-21T04:16:16.099+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.103+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex 2023-10-21T04:16:16.101+05:30 [Info] switched currmeta from 49 -> 50 force true 2023-10-21T04:16:16.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex 2023-10-21T04:16:16.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex 2023-10-21T04:16:16.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex 2023-10-21T04:16:16.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex 2023-10-21T04:16:16.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex 2023-10-21T04:16:16.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex 2023-10-21T04:16:16.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex 2023-10-21T04:16:16.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex 2023-10-21T04:16:16.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.127+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex 2023-10-21T04:16:16.179+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:16.202+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.203+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:16.203+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.203+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:16.205+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.205+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:16.205+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.206+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:16.206+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.206+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:16.207+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.208+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:16.208+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.261+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.270+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:16.273+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.274+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:16.280+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.280+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:16.286+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.286+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:16.286+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.287+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:16.289+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:16.289+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.289+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:16.289+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.289+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:16.290+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.290+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:16.290+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.290+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:16.291+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.291+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:16.292+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.292+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:16.293+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.293+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:16.294+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.294+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:16.294+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.294+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:16.294+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.295+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:16.295+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.295+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:16.296+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.296+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:16.296+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.296+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:16.297+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.297+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:16.298+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.298+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:16.298+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.298+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:16.299+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.322+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.322+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:16.323+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.324+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:16.325+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:16.326+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.327+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:16.331+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.331+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:16.333+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.334+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:16.336+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.339+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:16.337+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.355+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:16.361+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.367+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:16.370+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.370+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:16.371+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.371+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:16.371+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.371+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:16.372+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.372+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:16.372+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:16.379+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:16.372+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.372+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:16.368+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:16:16.383+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:16:16.382+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:16.385+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.388+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:16.385+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:16:16.389+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.390+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:16.389+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 112.941µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:16.389+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[7659339677706884510:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/7659339677706884510/shards/shard7659339677706884510 10890719122371003479:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/10890719122371003479/shards/shard10890719122371003479] cf56ae547f07d182689428d60f3e89c9 ShardToken8f:ea:6f:63:2c:70:f5:8c /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc005f330e0 0xc005f33140 map[] 0xc0072530e0 0xc007278cc0 false} 2023-10-21T04:16:16.391+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/7659339677706884510/shards/shard7659339677706884510 2023-10-21T04:16:16.396+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.396+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:16.396+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.396+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:16.396+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.396+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:16.396+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.397+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex 2023-10-21T04:16:16.401+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/10890719122371003479/shards/shard10890719122371003479 2023-10-21T04:16:16.402+05:30 [Info] switched currmeta from 50 -> 51 force true 2023-10-21T04:16:16.404+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/10890719122371003479/shards/shard10890719122371003479/data: super block snapshot head offset :0 tail offset:6340608 sbgen :1038 2023-10-21T04:16:16.405+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/10890719122371003479/shards/shard10890719122371003479/data/recovery: super block snapshot head offset :0 tail offset:6135808 sbgen :1038 2023-10-21T04:16:16.406+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.407+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:16.407+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/7659339677706884510/shards/shard7659339677706884510/data: super block snapshot head offset :0 tail offset:5578752 sbgen :912 2023-10-21T04:16:16.409+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.409+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:16.409+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/7659339677706884510/shards/shard7659339677706884510/data/recovery: super block snapshot head offset :0 tail offset:5386240 sbgen :912 2023-10-21T04:16:16.410+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.423+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:16.424+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.417+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/7659339677706884510/shards/shard7659339677706884510: metadata saved successfully 2023-10-21T04:16:16.425+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:16.426+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.426+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:16.427+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.427+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:16.427+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.427+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:16.427+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.427+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:16.427+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.427+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:16.427+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.428+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:16.428+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:16.428+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex 2023-10-21T04:16:16.429+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : replayOffset: 4882432 (RP:4787868) (CP:4882432) 2023-10-21T04:16:16.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [4882432] took [410.066424ms] 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 10757, memSzIndex 93, numRecords 138 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 9544, memSzIndex 0, numRecords 120 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 8918, memSzIndex 0, numRecords 124 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.429+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 10425, memSzIndex 93, numRecords 135 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9719, memSzIndex 0, numRecords 122 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9602, memSzIndex 76, numRecords 128 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 10282, memSzIndex 77, numRecords 134 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 8168, memSzIndex 0, numRecords 107 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 10953, memSzIndex 77, numRecords 140 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 8675, memSzIndex 0, numRecords 112 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 8367, memSzIndex 0, numRecords 109 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9576, memSzIndex 0, numRecords 121 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.430+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1056050, resident 1, memSz 67040, memSzIndex 856, numRecords 1000 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 10790, memSzIndex 77, numRecords 139 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 10321, memSzIndex 77, numRecords 134 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1056116, resident 1, memSz 67112, memSzIndex 921, numRecords 1000 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 10205, memSzIndex 77, numRecords 133 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 9629, memSzIndex 0, numRecords 121 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9392, memSzIndex 0, numRecords 119 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 8983, memSzIndex 0, numRecords 115 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 8591, memSzIndex 0, numRecords 111 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 10069, memSzIndex 77, numRecords 132 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 8808, memSzIndex 0, numRecords 113 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 10638, memSzIndex 91, numRecords 137 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053364, resident 1, memSz 10005, memSzIndex 76, numRecords 131 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.431+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9253, memSzIndex 92, numRecords 125 2023-10-21T04:16:16.432+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.432+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: request quota after header replay 1056180, resident 1, memSz 68037, memSzIndex 890, numRecords 1000 2023-10-21T04:16:16.432+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.432+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [4894720] replayOffset [4882432] 2023-10-21T04:16:16.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [412.691688ms] 2023-10-21T04:16:16.432+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.433+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.435+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.436+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.436+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.437+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : replayOffset: 5844992 (RP:5836800) (CP:5844992) 2023-10-21T04:16:16.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [5844992] took [414.307471ms] 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1054936, resident 1, memSz 60835, memSzIndex 307, numRecords 1000 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 13385, memSzIndex 0, numRecords 131 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 11411, memSzIndex 0, numRecords 111 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 13648, memSzIndex 0, numRecords 134 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 12328, memSzIndex 0, numRecords 120 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 12409, memSzIndex 0, numRecords 121 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 14296, memSzIndex 0, numRecords 140 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 13575, memSzIndex 0, numRecords 133 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 13980, memSzIndex 0, numRecords 137 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.437+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.438+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.438+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.438+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.438+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.438+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.438+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1055884, resident 1, memSz 69090, memSzIndex 444, numRecords 1000 2023-10-21T04:16:16.439+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.439+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.442+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/10890719122371003479/shards/shard10890719122371003479: metadata saved successfully 2023-10-21T04:16:16.438+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.443+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.443+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data], Shared [true]. Built [54] plasmas, took [423.886623ms] 2023-10-21T04:16:16.443+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:16.444+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 20 count: 121 metaSz: 2576 rp [1] sn: 360005 count: 121 metaSz: 2577 rp [2] sn: 720006 count: 121 metaSz: 2577 rp [3] sn: 1080007 count: 121 metaSz: 2577 2023-10-21T04:16:16.444+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 118231 recovery UsedSpace 140766 2023-10-21T04:16:16.439+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.445+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.445+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.445+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.445+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.445+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 14091, memSzIndex 0, numRecords 138 2023-10-21T04:16:16.446+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.446+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 11620, memSzIndex 0, numRecords 113 2023-10-21T04:16:16.446+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.446+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1054956, resident 1, memSz 60897, memSzIndex 322, numRecords 1000 2023-10-21T04:16:16.446+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.447+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 11004, memSzIndex 0, numRecords 107 2023-10-21T04:16:16.447+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.448+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.448+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.448+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 11195, memSzIndex 0, numRecords 109 2023-10-21T04:16:16.448+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.449+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 13066, memSzIndex 0, numRecords 128 2023-10-21T04:16:16.469+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.439+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 (7659339677706884510) added 2023-10-21T04:16:16.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 from metadata completed, tenant :bucket_%2_7007f0ab442ac00e52d057a30cd22644 2023-10-21T04:16:16.482+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 (10890719122371003479) added 2023-10-21T04:16:16.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 from metadata completed, tenant :bucket_%2_7007f0ab442ac00e52d057a30cd22644 2023-10-21T04:16:16.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/10890719122371003479/shards/shard10890719122371003479, newshardId 10890719122371003479, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479, shardsize 12502003, duration 81.529921ms 2023-10-21T04:16:16.483+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10890719122371003479, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479, err: , elapsed(sec): 0.091871093 2023-10-21T04:16:16.483+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10890719122371003479 bytesWritten: 12502003, totalBytes: 12502003, transferRate: 1.5434571604938272e+08 2023-10-21T04:16:16.472+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.490+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.490+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.490+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.490+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.490+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.490+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.491+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.491+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 13443, memSzIndex 0, numRecords 132 2023-10-21T04:16:16.491+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.491+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 13687, memSzIndex 0, numRecords 134 2023-10-21T04:16:16.491+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.491+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 12785, memSzIndex 0, numRecords 125 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 11491, memSzIndex 0, numRecords 112 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.492+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.493+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.493+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.494+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.494+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 14136, memSzIndex 0, numRecords 139 2023-10-21T04:16:16.494+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.494+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 13771, memSzIndex 0, numRecords 135 2023-10-21T04:16:16.494+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.497+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 11787, memSzIndex 0, numRecords 115 2023-10-21T04:16:16.497+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.497+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.497+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.498+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 12678, memSzIndex 0, numRecords 124 2023-10-21T04:16:16.498+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.501+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 12180, memSzIndex 0, numRecords 119 2023-10-21T04:16:16.500+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:16.501+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.502+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057426, resident 1, memSz 12495, memSzIndex 0, numRecords 122 2023-10-21T04:16:16.502+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.502+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.502+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.502+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.503+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.503+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1054958, resident 1, memSz 60918, memSzIndex 323, numRecords 1000 2023-10-21T04:16:16.503+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.503+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.503+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.503+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.504+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:16:16.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.505+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swapper started 2023-10-21T04:16:16.506+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:16:16.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex started 2023-10-21T04:16:16.504+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.510+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.508+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 7659339677706884510 bytesWritten: 10989542, totalBytes: 10989542, transferRate: 1.37369275e+08 2023-10-21T04:16:16.508+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.508+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/cf56ae547f07d182689428d60f3e89c9_ShardToken8f_ea_6f_63_2c_70_f5_8c/7659339677706884510/shards/shard7659339677706884510, newshardId 7659339677706884510, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510, shardsize 10989542, duration 114.419683ms 2023-10-21T04:16:16.510+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1055776, resident 0.07, memSz 437, memSzIndex 0, numRecords 27 2023-10-21T04:16:16.511+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.511+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1056190, resident 1, memSz 72977, memSzIndex 352, numRecords 1000 2023-10-21T04:16:16.512+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.512+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.512+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.512+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.511+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 7659339677706884510, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510, err: , elapsed(sec): 0.120191214 2023-10-21T04:16:16.513+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: cf56ae547f07d182689428d60f3e89c9, transferId: ShardToken8f:ea:6f:63:2c:70:f5:8c 2023-10-21T04:16:16.513+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: cf56ae547f07d182689428d60f3e89c9, transferId: ShardToken8f:ea:6f:63:2c:70:f5:8c, elapsed(sec): 0.000379834 2023-10-21T04:16:16.513+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[7659339677706884510: 10890719122371003479:], shardPaths: map[7659339677706884510:/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 10890719122371003479:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.122302606 2023-10-21T04:16:16.514+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [10890719122371003479 7659339677706884510], ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c, elapsed(sec): 0.1464416 2023-10-21T04:16:16.513+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 129934461 controller mem 16974 2023-10-21T04:16:16.524+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.524+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885885 controller mem 16974 2023-10-21T04:16:16.525+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1055776, resident 1, memSz 66042, memSzIndex 445, numRecords 1000 2023-10-21T04:16:16.525+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885885 controller mem 16974 2023-10-21T04:16:16.525+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1057276, resident 1, memSz 12356, memSzIndex 0, numRecords 121 2023-10-21T04:16:16.525+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885885 controller mem 16974 2023-10-21T04:16:16.525+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885885 controller mem 16974 2023-10-21T04:16:16.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885885 controller mem 16974 2023-10-21T04:16:16.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.535+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885885 controller mem 16974 2023-10-21T04:16:16.535+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:16.536+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 128885885 controller mem 16974 2023-10-21T04:16:16.536+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [5853184] replayOffset [5844992] 2023-10-21T04:16:16.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [514.254944ms] 2023-10-21T04:16:16.539+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.539+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.560+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.561+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.562+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.562+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.562+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:16.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data], Shared [true]. Built [61] plasmas, took [539.2647ms] 2023-10-21T04:16:16.562+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:16.563+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 121 metaSz: 2577 rp [1] sn: 1080007 count: 121 metaSz: 2577 2023-10-21T04:16:16.563+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 105671 recovery UsedSpace 149002 2023-10-21T04:16:16.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:16:16.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.571+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:16:16.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex started 2023-10-21T04:16:16.571+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.571+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.571+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:16.600+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 8 Warmup took 600.568309ms 2023-10-21T04:16:16.600+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 8 WriterThreads 1 2023-10-21T04:16:16.600+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index Index: 9912003609070511414 Partition: 8 2023-10-21T04:16:16.601+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:16.601+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:16.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.601+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:16:16.601+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 19 count: 134 metaSz: 2576 rp [1] sn: 360005 count: 134 metaSz: 2577 rp [2] sn: 720006 count: 134 metaSz: 2577 rp [3] sn: 1080007 count: 134 metaSz: 2577 2023-10-21T04:16:16.601+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 123906 recovery UsedSpace 141484 2023-10-21T04:16:16.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.602+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:16.602+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 134 metaSz: 2577 rp [1] sn: 1080007 count: 134 metaSz: 2577 2023-10-21T04:16:16.602+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 107778 recovery UsedSpace 149002 2023-10-21T04:16:16.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:16:16.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.629+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:16:16.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex started 2023-10-21T04:16:16.629+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.629+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:16:16.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.632+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:16:16.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex started 2023-10-21T04:16:16.632+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:16.632+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.632+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.670+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 1 Warmup took 69.132427ms 2023-10-21T04:16:16.671+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 1 WriterThreads 1 2023-10-21T04:16:16.671+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index Index: 9912003609070511414 Partition: 1 2023-10-21T04:16:16.671+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:16.672+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:16.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.678+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:16:16.678+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 21 count: 113 metaSz: 2576 rp [1] sn: 360005 count: 113 metaSz: 2577 rp [2] sn: 720006 count: 113 metaSz: 2577 rp [3] sn: 1080007 count: 113 metaSz: 2577 2023-10-21T04:16:16.678+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 117393 recovery UsedSpace 140766 2023-10-21T04:16:16.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.678+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:16.678+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 113 metaSz: 2577 rp [1] sn: 1080007 count: 113 metaSz: 2577 2023-10-21T04:16:16.678+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 105192 recovery UsedSpace 149002 2023-10-21T04:16:16.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:16:16.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.693+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:16:16.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex started 2023-10-21T04:16:16.694+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.694+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:16:16.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.703+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:16:16.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex started 2023-10-21T04:16:16.703+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.703+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:16.703+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.758+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 2 Warmup took 80.815497ms 2023-10-21T04:16:16.759+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 2 WriterThreads 1 2023-10-21T04:16:16.759+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index Index: 9912003609070511414 Partition: 2 2023-10-21T04:16:16.759+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:16.760+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:16.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.761+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:16:16.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.761+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:16.761+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 132 metaSz: 2577 rp [1] sn: 1080007 count: 132 metaSz: 2577 2023-10-21T04:16:16.761+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 107507 recovery UsedSpace 149002 2023-10-21T04:16:16.761+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 21 count: 132 metaSz: 2576 rp [1] sn: 360005 count: 132 metaSz: 2577 rp [2] sn: 720006 count: 132 metaSz: 2577 rp [3] sn: 1080007 count: 132 metaSz: 2577 2023-10-21T04:16:16.762+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 123612 recovery UsedSpace 141484 2023-10-21T04:16:16.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:16:16.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.767+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:16:16.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex started 2023-10-21T04:16:16.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:16:16.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.770+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:16:16.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex started 2023-10-21T04:16:16.771+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.771+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.771+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:16.801+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 3 Warmup took 40.652577ms 2023-10-21T04:16:16.801+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 3 WriterThreads 1 2023-10-21T04:16:16.802+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index Index: 9912003609070511414 Partition: 3 2023-10-21T04:16:16.802+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:16.802+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:16.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.803+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:16.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.803+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:16.803+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 124 metaSz: 2577 rp [1] sn: 1080007 count: 124 metaSz: 2577 2023-10-21T04:16:16.804+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 114423 recovery UsedSpace 149002 2023-10-21T04:16:16.804+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 20 count: 124 metaSz: 2576 rp [1] sn: 360005 count: 124 metaSz: 2577 rp [2] sn: 720006 count: 124 metaSz: 2577 rp [3] sn: 1080007 count: 124 metaSz: 2577 2023-10-21T04:16:16.805+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 118325 recovery UsedSpace 136670 2023-10-21T04:16:16.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:16:16.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.811+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:16:16.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex started 2023-10-21T04:16:16.811+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.811+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:16:16.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.815+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:16:16.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex started 2023-10-21T04:16:16.815+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:16.816+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.816+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.836+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 4 Warmup took 33.683283ms 2023-10-21T04:16:16.836+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 4 WriterThreads 1 2023-10-21T04:16:16.836+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index Index: 9912003609070511414 Partition: 4 2023-10-21T04:16:16.837+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:16.837+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:16.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.837+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:16:16.837+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 21 count: 111 metaSz: 2576 rp [1] sn: 360005 count: 111 metaSz: 2577 rp [2] sn: 720006 count: 111 metaSz: 2577 rp [3] sn: 1080007 count: 111 metaSz: 2577 2023-10-21T04:16:16.837+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 117212 recovery UsedSpace 140766 2023-10-21T04:16:16.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.838+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:16.838+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 111 metaSz: 2577 rp [1] sn: 1080007 count: 111 metaSz: 2577 2023-10-21T04:16:16.838+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 112846 recovery UsedSpace 149002 2023-10-21T04:16:16.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:16:16.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.843+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:16:16.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex started 2023-10-21T04:16:16.843+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.843+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:16:16.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.844+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:16:16.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex started 2023-10-21T04:16:16.844+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:16.845+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.845+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.862+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 5 Warmup took 25.176078ms 2023-10-21T04:16:16.862+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 5 WriterThreads 1 2023-10-21T04:16:16.862+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index Index: 9912003609070511414 Partition: 5 2023-10-21T04:16:16.862+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:16.863+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:16.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.863+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:16:16.863+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 140 metaSz: 2577 rp [1] sn: 1080007 count: 140 metaSz: 2577 2023-10-21T04:16:16.863+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 99300 recovery UsedSpace 157912 2023-10-21T04:16:16.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.863+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:16.863+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 140 metaSz: 2577 rp [1] sn: 1080007 count: 140 metaSz: 2577 2023-10-21T04:16:16.864+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 117207 recovery UsedSpace 149002 2023-10-21T04:16:16.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:16:16.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.872+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:16:16.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex started 2023-10-21T04:16:16.872+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.872+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:16:16.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.873+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:16:16.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex started 2023-10-21T04:16:16.873+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:16.873+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.873+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.919+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 6 Warmup took 56.359981ms 2023-10-21T04:16:16.919+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 6 WriterThreads 1 2023-10-21T04:16:16.919+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index Index: 9912003609070511414 Partition: 6 2023-10-21T04:16:16.919+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:16.920+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:16.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:16.920+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:16.920+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 22 count: 125 metaSz: 2576 rp [1] sn: 360005 count: 125 metaSz: 2577 rp [2] sn: 720006 count: 125 metaSz: 2577 rp [3] sn: 1080007 count: 125 metaSz: 2577 2023-10-21T04:16:16.920+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 129666 recovery UsedSpace 141286 2023-10-21T04:16:16.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:16.920+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:16.920+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 125 metaSz: 2577 rp [1] sn: 1080007 count: 125 metaSz: 2577 2023-10-21T04:16:16.921+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 114946 recovery UsedSpace 149002 2023-10-21T04:16:16.921+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:16.936+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex started 2023-10-21T04:16:16.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:16.936+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:16:16.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex started 2023-10-21T04:16:16.936+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:16.938+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:16.938+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:16.959+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 7 Warmup took 39.781761ms 2023-10-21T04:16:16.960+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 7 WriterThreads 1 2023-10-21T04:16:16.960+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index Index: 9912003609070511414 Partition: 7 2023-10-21T04:16:16.960+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 9912003609070511414, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:16.960+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 9912003609070511414, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:16.960+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:16.961+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 9912003609070511414 2023-10-21T04:16:16.961+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:16.961+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:16.961+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_1:1697841976960500515] 2023-10-21T04:16:16.961+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:16.961+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:16.962+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 9912003609070511414 Partitions [7 8 1 2 3 4 5 6] 2023-10-21T04:16:16.964+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:7 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:16:16.965+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:8 Attempting to open snapshot (SnapshotInfo: count:121 committed:false) 2023-10-21T04:16:16.966+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:1 Attempting to open snapshot (SnapshotInfo: count:134 committed:false) 2023-10-21T04:16:16.967+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:2 Attempting to open snapshot (SnapshotInfo: count:113 committed:false) 2023-10-21T04:16:16.968+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:3 Attempting to open snapshot (SnapshotInfo: count:132 committed:false) 2023-10-21T04:16:16.969+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:4 Attempting to open snapshot (SnapshotInfo: count:124 committed:false) 2023-10-21T04:16:16.970+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:5 Attempting to open snapshot (SnapshotInfo: count:111 committed:false) 2023-10-21T04:16:16.971+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:6 Attempting to open snapshot (SnapshotInfo: count:140 committed:false) 2023-10-21T04:16:16.977+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.580026ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:16.977+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 122.288µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:17.027+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[9912003609070511414:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:17.027+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 17656144409930763141, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:17.028+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 17656144409930763141, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:17.154+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.154+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.167+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7315818257769233877,realInstId: 0, indexDefn DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:17.168+05:30 [Info] switched currmeta from 51 -> 54 force true 2023-10-21T04:16:17.168+05:30 [Info] Indexer::handleRecoverIndex InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:17.168+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 643.589µs 2023-10-21T04:16:17.168+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 719.874µs 2023-10-21T04:16:17.168+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.168+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7315818257769233877, indexDefn DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:17.169+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 141.28003ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:17.169+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.169+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.169+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.170+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.171+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.178+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c 2023-10-21T04:16:17.178+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[7659339677706884510:true 10890719122371003479:true] 2023-10-21T04:16:17.178+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.888µs 2023-10-21T04:16:17.178+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [10890719122371003479 7659339677706884510], isLockedForRecovery: true 2023-10-21T04:16:17.178+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [10890719122371003479 7659339677706884510], errMap: map[7659339677706884510: 10890719122371003479:], elapsed: 9.531µs 2023-10-21T04:16:17.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 126 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:17.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.185+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:17.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex started 2023-10-21T04:16:17.185+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.185+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 113 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.190+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex started 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.190+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:17.190+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:16:17.190+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:17.190+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:16:17.190+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:16:17.190+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex stopped 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.190+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.191+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:17.195+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 8669649359784409226, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:17.195+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 8669649359784409226, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:17.214+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:16:17.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex closed 2023-10-21T04:16:17.214+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.214+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.214+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.214+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:16:17.214+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:17.215+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:16:17.215+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:16:17.215+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:17.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex stopped 2023-10-21T04:16:17.215+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.215+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.215+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.215+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.225+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:16:17.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex closed 2023-10-21T04:16:17.225+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:16:17.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 113 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:16:17.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... 2023-10-21T04:16:17.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 126 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... 2023-10-21T04:16:17.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index destroyed 2023-10-21T04:16:17.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.239+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.240+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.245+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.246+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 139 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:17.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.254+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:17.254+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:16:17.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex started 2023-10-21T04:16:17.254+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.254+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.265+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.265+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.268+05:30 [Info] switched currmeta from 54 -> 55 force true 2023-10-21T04:16:17.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 155 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:17.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.271+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:17.271+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:16:17.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex started 2023-10-21T04:16:17.271+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Warmup took 102.520888ms 2023-10-21T04:16:17.272+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 1 WriterThreads 1 2023-10-21T04:16:17.272+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Stopped Billing WUs 2023-10-21T04:16:17.272+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index Index: 7315818257769233877 Partition: 1 2023-10-21T04:16:17.272+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.272+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.273+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.273+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.273+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.273+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.273+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.277+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.277+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 127 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex started 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 114 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex started 2023-10-21T04:16:17.281+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:17.281+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:16:17.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:17.281+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:16:17.281+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:16:17.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:17.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex stopped 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.282+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.282+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.285+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15425360657090425347,realInstId: 0, indexDefn DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:17.285+05:30 [Info] Indexer::handleRecoverIndex InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:17.285+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 313.362µs 2023-10-21T04:16:17.286+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15425360657090425347 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:17.286+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15425360657090425347 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:17.286+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': added 2023-10-21T04:16:17.287+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15425360657090425347, indexDefn DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:17.287+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 91.58658ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:16:17.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex closed 2023-10-21T04:16:17.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is disabled 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:16:17.288+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:17.288+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:16:17.288+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:16:17.288+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:17.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex stopped 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.288+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.288+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner initialized 2023-10-21T04:16:17.288+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:17.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:17.285+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 355.138µs 2023-10-21T04:16:17.289+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner initialized 2023-10-21T04:16:17.289+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:17.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:17.291+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCleaner initialized for recovery 2023-10-21T04:16:17.291+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:17.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery 2023-10-21T04:16:17.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:17.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data], Shared [true] 2023-10-21T04:16:17.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[5386240] Data log Head [0] Tail [5578752] 2023-10-21T04:16:17.291+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [5386240] 2023-10-21T04:16:17.297+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCleaner initialized for recovery 2023-10-21T04:16:17.297+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:16:17.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery 2023-10-21T04:16:17.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:17.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data], Shared [true] 2023-10-21T04:16:17.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[6135808] Data log Head [0] Tail [6340608] 2023-10-21T04:16:17.297+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [6135808] 2023-10-21T04:16:17.299+05:30 [Info] switched currmeta from 55 -> 56 force true 2023-10-21T04:16:17.299+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:16:17.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex closed 2023-10-21T04:16:17.299+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.301+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex 2023-10-21T04:16:17.301+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.301+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:16:17.301+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.301+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:16:17.301+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.302+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:16:17.302+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.302+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:16:17.302+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.302+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.302+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex 2023-10-21T04:16:17.302+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:16:17.302+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.302+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:16:17.303+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.303+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:16:17.303+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:16:17.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 114 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:16:17.304+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.304+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:16:17.304+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.306+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:16:17.306+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.307+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:16:17.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:16:17.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 127 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:16:17.308+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.310+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:16:17.310+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.316+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:16:17.309+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.317+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.317+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:16:17.317+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.317+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:16:17.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index destroyed 2023-10-21T04:16:17.317+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.317+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex 2023-10-21T04:16:17.318+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.318+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex 2023-10-21T04:16:17.319+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.319+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:16:17.320+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.320+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:16:17.321+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.320+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.321+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.321+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex 2023-10-21T04:16:17.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.324+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.326+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.323+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:16:17.326+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.326+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex 2023-10-21T04:16:17.326+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.327+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex 2023-10-21T04:16:17.327+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.327+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex 2023-10-21T04:16:17.327+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.327+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex 2023-10-21T04:16:17.327+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.335+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex 2023-10-21T04:16:17.328+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.329+05:30 [Info] switched currmeta from 56 -> 57 force true 2023-10-21T04:16:17.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 140 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:17.335+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:16:17.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.338+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.338+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:17.338+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.340+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.340+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex 2023-10-21T04:16:17.341+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.341+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex 2023-10-21T04:16:17.341+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.341+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex 2023-10-21T04:16:17.341+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.341+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex 2023-10-21T04:16:17.341+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.341+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex 2023-10-21T04:16:17.342+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.342+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex 2023-10-21T04:16:17.342+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.342+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex 2023-10-21T04:16:17.342+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.342+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex 2023-10-21T04:16:17.342+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.342+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex 2023-10-21T04:16:17.344+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:16:17.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex started 2023-10-21T04:16:17.344+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.344+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 156 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:17.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.349+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:17.349+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:16:17.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex started 2023-10-21T04:16:17.349+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Warmup took 76.91424ms 2023-10-21T04:16:17.349+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 2 WriterThreads 1 2023-10-21T04:16:17.349+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Stopped Billing WUs 2023-10-21T04:16:17.349+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index Index: 7315818257769233877 Partition: 2 2023-10-21T04:16:17.349+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.350+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.350+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.350+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.356+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.359+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex 2023-10-21T04:16:17.356+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.357+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.357+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.360+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex 2023-10-21T04:16:17.362+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.362+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex 2023-10-21T04:16:17.364+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.364+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex 2023-10-21T04:16:17.364+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.364+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex 2023-10-21T04:16:17.365+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.365+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex 2023-10-21T04:16:17.365+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.365+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex 2023-10-21T04:16:17.365+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.365+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex 2023-10-21T04:16:17.366+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.366+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex 2023-10-21T04:16:17.366+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.366+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex 2023-10-21T04:16:17.374+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.374+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 115 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:17.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.374+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:17.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex started 2023-10-21T04:16:17.374+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.374+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.377+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.378+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex 2023-10-21T04:16:17.381+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.381+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex 2023-10-21T04:16:17.383+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.384+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex 2023-10-21T04:16:17.385+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.385+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex 2023-10-21T04:16:17.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 128 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:17.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.387+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:17.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex started 2023-10-21T04:16:17.387+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:17.387+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.387+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.387+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.387+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.387+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:16:17.387+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:17.387+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.388+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:16:17.388+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:16:17.388+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:17.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex stopped 2023-10-21T04:16:17.390+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.390+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.390+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.390+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.389+05:30 [Info] switched currmeta from 57 -> 57 force true 2023-10-21T04:16:17.394+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.394+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex 2023-10-21T04:16:17.394+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.394+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex 2023-10-21T04:16:17.394+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.394+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex 2023-10-21T04:16:17.395+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.395+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex 2023-10-21T04:16:17.398+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.402+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex 2023-10-21T04:16:17.403+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:16:17.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex closed 2023-10-21T04:16:17.412+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.422+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.422+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.423+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.423+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:16:17.423+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:17.423+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:16:17.424+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:16:17.424+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:17.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex stopped 2023-10-21T04:16:17.424+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.424+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.425+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.425+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.433+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.433+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.434+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.434+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:17.434+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:16:17.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex closed 2023-10-21T04:16:17.437+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.437+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.438+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.437+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.437+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.438+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:17.439+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.439+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:17.440+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.440+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.440+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:17.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:16:17.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 115 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:16:17.442+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.442+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:17.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:17.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:17.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:17.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:17.444+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.444+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:17.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.445+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:17.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.445+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:17.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.446+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:17.446+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.446+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:17.446+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.446+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:17.446+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.446+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:17.446+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.446+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:17.447+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.447+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:17.447+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.447+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:17.447+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.447+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:17.447+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.447+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:17.448+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.448+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:17.448+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.448+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:17.448+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.448+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:17.448+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.448+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:17.449+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.449+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:17.449+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.449+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:17.449+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.449+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:17.450+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.450+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex 2023-10-21T04:16:17.451+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : replayOffset: 5566464 (RP:5268593) (CP:5566464) 2023-10-21T04:16:17.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [5566464] took [160.394885ms] 2023-10-21T04:16:17.451+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.451+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10069, memSzIndex 77, numRecords 132 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9253, memSzIndex 76, numRecords 125 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9800, memSzIndex 0, numRecords 123 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9091, memSzIndex 0, numRecords 116 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 11710, memSzIndex 76, numRecords 148 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8687, memSzIndex 0, numRecords 112 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9803, memSzIndex 77, numRecords 129 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9808, memSzIndex 77, numRecords 129 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1056174, resident 1, memSz 67309, memSzIndex 964, numRecords 1000 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9847, memSzIndex 75, numRecords 130 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 7660, memSzIndex 0, numRecords 102 2023-10-21T04:16:17.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.454+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.454+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:17.454+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10119, memSzIndex 77, numRecords 132 2023-10-21T04:16:17.454+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 101617893 controller mem 15880 2023-10-21T04:16:17.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 102666469 controller mem 15880 2023-10-21T04:16:17.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10613, memSzIndex 76, numRecords 137 2023-10-21T04:16:17.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 102666469 controller mem 15880 2023-10-21T04:16:17.456+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9817, memSzIndex 0, numRecords 123 2023-10-21T04:16:17.455+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.456+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:17.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:16:17.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 128 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:16:17.462+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.462+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:17.462+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.462+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:17.456+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 102666469 controller mem 15880 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 104763621 controller mem 15880 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 104763621 controller mem 15880 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 104763621 controller mem 15880 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 104763621 controller mem 15880 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9582, memSzIndex 76, numRecords 127 2023-10-21T04:16:17.462+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 104763621 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10903, memSzIndex 76, numRecords 140 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10709, memSzIndex 76, numRecords 138 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9516, memSzIndex 0, numRecords 120 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1056230, resident 1, memSz 67413, memSzIndex 1013, numRecords 1000 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8462, memSzIndex 0, numRecords 110 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053364, resident 1, memSz 8079, memSzIndex 0, numRecords 106 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1056168, resident 1, memSz 67228, memSzIndex 966, numRecords 1000 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9787, memSzIndex 92, numRecords 129 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9174, memSzIndex 0, numRecords 117 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9791, memSzIndex 0, numRecords 123 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8911, memSzIndex 0, numRecords 124 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9705, memSzIndex 77, numRecords 128 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.463+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 105812197 controller mem 15880 2023-10-21T04:16:17.463+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [5578752] replayOffset [5566464] 2023-10-21T04:16:17.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [172.813841ms] 2023-10-21T04:16:17.464+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.464+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.464+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.464+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.464+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.463+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.464+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:17.465+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.465+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:17.466+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.466+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:17.466+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.466+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:17.466+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:17.467+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:17.467+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:17.467+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:17.468+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.468+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:17.468+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.468+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:17.469+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.469+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:17.469+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.469+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:17.469+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.469+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:17.470+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:17.470+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:17.470+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:17.470+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.471+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:17.471+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.471+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:17.471+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.471+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:17.471+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.471+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:17.472+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.472+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:17.472+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.472+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:17.473+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.473+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex 2023-10-21T04:16:17.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index destroyed 2023-10-21T04:16:17.464+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.485+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:36028:172.31.5.112:12000] 2023-10-21T04:16:17.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.494+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : replayOffset: 6332416 (RP:6324224) (CP:6332416) 2023-10-21T04:16:17.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [6332416] took [197.775412ms] 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12671, memSzIndex 0, numRecords 124 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 15022, memSzIndex 0, numRecords 148 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13443, memSzIndex 0, numRecords 132 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 10516, memSzIndex 0, numRecords 102 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 14061, memSzIndex 0, numRecords 138 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13095, memSzIndex 0, numRecords 128 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11503, memSzIndex 0, numRecords 112 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13968, memSzIndex 0, numRecords 137 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12589, memSzIndex 0, numRecords 123 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11286, memSzIndex 0, numRecords 110 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1054618, resident 1, memSz 56829, memSzIndex 398, numRecords 1000 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1055776, resident 1, memSz 65938, memSzIndex 473, numRecords 1000 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1055776, resident 0.07, memSz 437, memSzIndex 0, numRecords 27 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13194, memSzIndex 0, numRecords 129 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12572, memSzIndex 0, numRecords 123 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1056522, resident 1, memSz 76940, memSzIndex 281, numRecords 1000 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1055282, resident 1, memSz 64862, memSzIndex 245, numRecords 1000 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12563, memSzIndex 0, numRecords 123 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.497+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.497+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12978, memSzIndex 0, numRecords 127 2023-10-21T04:16:17.497+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.498+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.494+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1054936, resident 1, memSz 60847, memSzIndex 306, numRecords 1000 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11970, memSzIndex 0, numRecords 117 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13493, memSzIndex 0, numRecords 132 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12801, memSzIndex 0, numRecords 125 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12300, memSzIndex 0, numRecords 120 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13233, memSzIndex 0, numRecords 130 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11891, memSzIndex 0, numRecords 116 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 14247, memSzIndex 0, numRecords 140 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 10919, memSzIndex 0, numRecords 106 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13189, memSzIndex 0, numRecords 129 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1056524, resident 1, memSz 76936, memSzIndex 283, numRecords 1000 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13159, memSzIndex 0, numRecords 129 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 129935117 controller mem 17010 2023-10-21T04:16:17.500+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [6340608] replayOffset [6332416] 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data], Shared [true]. Built [54] plasmas, took [208.94742ms] 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [203.438448ms] 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.502+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.503+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.503+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.503+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.504+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.504+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.504+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.504+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.504+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:16:17.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data], Shared [true]. Built [61] plasmas, took [209.402603ms] 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.507+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.507+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 141 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:17.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.508+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:17.508+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:16:17.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex started 2023-10-21T04:16:17.509+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.509+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 124 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:17.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:17.511+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swapper started 2023-10-21T04:16:17.511+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex started 2023-10-21T04:16:17.511+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.511+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 111 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:17.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:17.519+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex started 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.519+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15425360657090425347 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:17.519+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15425360657090425347 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:16:17.519+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:16:17.519+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex stopped 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.519+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.520+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 157 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:17.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.520+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:17.520+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:16:17.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex started 2023-10-21T04:16:17.520+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Warmup took 170.009213ms 2023-10-21T04:16:17.520+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 3 WriterThreads 1 2023-10-21T04:16:17.520+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Stopped Billing WUs 2023-10-21T04:16:17.521+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index Index: 7315818257769233877 Partition: 3 2023-10-21T04:16:17.521+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.521+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.521+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.520+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.520+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.523+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:16:17.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex closed 2023-10-21T04:16:17.524+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.524+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.524+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.524+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:16:17.524+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.524+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:16:17.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.524+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.526+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 129 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:17.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.535+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:17.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 116 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:17.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.535+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:17.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex started 2023-10-21T04:16:17.535+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.535+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex started 2023-10-21T04:16:17.535+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:17.535+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.535+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.535+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.535+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.535+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:16:17.536+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:17.536+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:16:17.536+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:16:17.536+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:17.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex stopped 2023-10-21T04:16:17.536+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.536+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.536+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.536+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.536+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.542+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:16:17.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex closed 2023-10-21T04:16:17.542+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.542+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.543+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.543+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:16:17.543+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:17.543+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:16:17.543+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:16:17.543+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:17.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex stopped 2023-10-21T04:16:17.543+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.543+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.543+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.543+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.551+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:16:17.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex closed 2023-10-21T04:16:17.554+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:16:17.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 116 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:16:17.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:16:17.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 129 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:16:17.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex sucessfully destroyed 2023-10-21T04:16:17.602+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36054:172.31.5.112:12000] 2023-10-21T04:16:17.612+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:16:17.612+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex stopped 2023-10-21T04:16:17.612+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.613+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index destroyed 2023-10-21T04:16:17.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.619+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.620+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.624+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:16:17.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex closed 2023-10-21T04:16:17.625+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex ... 2023-10-21T04:16:17.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 111 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex ... 2023-10-21T04:16:17.635+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.636+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 142 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:17.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.645+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:17.645+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:16:17.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex started 2023-10-21T04:16:17.645+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.645+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex ... 2023-10-21T04:16:17.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 124 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex ... 2023-10-21T04:16:17.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 158 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:17.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.654+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:17.654+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:16:17.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex started 2023-10-21T04:16:17.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Warmup took 132.892545ms 2023-10-21T04:16:17.655+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 4 WriterThreads 1 2023-10-21T04:16:17.655+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Stopped Billing WUs 2023-10-21T04:16:17.655+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index Index: 7315818257769233877 Partition: 4 2023-10-21T04:16:17.655+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.655+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.655+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.655+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index destroyed 2023-10-21T04:16:17.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.656+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.657+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.658+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.658+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.659+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36052:172.31.5.112:12000] 2023-10-21T04:16:17.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:17.661+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.661+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:17.665+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.665+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 117 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:17.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.665+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:17.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex started 2023-10-21T04:16:17.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 130 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:17.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.666+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:17.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex started 2023-10-21T04:16:17.666+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:17.666+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.666+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.666+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.666+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.666+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:16:17.667+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:17.667+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:16:17.667+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:16:17.667+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:17.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex stopped 2023-10-21T04:16:17.667+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.667+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.667+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.667+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.667+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 155 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:17.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:17.669+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.669+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:16:17.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex started 2023-10-21T04:16:17.669+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.670+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 139 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:17.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:17.674+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.674+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:16:17.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex started 2023-10-21T04:16:17.675+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15425360657090425347 PartitionId 0 Warmup took 388.589449ms 2023-10-21T04:16:17.675+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15425360657090425347 partitionId 0 WriterThreads 4 2023-10-21T04:16:17.675+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15425360657090425347 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index Index: 15425360657090425347 Partition: 0 2023-10-21T04:16:17.675+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15425360657090425347, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:17.675+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15425360657090425347, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:17.675+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:17.676+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15425360657090425347 2023-10-21T04:16:17.676+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:17.676+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:17.676+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:17.676+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:17.676+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:17.676+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15425360657090425347 Partitions [0] 2023-10-21T04:16:17.676+05:30 [Info] StorageMgr::openSnapshot IndexInst:15425360657090425347 Partition:0 No Snapshot Found. 2023-10-21T04:16:17.677+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15425360657090425347 Adding Nil Snapshot. 2023-10-21T04:16:17.677+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.677+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:16:17.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex closed 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:16:17.679+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:17.679+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:16:17.679+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:16:17.679+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:17.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex stopped 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.679+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.680+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.685+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:16:17.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex closed 2023-10-21T04:16:17.685+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:16:17.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 117 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:16:17.688+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.553452ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:17.689+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 149.44µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:17.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:16:17.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 130 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:16:17.696+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.696+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index destroyed 2023-10-21T04:16:17.700+05:30 [Info] switched currmeta from 57 -> 58 force true 2023-10-21T04:16:17.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.703+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.703+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.707+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.707+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 143 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:17.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.710+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:17.710+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:16:17.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex started 2023-10-21T04:16:17.710+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.710+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 159 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:17.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.714+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:17.714+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:16:17.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex started 2023-10-21T04:16:17.714+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Warmup took 59.122584ms 2023-10-21T04:16:17.715+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.715+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.715+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 5 WriterThreads 1 2023-10-21T04:16:17.715+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Stopped Billing WUs 2023-10-21T04:16:17.715+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index Index: 7315818257769233877 Partition: 5 2023-10-21T04:16:17.715+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.715+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.715+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.716+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.717+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.717+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 131 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.723+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex started 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 118 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.723+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex started 2023-10-21T04:16:17.723+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:17.723+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:16:17.723+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:17.723+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:16:17.723+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:16:17.723+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:17.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex stopped 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.723+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.724+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.724+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.729+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36056:172.31.5.112:12000] 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:16:17.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex closed 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:16:17.731+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:17.731+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:16:17.731+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:16:17.731+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:17.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex stopped 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.742+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:16:17.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex closed 2023-10-21T04:16:17.742+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:16:17.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 118 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:16:17.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:16:17.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 131 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:16:17.748+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36058:172.31.5.112:12000] 2023-10-21T04:16:17.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index destroyed 2023-10-21T04:16:17.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.757+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.760+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.760+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 144 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:17.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:17.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:16:17.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex started 2023-10-21T04:16:17.769+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.769+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 160 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:17.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.773+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:17.773+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:16:17.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex started 2023-10-21T04:16:17.773+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Warmup took 57.929365ms 2023-10-21T04:16:17.773+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.773+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.774+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 6 WriterThreads 1 2023-10-21T04:16:17.774+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Stopped Billing WUs 2023-10-21T04:16:17.774+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index Index: 7315818257769233877 Partition: 6 2023-10-21T04:16:17.774+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.774+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.775+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.775+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.775+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.785+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36062:172.31.5.112:12000] 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 119 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.786+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex started 2023-10-21T04:16:17.786+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 132 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.786+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex started 2023-10-21T04:16:17.786+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:17.786+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.786+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.786+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.786+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.786+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.786+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:16:17.786+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:17.786+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:16:17.786+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:16:17.786+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:17.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex stopped 2023-10-21T04:16:17.786+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.787+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.787+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.787+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.787+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.811+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.811+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.811+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:16:17.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex closed 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:16:17.812+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:17.812+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:16:17.812+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:16:17.812+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:17.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex stopped 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.812+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.815+05:30 [Info] switched currmeta from 58 -> 59 force true 2023-10-21T04:16:17.819+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:16:17.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex closed 2023-10-21T04:16:17.819+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:16:17.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 119 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:16:17.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:16:17.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 132 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:16:17.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index destroyed 2023-10-21T04:16:17.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.832+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.832+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.837+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.837+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36064:172.31.5.112:12000] 2023-10-21T04:16:17.837+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 145 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:17.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.849+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:17.849+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:16:17.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex started 2023-10-21T04:16:17.849+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.861+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36060:172.31.5.112:12000] 2023-10-21T04:16:17.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 161 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:17.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.866+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:17.866+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:16:17.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex started 2023-10-21T04:16:17.866+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Warmup took 92.237747ms 2023-10-21T04:16:17.867+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 7 WriterThreads 1 2023-10-21T04:16:17.867+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Stopped Billing WUs 2023-10-21T04:16:17.867+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index Index: 7315818257769233877 Partition: 7 2023-10-21T04:16:17.867+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:17.867+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:17.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.868+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.868+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:17.868+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.868+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.869+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:17.869+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:17.878+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36066:172.31.5.112:12000] 2023-10-21T04:16:17.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 112 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:17.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.884+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:17.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex started 2023-10-21T04:16:17.884+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.884+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 125 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:17.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.885+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:17.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex started 2023-10-21T04:16:17.886+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:17.886+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:16:17.886+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:17.886+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:16:17.886+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:16:17.886+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:17.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex stopped 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.895+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:16:17.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex closed 2023-10-21T04:16:17.895+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:17.895+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:17.895+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:17.895+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:17.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:16:17.895+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:17.896+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:16:17.896+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:16:17.896+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:17.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex stopped 2023-10-21T04:16:17.896+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:17.896+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:17.896+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:17.896+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:17.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:17.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:16:17.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex closed 2023-10-21T04:16:17.902+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:17.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:16:17.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 112 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:16:17.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex sucessfully destroyed 2023-10-21T04:16:17.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... 2023-10-21T04:16:17.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 125 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... 2023-10-21T04:16:17.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index destroyed 2023-10-21T04:16:17.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:17.918+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.919+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.922+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36070:172.31.5.112:12000] 2023-10-21T04:16:17.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:17.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:17.923+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:17.923+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:17.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 146 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:17.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:17.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:17.934+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:17.934+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:16:17.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex started 2023-10-21T04:16:17.934+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.934+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 162 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:17.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:17.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:17.935+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:17.935+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:16:17.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex started 2023-10-21T04:16:17.936+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Warmup took 68.597267ms 2023-10-21T04:16:17.936+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:17.936+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:17.937+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 8 WriterThreads 1 2023-10-21T04:16:17.938+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Stopped Billing WUs 2023-10-21T04:16:17.938+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index Index: 7315818257769233877 Partition: 8 2023-10-21T04:16:17.938+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7315818257769233877, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:17.938+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7315818257769233877, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:17.938+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:17.940+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7315818257769233877 2023-10-21T04:16:17.941+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7315818257769233877 Name: idx_partitioned_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:17.941+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:17.941+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:17.941+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7315818257769233877 Name: idx_partitioned_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:17.942+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:17.947+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7315818257769233877 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:16:17.947+05:30 [Info] StorageMgr::openSnapshot IndexInst:7315818257769233877 Partition:8 No Snapshot Found. 2023-10-21T04:16:17.947+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7315818257769233877 Adding Nil Snapshot. 2023-10-21T04:16:17.955+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.671909ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:17.955+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 168.689µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:17.974+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:17.975+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:17.980+05:30 [Info] switched currmeta from 59 -> 60 force true 2023-10-21T04:16:17.990+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36068:172.31.5.112:12000] 2023-10-21T04:16:17.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9918189901590968, throttleDelayMs (new, old, change): (4, 0, 4) 2023-10-21T04:16:18.182+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[7315818257769233877:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:18.182+05:30 [Info] Indexer::handleMergePartition Source 11579858587830428859 Target 7315818257769233877 2023-10-21T04:16:18.182+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00877b740 2023-10-21T04:16:18.182+05:30 [Info] MergePartition: sent async request to update index instance 7315818257769233877 rstate moved to ACTIVE 2023-10-21T04:16:18.194+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.125209ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:18.194+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 11579858587830428859, realInst: 7315818257769233877 2023-10-21T04:16:18.195+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 14292583894719865678, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:18.195+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 14292583894719865678, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:18.209+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.214+05:30 [Info] switched currmeta from 60 -> 61 force true 2023-10-21T04:16:18.226+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3100906924534408756,realInstId: 0, indexDefn DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:18.226+05:30 [Info] Indexer::handleRecoverIndex InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:18.226+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 336.621µs 2023-10-21T04:16:18.226+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 384.45µs 2023-10-21T04:16:18.227+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3100906924534408756 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:18.227+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3100906924534408756 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:18.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:18.227+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1056116 memSz 2026 2023-10-21T04:16:18.227+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 16 numRPs: 3 rp [0] sn: 360005 count: 1000 metaSz: 2578 rp [1] sn: 720006 count: 1000 metaSz: 2578 rp [2] sn: 1080007 count: 1000 metaSz: 2578 2023-10-21T04:16:18.228+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 200953 recovery UsedSpace 153270 2023-10-21T04:16:18.230+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3100906924534408756, indexDefn DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:18.231+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 35.446974ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:18.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:18.231+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1055884 memSz 1120 2023-10-21T04:16:18.231+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 1000 metaSz: 2578 rp [1] sn: 1080007 count: 1000 metaSz: 2578 2023-10-21T04:16:18.231+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 215592 recovery UsedSpace 157521 2023-10-21T04:16:18.241+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.241+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.244+05:30 [Info] switched currmeta from 61 -> 62 force true 2023-10-21T04:16:18.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex 2023-10-21T04:16:18.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:18.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:18.247+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex 2023-10-21T04:16:18.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex started 2023-10-21T04:16:18.247+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.247+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex 2023-10-21T04:16:18.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:18.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:18.252+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex 2023-10-21T04:16:18.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex started 2023-10-21T04:16:18.252+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.253+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.253+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3100906924534408756 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:18.285+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3100906924534408756 PartitionId 0 Warmup took 58.253332ms 2023-10-21T04:16:18.286+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3100906924534408756 partitionId 0 WriterThreads 4 2023-10-21T04:16:18.286+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3100906924534408756 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:18.286+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3100906924534408756 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index Index: 3100906924534408756 Partition: 0 2023-10-21T04:16:18.286+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3100906924534408756, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:18.286+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3100906924534408756, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:18.286+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:18.286+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3100906924534408756 2023-10-21T04:16:18.287+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:18.287+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:18.287+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:18.287+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:18.287+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:18.287+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3100906924534408756 Partitions [0] 2023-10-21T04:16:18.289+05:30 [Info] StorageMgr::openSnapshot IndexInst:3100906924534408756 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:18.302+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.75033ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:18.302+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 80.124µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:18.308+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[15425360657090425347:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:18.308+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc008acefc0 2023-10-21T04:16:18.308+05:30 [Info] handleUpdateIndexRState: Index instance 15425360657090425347 rstate moved to ACTIVE 2023-10-21T04:16:18.313+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 4.488439ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:18.314+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 2374548411389959576, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:18.314+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 2374548411389959576, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:18.319+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.320+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.322+05:30 [Info] switched currmeta from 62 -> 65 force true 2023-10-21T04:16:18.330+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.330+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.335+05:30 [Info] switched currmeta from 65 -> 65 force true 2023-10-21T04:16:18.378+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12762948908485355776,realInstId: 0, indexDefn DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:18.392+05:30 [Info] Indexer::handleRecoverIndex InstId: 12762948908485355776 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:18.394+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 1.765177ms 2023-10-21T04:16:18.394+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:18.394+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12762948908485355776, indexDefn DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:18.395+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 81.318489ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:18.395+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 2.390617ms 2023-10-21T04:16:18.401+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:18.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.402+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.403+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.403+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:18.404+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:18.404+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.405+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 126 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:18.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.410+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:18.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex started 2023-10-21T04:16:18.410+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.410+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.412+05:30 [Info] switched currmeta from 65 -> 66 force true 2023-10-21T04:16:18.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 113 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:16:18.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.421+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:16:18.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex started 2023-10-21T04:16:18.422+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:18.422+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.422+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.423+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:18.423+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.424+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.424+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:16:18.424+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:18.424+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:16:18.425+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:16:18.425+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:18.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex stopped 2023-10-21T04:16:18.425+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.425+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.425+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.425+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.437+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:16:18.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex closed 2023-10-21T04:16:18.438+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:18.439+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.439+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.439+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.440+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:16:18.440+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:16:18.440+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:16:18.440+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:16:18.440+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:16:18.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex stopped 2023-10-21T04:16:18.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.444+05:30 [Info] switched currmeta from 66 -> 67 force true 2023-10-21T04:16:18.452+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:16:18.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex closed 2023-10-21T04:16:18.454+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:18.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex ... 2023-10-21T04:16:18.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 113 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex ... 2023-10-21T04:16:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex sucessfully destroyed 2023-10-21T04:16:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex ... 2023-10-21T04:16:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 126 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex ... 2023-10-21T04:16:18.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index destroyed 2023-10-21T04:16:18.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.494+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.495+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.504+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.505+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 140 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:16:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.511+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:16:18.511+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:16:18.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex started 2023-10-21T04:16:18.511+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.511+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 156 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:18.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.518+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:18.518+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:16:18.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex started 2023-10-21T04:16:18.518+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.518+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 4 Warmup took 116.040526ms 2023-10-21T04:16:18.518+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.518+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 4 WriterThreads 1 2023-10-21T04:16:18.518+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index Index: 12762948908485355776 Partition: 4 2023-10-21T04:16:18.518+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:18.518+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:18.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.519+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.519+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.519+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:18.520+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:18.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 127 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:18.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.537+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:18.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex started 2023-10-21T04:16:18.537+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.537+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 114 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:18.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.542+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:18.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex started 2023-10-21T04:16:18.542+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:18.542+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:18.542+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.542+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.543+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:16:18.543+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:18.543+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:16:18.543+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:16:18.543+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:18.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex stopped 2023-10-21T04:16:18.543+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.543+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.544+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.544+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.544+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.544+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.548+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:18.550+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:16:18.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex closed 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:16:18.551+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:16:18.551+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:18.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex stopped 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.562+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:16:18.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex closed 2023-10-21T04:16:18.562+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:18.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex ... 2023-10-21T04:16:18.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 114 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex ... 2023-10-21T04:16:18.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex sucessfully destroyed 2023-10-21T04:16:18.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex ... 2023-10-21T04:16:18.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 127 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex ... 2023-10-21T04:16:18.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index destroyed 2023-10-21T04:16:18.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.574+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.579+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 141 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:18.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.580+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:18.580+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:16:18.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex started 2023-10-21T04:16:18.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 157 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:18.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.585+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:18.586+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:16:18.586+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex started 2023-10-21T04:16:18.586+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 5 Warmup took 67.550405ms 2023-10-21T04:16:18.586+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.586+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.587+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 5 WriterThreads 1 2023-10-21T04:16:18.587+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index Index: 12762948908485355776 Partition: 5 2023-10-21T04:16:18.588+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:18.592+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:18.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.595+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.595+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:18.599+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.599+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 128 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:18.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.600+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:18.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex started 2023-10-21T04:16:18.600+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.601+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.601+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:18.602+05:30 [Info] switched currmeta from 67 -> 68 force true 2023-10-21T04:16:18.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 115 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:18.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.610+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:18.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex started 2023-10-21T04:16:18.610+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.610+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.611+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:18.611+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:18.612+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.612+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.612+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:16:18.613+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:18.613+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:16:18.613+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:16:18.614+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:18.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex stopped 2023-10-21T04:16:18.614+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.615+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.616+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.616+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.618+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.618+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.625+05:30 [Info] switched currmeta from 68 -> 68 force true 2023-10-21T04:16:18.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:16:18.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex closed 2023-10-21T04:16:18.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:18.629+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.629+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.629+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:16:18.629+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:18.629+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:16:18.629+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:16:18.629+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:18.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex stopped 2023-10-21T04:16:18.630+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.630+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.630+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.630+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.640+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:16:18.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex closed 2023-10-21T04:16:18.640+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:18.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex ... 2023-10-21T04:16:18.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 115 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex ... 2023-10-21T04:16:18.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex sucessfully destroyed 2023-10-21T04:16:18.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex ... 2023-10-21T04:16:18.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 128 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex ... 2023-10-21T04:16:18.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index destroyed 2023-10-21T04:16:18.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.649+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.656+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.656+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 142 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:18.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.661+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:18.661+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:16:18.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex started 2023-10-21T04:16:18.662+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.662+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 158 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:18.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.666+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:18.666+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:16:18.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex started 2023-10-21T04:16:18.666+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 6 Warmup took 72.163142ms 2023-10-21T04:16:18.667+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.667+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 6 WriterThreads 1 2023-10-21T04:16:18.667+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index Index: 12762948908485355776 Partition: 6 2023-10-21T04:16:18.667+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:18.667+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.667+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:18.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.667+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.667+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.668+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:18.668+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:18.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 116 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:18.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.676+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:18.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex started 2023-10-21T04:16:18.677+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.677+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 129 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:18.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.677+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:18.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex started 2023-10-21T04:16:18.678+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:18.678+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:16:18.678+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:16:18.678+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:18.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex stopped 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.679+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.683+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:16:18.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex closed 2023-10-21T04:16:18.683+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:18.683+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:16:18.684+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:16:18.684+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:18.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex stopped 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.684+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.691+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:16:18.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex closed 2023-10-21T04:16:18.692+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:18.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex ... 2023-10-21T04:16:18.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 116 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex ... 2023-10-21T04:16:18.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex sucessfully destroyed 2023-10-21T04:16:18.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex ... 2023-10-21T04:16:18.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 129 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex ... 2023-10-21T04:16:18.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index destroyed 2023-10-21T04:16:18.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.702+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.702+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.706+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.706+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 143 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:18.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.717+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:18.717+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:16:18.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex started 2023-10-21T04:16:18.717+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.717+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 159 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:18.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.719+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:18.719+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:16:18.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex started 2023-10-21T04:16:18.719+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 7 Warmup took 51.662608ms 2023-10-21T04:16:18.719+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.719+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.719+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 7 WriterThreads 1 2023-10-21T04:16:18.719+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index Index: 12762948908485355776 Partition: 7 2023-10-21T04:16:18.719+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:18.719+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:18.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.720+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.720+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:18.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:18.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 130 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:18.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.725+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:18.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex started 2023-10-21T04:16:18.725+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.725+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 117 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:18.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.726+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:18.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex started 2023-10-21T04:16:18.726+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:18.726+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:16:18.726+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:16:18.726+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:18.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex stopped 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.727+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.727+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.738+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:16:18.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex closed 2023-10-21T04:16:18.738+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:18.738+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.738+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.738+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:16:18.739+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:18.739+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:16:18.739+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:16:18.739+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:18.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex stopped 2023-10-21T04:16:18.739+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.739+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.739+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.739+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.750+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:16:18.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex closed 2023-10-21T04:16:18.751+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:18.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex ... 2023-10-21T04:16:18.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 117 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex ... 2023-10-21T04:16:18.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex sucessfully destroyed 2023-10-21T04:16:18.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex ... 2023-10-21T04:16:18.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 130 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex ... 2023-10-21T04:16:18.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index destroyed 2023-10-21T04:16:18.762+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.762+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.763+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.767+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.768+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 144 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:18.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.770+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:18.770+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:16:18.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex started 2023-10-21T04:16:18.770+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.770+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 160 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:18.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.777+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:18.777+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:16:18.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex started 2023-10-21T04:16:18.777+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 8 Warmup took 57.178858ms 2023-10-21T04:16:18.777+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 8 WriterThreads 1 2023-10-21T04:16:18.777+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index Index: 12762948908485355776 Partition: 8 2023-10-21T04:16:18.777+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:18.777+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:18.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.778+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.778+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:18.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.779+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:18.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 131 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:18.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:18.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex started 2023-10-21T04:16:18.789+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.789+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 118 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:18.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.794+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:18.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex started 2023-10-21T04:16:18.794+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:18.794+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:16:18.794+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:16:18.794+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:18.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex stopped 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.795+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.795+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.795+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.801+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:16:18.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex closed 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:16:18.802+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:16:18.802+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:18.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex stopped 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.802+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.824+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.824+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.824+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.824+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:16:18.824+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex closed 2023-10-21T04:16:18.824+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:18.824+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex ... 2023-10-21T04:16:18.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 118 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex ... 2023-10-21T04:16:18.829+05:30 [Info] switched currmeta from 68 -> 69 force true 2023-10-21T04:16:18.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex sucessfully destroyed 2023-10-21T04:16:18.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex ... 2023-10-21T04:16:18.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 131 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex ... 2023-10-21T04:16:18.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index destroyed 2023-10-21T04:16:18.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.867+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 161 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:18.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.878+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:18.878+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:16:18.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex started 2023-10-21T04:16:18.878+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.878+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.880+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.885+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:18.885+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:18.891+05:30 [Info] switched currmeta from 69 -> 70 force true 2023-10-21T04:16:18.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 145 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:18.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.912+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:18.916+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:16:18.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex started 2023-10-21T04:16:18.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.916+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 1 Warmup took 138.741764ms 2023-10-21T04:16:18.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.917+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 1 WriterThreads 1 2023-10-21T04:16:18.917+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index Index: 12762948908485355776 Partition: 1 2023-10-21T04:16:18.917+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:18.917+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:18.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.917+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.917+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:18.918+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:18.918+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:18.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 132 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:18.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:18.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:18.931+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:18.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex started 2023-10-21T04:16:18.931+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.931+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 119 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:18.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.936+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:18.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex started 2023-10-21T04:16:18.936+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:18.936+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:18.936+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.936+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.936+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:16:18.936+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:18.936+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:16:18.936+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:16:18.937+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:18.937+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.937+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex stopped 2023-10-21T04:16:18.938+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.938+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.938+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.938+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.947+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:16:18.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex closed 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:18.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:16:18.948+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:16:18.948+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:18.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex stopped 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:18.948+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:18.957+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:18.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:16:18.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex closed 2023-10-21T04:16:18.957+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:18.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex ... 2023-10-21T04:16:18.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 119 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex ... 2023-10-21T04:16:18.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex sucessfully destroyed 2023-10-21T04:16:18.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex ... 2023-10-21T04:16:18.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 132 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex ... 2023-10-21T04:16:18.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index destroyed 2023-10-21T04:16:18.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:18.972+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.972+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:18.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:18.978+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:18.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:18.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 146 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:18.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:18.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:18.987+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:18.987+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:16:18.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex started 2023-10-21T04:16:18.987+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:18.987+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:18.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9850265027602356, throttleDelayMs (new, old, change): (8, 4, 4) 2023-10-21T04:16:18.995+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno cache..., established connections: map[172.31.5.112:36178:172.31.5.112:12000] 2023-10-21T04:16:19.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 162 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:19.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.005+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:19.005+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:16:19.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex started 2023-10-21T04:16:19.005+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 2 Warmup took 88.188222ms 2023-10-21T04:16:19.005+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.005+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.006+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 2 WriterThreads 1 2023-10-21T04:16:19.006+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index Index: 12762948908485355776 Partition: 2 2023-10-21T04:16:19.006+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12762948908485355776 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.006+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12762948908485355776 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.006+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:19.007+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:19.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.007+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:19.007+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:19.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 112 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:19.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.020+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:19.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex started 2023-10-21T04:16:19.021+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.021+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 125 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:19.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.021+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:19.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex started 2023-10-21T04:16:19.021+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.021+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.022+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:19.022+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:19.022+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:19.022+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:19.022+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:19.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:16:19.023+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:19.023+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:16:19.024+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:16:19.024+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:19.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex stopped 2023-10-21T04:16:19.025+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:19.026+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:19.026+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:19.027+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:19.044+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:19.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:16:19.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex closed 2023-10-21T04:16:19.045+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:19.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:19.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:19.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:19.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:16:19.048+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:19.048+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:16:19.049+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:16:19.049+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:19.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex stopped 2023-10-21T04:16:19.049+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:19.049+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:19.049+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:19.049+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:19.058+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:19.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:16:19.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex closed 2023-10-21T04:16:19.059+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:19.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex ... 2023-10-21T04:16:19.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 112 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex ... 2023-10-21T04:16:19.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex sucessfully destroyed 2023-10-21T04:16:19.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex ... 2023-10-21T04:16:19.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 125 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex ... 2023-10-21T04:16:19.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index destroyed 2023-10-21T04:16:19.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.095+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:19.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:19.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.103+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:19.103+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:19.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 147 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:19.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.112+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:19.112+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:16:19.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex started 2023-10-21T04:16:19.112+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.113+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 163 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:19.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.122+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:19.122+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:16:19.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex started 2023-10-21T04:16:19.122+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12762948908485355776 PartitionId 3 Warmup took 116.325466ms 2023-10-21T04:16:19.123+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.123+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12762948908485355776 partitionId 3 WriterThreads 1 2023-10-21T04:16:19.123+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12762948908485355776 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index Index: 12762948908485355776 Partition: 3 2023-10-21T04:16:19.123+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12762948908485355776, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:19.123+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12762948908485355776, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:19.123+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12762948908485355776 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:19.124+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12762948908485355776 2023-10-21T04:16:19.124+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12762948908485355776 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:19.124+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:19.124+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:19.124+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12762948908485355776 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:19.124+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12762948908485355776 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:19.125+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12762948908485355776 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:16:19.125+05:30 [Info] StorageMgr::openSnapshot IndexInst:12762948908485355776 Partition:4 No Snapshot Found. 2023-10-21T04:16:19.125+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12762948908485355776 Adding Nil Snapshot. 2023-10-21T04:16:19.123+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.135+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.648906ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.135+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 462.827µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:19.145+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.145+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.154+05:30 [Info] switched currmeta from 70 -> 71 force true 2023-10-21T04:16:19.218+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.218+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.230+05:30 [Info] switched currmeta from 71 -> 72 force true 2023-10-21T04:16:19.236+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3100906924534408756:INDEX_STATE_RECOVERED 9912003609070511414:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:19.237+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 15945117906390072636, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:19.237+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 15945117906390072636, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:19.264+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14343533858105260407,realInstId: 0, indexDefn DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:19.264+05:30 [Info] Indexer::handleRecoverIndex InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:19.264+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 365.884µs 2023-10-21T04:16:19.264+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 413.652µs 2023-10-21T04:16:19.264+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14343533858105260407 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:19.265+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14343533858105260407 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:19.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:19.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:16:19.266+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14343533858105260407, indexDefn DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:19.266+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.181068ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:19.267+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:19.267+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:19.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 124 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:19.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:19.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:19.282+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:19.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex started 2023-10-21T04:16:19.282+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.283+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 111 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:19.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:19.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:19.284+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:19.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex started 2023-10-21T04:16:19.284+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14343533858105260407 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:19.284+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14343533858105260407 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:19.284+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:19.285+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.285+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.285+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.285+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.284+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:19.285+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:19.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:16:19.285+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:19.285+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:16:19.285+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:16:19.285+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:19.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex stopped 2023-10-21T04:16:19.285+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:19.285+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:19.285+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:19.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:19.287+05:30 [Info] switched currmeta from 72 -> 73 force true 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:16:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex closed 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:16:19.300+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:19.300+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:16:19.300+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:16:19.300+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:19.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex stopped 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:19.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:19.306+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36238:172.31.5.112:12000] 2023-10-21T04:16:19.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:19.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:16:19.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex closed 2023-10-21T04:16:19.311+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:19.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex ... 2023-10-21T04:16:19.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 111 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex ... 2023-10-21T04:16:19.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex sucessfully destroyed 2023-10-21T04:16:19.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex ... 2023-10-21T04:16:19.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 124 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex ... 2023-10-21T04:16:19.321+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36240:172.31.5.112:12000] 2023-10-21T04:16:19.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index destroyed 2023-10-21T04:16:19.362+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.362+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:19.363+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:19.363+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:19.366+05:30 [Info] switched currmeta from 73 -> 74 force true 2023-10-21T04:16:19.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:19.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:19.368+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:19.368+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:19.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 147 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:19.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:19.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:19.372+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:19.372+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:16:19.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex started 2023-10-21T04:16:19.372+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.372+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 163 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:19.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:19.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:19.376+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:19.376+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:16:19.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex started 2023-10-21T04:16:19.376+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14343533858105260407 PartitionId 0 Warmup took 110.788878ms 2023-10-21T04:16:19.376+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14343533858105260407 partitionId 0 WriterThreads 4 2023-10-21T04:16:19.376+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14343533858105260407 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:19.376+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14343533858105260407 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index Index: 14343533858105260407 Partition: 0 2023-10-21T04:16:19.376+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14343533858105260407, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:19.376+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14343533858105260407, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:19.376+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:19.377+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14343533858105260407 2023-10-21T04:16:19.377+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:19.377+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:19.377+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:19.377+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:19.377+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:19.378+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14343533858105260407 Partitions [0] 2023-10-21T04:16:19.378+05:30 [Info] StorageMgr::openSnapshot IndexInst:14343533858105260407 Partition:0 No Snapshot Found. 2023-10-21T04:16:19.378+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14343533858105260407 Adding Nil Snapshot. 2023-10-21T04:16:19.378+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.378+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.390+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.858909ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.394+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 91.892µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.403+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[12762948908485355776:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:19.403+05:30 [Info] Indexer::handleMergePartition Source 3288617707771216315 Target 12762948908485355776 2023-10-21T04:16:19.403+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12762948908485355776 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009e4d8c0 2023-10-21T04:16:19.403+05:30 [Info] MergePartition: sent async request to update index instance 12762948908485355776 rstate moved to ACTIVE 2023-10-21T04:16:19.408+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.408+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.411+05:30 [Info] switched currmeta from 74 -> 75 force true 2023-10-21T04:16:19.424+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36246:172.31.5.112:12000] 2023-10-21T04:16:19.426+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.179754ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.427+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 3288617707771216315, realInst: 12762948908485355776 2023-10-21T04:16:19.433+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 15991242037348127852, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:19.433+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 15991242037348127852, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:19.449+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.449+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.455+05:30 [Info] switched currmeta from 75 -> 77 force true 2023-10-21T04:16:19.474+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.474+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.478+05:30 [Info] switched currmeta from 77 -> 77 force true 2023-10-21T04:16:19.482+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14579235200590321485,realInstId: 0, indexDefn DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:19.482+05:30 [Info] Indexer::handleRecoverIndex InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:19.483+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 800.064µs 2023-10-21T04:16:19.483+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 844.266µs 2023-10-21T04:16:19.483+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.484+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.484+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:16:19.484+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 106 metaSz: 2571 rp [1] sn: 720006 count: 106 metaSz: 2571 rp [2] sn: 720007 count: 106 metaSz: 2571 rp [3] sn: 1080007 count: 106 metaSz: 2572 2023-10-21T04:16:19.487+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 133100 recovery UsedSpace 140766 2023-10-21T04:16:19.489+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14579235200590321485, indexDefn DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:19.490+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 56.666975ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.491+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:19.491+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 106 metaSz: 2571 rp [1] sn: 1080007 count: 106 metaSz: 2572 2023-10-21T04:16:19.491+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 121008 recovery UsedSpace 149002 2023-10-21T04:16:19.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex 2023-10-21T04:16:19.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.498+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex 2023-10-21T04:16:19.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex started 2023-10-21T04:16:19.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.498+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex 2023-10-21T04:16:19.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.512+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex 2023-10-21T04:16:19.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex started 2023-10-21T04:16:19.513+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:19.513+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.513+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.514+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.518+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.536+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36248:172.31.5.112:12000] 2023-10-21T04:16:19.536+05:30 [Info] switched currmeta from 77 -> 78 force true 2023-10-21T04:16:19.539+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.539+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.541+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.541+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.542+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.542+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.544+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.544+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.544+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.545+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.545+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.545+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.545+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:19.545+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 6 Warmup took 60.960851ms 2023-10-21T04:16:19.546+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 6 WriterThreads 1 2023-10-21T04:16:19.547+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 6 Stopped Billing WUs 2023-10-21T04:16:19.547+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index Index: 14579235200590321485 Partition: 6 2023-10-21T04:16:19.547+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.548+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.549+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.549+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:16:19.549+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.550+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:19.551+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 132 metaSz: 2572 rp [1] sn: 1080007 count: 132 metaSz: 2573 2023-10-21T04:16:19.550+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 132 metaSz: 2574 rp [1] sn: 720006 count: 132 metaSz: 2574 rp [2] sn: 720007 count: 132 metaSz: 2572 rp [3] sn: 1080007 count: 132 metaSz: 2573 2023-10-21T04:16:19.552+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 124758 recovery UsedSpace 149002 2023-10-21T04:16:19.553+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 141050 recovery UsedSpace 141484 2023-10-21T04:16:19.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex 2023-10-21T04:16:19.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.558+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex 2023-10-21T04:16:19.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex started 2023-10-21T04:16:19.559+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.559+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex 2023-10-21T04:16:19.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.560+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex 2023-10-21T04:16:19.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex started 2023-10-21T04:16:19.560+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:19.560+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.560+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.581+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 7 Warmup took 32.529434ms 2023-10-21T04:16:19.581+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 7 WriterThreads 1 2023-10-21T04:16:19.581+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 7 Stopped Billing WUs 2023-10-21T04:16:19.581+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index Index: 14579235200590321485 Partition: 7 2023-10-21T04:16:19.581+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.582+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.582+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:19.582+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 123 metaSz: 2574 rp [1] sn: 720006 count: 123 metaSz: 2574 rp [2] sn: 720007 count: 123 metaSz: 2572 rp [3] sn: 1080007 count: 123 metaSz: 2573 2023-10-21T04:16:19.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.582+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:19.582+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 123 metaSz: 2572 rp [1] sn: 1080007 count: 123 metaSz: 2573 2023-10-21T04:16:19.582+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 135071 recovery UsedSpace 140766 2023-10-21T04:16:19.583+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 123291 recovery UsedSpace 149002 2023-10-21T04:16:19.595+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36252:172.31.5.112:12000] 2023-10-21T04:16:19.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex 2023-10-21T04:16:19.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.596+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex 2023-10-21T04:16:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex started 2023-10-21T04:16:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex 2023-10-21T04:16:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.597+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex 2023-10-21T04:16:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex started 2023-10-21T04:16:19.597+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:19.599+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.599+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.641+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 8 Warmup took 59.209551ms 2023-10-21T04:16:19.641+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 8 WriterThreads 1 2023-10-21T04:16:19.641+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 8 Stopped Billing WUs 2023-10-21T04:16:19.641+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index Index: 14579235200590321485 Partition: 8 2023-10-21T04:16:19.641+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.642+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.642+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:16:19.642+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 120 metaSz: 2574 rp [1] sn: 720006 count: 120 metaSz: 2574 rp [2] sn: 720007 count: 120 metaSz: 2572 rp [3] sn: 1080007 count: 120 metaSz: 2573 2023-10-21T04:16:19.642+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 134835 recovery UsedSpace 140766 2023-10-21T04:16:19.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.643+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:19.643+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 120 metaSz: 2572 rp [1] sn: 1080007 count: 120 metaSz: 2573 2023-10-21T04:16:19.643+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 122741 recovery UsedSpace 149002 2023-10-21T04:16:19.646+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36256:172.31.5.112:12000] 2023-10-21T04:16:19.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex 2023-10-21T04:16:19.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.651+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex 2023-10-21T04:16:19.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex started 2023-10-21T04:16:19.651+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.651+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex 2023-10-21T04:16:19.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.654+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex 2023-10-21T04:16:19.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex started 2023-10-21T04:16:19.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:19.655+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.655+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.678+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 1 Warmup took 36.097121ms 2023-10-21T04:16:19.678+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 1 WriterThreads 1 2023-10-21T04:16:19.678+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 1 Stopped Billing WUs 2023-10-21T04:16:19.678+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index Index: 14579235200590321485 Partition: 1 2023-10-21T04:16:19.678+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.678+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.679+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:19.679+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 129 metaSz: 2572 rp [1] sn: 1080007 count: 129 metaSz: 2573 2023-10-21T04:16:19.680+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 124091 recovery UsedSpace 153098 2023-10-21T04:16:19.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.681+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:16:19.681+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 129 metaSz: 2574 rp [1] sn: 720006 count: 129 metaSz: 2574 rp [2] sn: 720007 count: 129 metaSz: 2572 rp [3] sn: 1080007 count: 129 metaSz: 2573 2023-10-21T04:16:19.682+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 140564 recovery UsedSpace 141484 2023-10-21T04:16:19.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex 2023-10-21T04:16:19.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.689+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex 2023-10-21T04:16:19.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex started 2023-10-21T04:16:19.689+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.689+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex 2023-10-21T04:16:19.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.691+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex 2023-10-21T04:16:19.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex started 2023-10-21T04:16:19.691+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:19.692+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.692+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.704+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36260:172.31.5.112:12000] 2023-10-21T04:16:19.707+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36254:172.31.5.112:12000] 2023-10-21T04:16:19.710+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 2 Warmup took 31.792998ms 2023-10-21T04:16:19.711+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 2 WriterThreads 1 2023-10-21T04:16:19.711+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 2 Stopped Billing WUs 2023-10-21T04:16:19.711+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index Index: 14579235200590321485 Partition: 2 2023-10-21T04:16:19.711+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.711+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.712+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 258 2023-10-21T04:16:19.713+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 132 metaSz: 2574 rp [1] sn: 720006 count: 132 metaSz: 2574 rp [2] sn: 720007 count: 132 metaSz: 2572 rp [3] sn: 1080007 count: 132 metaSz: 2573 2023-10-21T04:16:19.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:19.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 132 metaSz: 2572 rp [1] sn: 1080007 count: 132 metaSz: 2573 2023-10-21T04:16:19.714+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 140757 recovery UsedSpace 141484 2023-10-21T04:16:19.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 124425 recovery UsedSpace 149002 2023-10-21T04:16:19.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex 2023-10-21T04:16:19.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.719+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex 2023-10-21T04:16:19.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex started 2023-10-21T04:16:19.719+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.719+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex 2023-10-21T04:16:19.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.719+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex 2023-10-21T04:16:19.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex started 2023-10-21T04:16:19.719+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:19.720+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.739+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 3 Warmup took 26.90269ms 2023-10-21T04:16:19.739+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 3 WriterThreads 1 2023-10-21T04:16:19.739+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 3 Stopped Billing WUs 2023-10-21T04:16:19.739+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index Index: 14579235200590321485 Partition: 3 2023-10-21T04:16:19.739+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.739+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.740+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 254 2023-10-21T04:16:19.740+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 130 metaSz: 2574 rp [1] sn: 720006 count: 130 metaSz: 2574 rp [2] sn: 720007 count: 130 metaSz: 2572 rp [3] sn: 1080007 count: 130 metaSz: 2573 2023-10-21T04:16:19.740+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 140242 recovery UsedSpace 141464 2023-10-21T04:16:19.741+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.741+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:19.741+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 130 metaSz: 2572 rp [1] sn: 1080007 count: 130 metaSz: 2573 2023-10-21T04:16:19.741+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 123983 recovery UsedSpace 149002 2023-10-21T04:16:19.744+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36258:172.31.5.112:12000] 2023-10-21T04:16:19.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex 2023-10-21T04:16:19.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.754+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex 2023-10-21T04:16:19.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex started 2023-10-21T04:16:19.754+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex 2023-10-21T04:16:19.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.757+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex 2023-10-21T04:16:19.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex started 2023-10-21T04:16:19.757+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:19.761+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.761+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.767+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:36264:172.31.5.112:12000] 2023-10-21T04:16:19.791+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 4 Warmup took 51.768865ms 2023-10-21T04:16:19.792+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 4 WriterThreads 1 2023-10-21T04:16:19.792+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 4 Stopped Billing WUs 2023-10-21T04:16:19.792+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index Index: 14579235200590321485 Partition: 4 2023-10-21T04:16:19.792+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14579235200590321485 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:19.792+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14579235200590321485 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:19.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:19.792+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 258 2023-10-21T04:16:19.792+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 128 metaSz: 2574 rp [1] sn: 720006 count: 128 metaSz: 2574 rp [2] sn: 720007 count: 128 metaSz: 2572 rp [3] sn: 1080007 count: 128 metaSz: 2573 2023-10-21T04:16:19.793+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 140392 recovery UsedSpace 141484 2023-10-21T04:16:19.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:19.793+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:19.793+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 128 metaSz: 2572 rp [1] sn: 1080007 count: 128 metaSz: 2573 2023-10-21T04:16:19.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 123858 recovery UsedSpace 149002 2023-10-21T04:16:19.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex 2023-10-21T04:16:19.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:19.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:19.807+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex 2023-10-21T04:16:19.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex started 2023-10-21T04:16:19.807+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.807+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex 2023-10-21T04:16:19.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:19.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:19.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex 2023-10-21T04:16:19.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex started 2023-10-21T04:16:19.808+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:19.809+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:19.809+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:19.824+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14579235200590321485 PartitionId 5 Warmup took 31.450435ms 2023-10-21T04:16:19.824+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14579235200590321485 partitionId 5 WriterThreads 1 2023-10-21T04:16:19.824+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14579235200590321485 PartitionId 5 Stopped Billing WUs 2023-10-21T04:16:19.824+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14579235200590321485 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index Index: 14579235200590321485 Partition: 5 2023-10-21T04:16:19.824+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14579235200590321485, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:19.824+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14579235200590321485, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:19.825+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:19.825+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14579235200590321485 2023-10-21T04:16:19.825+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:19.826+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:19.826+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:19.826+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:19.826+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:19.828+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14579235200590321485 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:16:19.831+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:7 Attempting to open snapshot (SnapshotInfo: count:132 committed:false) 2023-10-21T04:16:19.832+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:8 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:16:19.833+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.538092ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.833+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:1 Attempting to open snapshot (SnapshotInfo: count:120 committed:false) 2023-10-21T04:16:19.834+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:2 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:16:19.838+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 5.351756ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:19.853+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:3 Attempting to open snapshot (SnapshotInfo: count:132 committed:false) 2023-10-21T04:16:19.854+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:4 Attempting to open snapshot (SnapshotInfo: count:130 committed:false) 2023-10-21T04:16:19.856+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:5 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:16:19.858+05:30 [Info] StorageMgr::openSnapshot IndexInst:14579235200590321485 Partition:6 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:16:19.859+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.859+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.861+05:30 [Info] switched currmeta from 78 -> 79 force true 2023-10-21T04:16:19.893+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.893+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.898+05:30 [Info] switched currmeta from 79 -> 80 force true 2023-10-21T04:16:19.989+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:19.990+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:19.994+05:30 [Info] switched currmeta from 80 -> 81 force true 2023-10-21T04:16:19.995+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9566966241835503, throttleDelayMs (new, old, change): (9, 8, 1) 2023-10-21T04:16:20.280+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14343533858105260407:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:20.280+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005234360 2023-10-21T04:16:20.281+05:30 [Info] handleUpdateIndexRState: Index instance 14343533858105260407 rstate moved to ACTIVE 2023-10-21T04:16:20.286+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.371539ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:20.287+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 12338036531800130039, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:20.287+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 12338036531800130039, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:20.297+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.297+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.301+05:30 [Info] switched currmeta from 81 -> 82 force true 2023-10-21T04:16:20.314+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15667952562388529486,realInstId: 0, indexDefn DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:20.315+05:30 [Info] Indexer::handleRecoverIndex InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:20.319+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 3.629132ms 2023-10-21T04:16:20.319+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15667952562388529486, indexDefn DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:20.319+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.799179ms 2023-10-21T04:16:20.319+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 31.964343ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:20.319+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15667952562388529486 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:20.319+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15667952562388529486 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:20.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:20.320+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1054958 memSz 790 2023-10-21T04:16:20.320+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 1000 metaSz: 2556 rp [1] sn: 1080007 count: 1000 metaSz: 2557 2023-10-21T04:16:20.320+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 192030 recovery UsedSpace 151975 2023-10-21T04:16:20.324+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:16:20.324+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:20.324+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:20.324+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:16:20.324+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex started 2023-10-21T04:16:20.325+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15667952562388529486 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:20.325+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.325+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.325+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:20.325+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:20.328+05:30 [Info] switched currmeta from 82 -> 83 force true 2023-10-21T04:16:20.334+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15667952562388529486 PartitionId 0 Warmup took 14.662421ms 2023-10-21T04:16:20.334+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15667952562388529486 partitionId 0 WriterThreads 4 2023-10-21T04:16:20.334+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15667952562388529486 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index Index: 15667952562388529486 Partition: 0 2023-10-21T04:16:20.334+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15667952562388529486, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:16:20.334+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15667952562388529486, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:16:20.334+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:20.335+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15667952562388529486 2023-10-21T04:16:20.335+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:20.335+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:20.335+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:20.335+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:20.335+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:20.335+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15667952562388529486 Partitions [0] 2023-10-21T04:16:20.336+05:30 [Info] StorageMgr::openSnapshot IndexInst:15667952562388529486 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:20.343+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.52847ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:20.343+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 67.256µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:20.353+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.354+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.358+05:30 [Info] switched currmeta from 83 -> 85 force true 2023-10-21T04:16:20.375+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.376+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.482+05:30 [Info] switched currmeta from 85 -> 85 force true 2023-10-21T04:16:20.494+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.494+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.500+05:30 [Info] switched currmeta from 85 -> 86 force true 2023-10-21T04:16:20.510+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14579235200590321485:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:20.514+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 14741949230702299196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:20.514+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 14741949230702299196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:20.541+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11050396066566371125,realInstId: 0, indexDefn DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:20.541+05:30 [Info] Indexer::handleRecoverIndex InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:20.541+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 226.499µs 2023-10-21T04:16:20.541+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 270.669µs 2023-10-21T04:16:20.542+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11050396066566371125 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:20.543+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11050396066566371125, indexDefn DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:20.543+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 28.550904ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:20.543+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11050396066566371125 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:20.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:20.543+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1056174 memSz 2184 2023-10-21T04:16:20.543+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 360005 count: 1000 metaSz: 2578 rp [1] sn: 720006 count: 1000 metaSz: 2578 rp [2] sn: 720007 count: 1000 metaSz: 2576 rp [3] sn: 1080007 count: 1000 metaSz: 2577 2023-10-21T04:16:20.544+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 247828 recovery UsedSpace 149819 2023-10-21T04:16:20.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:20.548+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1056522 memSz 666 2023-10-21T04:16:20.548+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 1000 metaSz: 2576 rp [1] sn: 1080007 count: 1000 metaSz: 2577 2023-10-21T04:16:20.549+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 223833 recovery UsedSpace 151592 2023-10-21T04:16:20.553+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.553+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex 2023-10-21T04:16:20.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:20.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:20.554+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex 2023-10-21T04:16:20.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex started 2023-10-21T04:16:20.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:20.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:20.557+05:30 [Info] switched currmeta from 86 -> 87 force true 2023-10-21T04:16:20.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex 2023-10-21T04:16:20.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:20.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:20.560+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex 2023-10-21T04:16:20.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex started 2023-10-21T04:16:20.560+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11050396066566371125 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:20.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:20.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:20.593+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11050396066566371125 PartitionId 0 Warmup took 50.204287ms 2023-10-21T04:16:20.594+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11050396066566371125 partitionId 0 WriterThreads 4 2023-10-21T04:16:20.594+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11050396066566371125 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:20.594+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11050396066566371125 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index Index: 11050396066566371125 Partition: 0 2023-10-21T04:16:20.594+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11050396066566371125, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:20.594+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11050396066566371125, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:20.594+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:20.594+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11050396066566371125 2023-10-21T04:16:20.594+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:20.595+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:20.595+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:20.595+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:20.595+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:20.595+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11050396066566371125 Partitions [0] 2023-10-21T04:16:20.596+05:30 [Info] StorageMgr::openSnapshot IndexInst:11050396066566371125 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:20.612+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.987852ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:20.613+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 358.521µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:20.622+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.630+05:30 [Info] switched currmeta from 87 -> 88 force true 2023-10-21T04:16:20.934+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.934+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.938+05:30 [Info] switched currmeta from 88 -> 89 force true 2023-10-21T04:16:20.972+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:20.972+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:20.977+05:30 [Info] switched currmeta from 89 -> 90 force true 2023-10-21T04:16:20.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8182180402848888, throttleDelayMs (new, old, change): (4, 9, -5) 2023-10-21T04:16:21.010+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.010+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.013+05:30 [Info] switched currmeta from 90 -> 91 force true 2023-10-21T04:16:21.051+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.051+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.054+05:30 [Info] switched currmeta from 91 -> 92 force true 2023-10-21T04:16:21.326+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3100906924534408756:INDEX_STATE_RECOVERED 9912003609070511414:INDEX_STATE_RECOVERED 15667952562388529486:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:21.327+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 11619165464560280276, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:21.327+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 11619165464560280276, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:21.339+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14271070195512901425,realInstId: 0, indexDefn DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:21.339+05:30 [Info] Indexer::handleRecoverIndex InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:21.339+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 128.251µs 2023-10-21T04:16:21.339+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 173.147µs 2023-10-21T04:16:21.339+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14271070195512901425 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:21.339+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14271070195512901425 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:21.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:21.340+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:21.340+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:21.340+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14271070195512901425, indexDefn DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:21.341+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 13.757937ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:21.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 133 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:21.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:21.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:21.349+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:21.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex started 2023-10-21T04:16:21.349+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:21.349+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:21.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:16:21.349+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:21.349+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:16:21.349+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:16:21.349+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:21.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex stopped 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:21.349+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:21.350+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:21.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.360+05:30 [Info] switched currmeta from 92 -> 93 force true 2023-10-21T04:16:21.360+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:21.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:16:21.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex closed 2023-10-21T04:16:21.360+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:21.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:16:21.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 133 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:16:21.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:21.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index destroyed 2023-10-21T04:16:21.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:21.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:21.369+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:21.369+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:21.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 164 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:21.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:21.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:21.373+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:21.373+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:16:21.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex started 2023-10-21T04:16:21.373+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Warmup took 33.756783ms 2023-10-21T04:16:21.373+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14271070195512901425 partitionId 0 WriterThreads 4 2023-10-21T04:16:21.373+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:21.374+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14271070195512901425 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index Index: 14271070195512901425 Partition: 0 2023-10-21T04:16:21.374+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14271070195512901425, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:16:21.374+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:21.374+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14271070195512901425, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:16:21.374+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:21.374+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:21.374+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14271070195512901425 2023-10-21T04:16:21.374+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14271070195512901425 Name: #primary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:21.375+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:21.375+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:21.375+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14271070195512901425 Name: #primary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:21.375+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:21.375+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14271070195512901425 Partitions [0] 2023-10-21T04:16:21.375+05:30 [Info] StorageMgr::openSnapshot IndexInst:14271070195512901425 Partition:0 No Snapshot Found. 2023-10-21T04:16:21.375+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14271070195512901425 Adding Nil Snapshot. 2023-10-21T04:16:21.383+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.185832ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:21.384+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 67.614µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:21.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.391+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.395+05:30 [Info] switched currmeta from 93 -> 94 force true 2023-10-21T04:16:21.416+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.416+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.420+05:30 [Info] switched currmeta from 94 -> 95 force true 2023-10-21T04:16:21.454+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.454+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.457+05:30 [Info] switched currmeta from 95 -> 96 force true 2023-10-21T04:16:21.486+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.486+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.490+05:30 [Info] switched currmeta from 96 -> 97 force true 2023-10-21T04:16:21.549+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11050396066566371125:INDEX_STATE_RECOVERED 14579235200590321485:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:21.550+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 7874378322029374196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:21.550+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 7874378322029374196, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:21.565+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 13739166377675070800,realInstId: 0, indexDefn DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:21.565+05:30 [Info] Indexer::handleRecoverIndex InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:21.565+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 138.807µs 2023-10-21T04:16:21.565+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 194.666µs 2023-10-21T04:16:21.565+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13739166377675070800 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:21.565+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 13739166377675070800, indexDefn DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:21.565+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 15.626412ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:21.566+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13739166377675070800 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:21.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:21.566+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1055282 memSz 610 2023-10-21T04:16:21.566+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720007 count: 1000 metaSz: 2552 rp [1] sn: 1080007 count: 1000 metaSz: 2552 2023-10-21T04:16:21.566+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 200466 recovery UsedSpace 151208 2023-10-21T04:16:21.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex 2023-10-21T04:16:21.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:21.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:21.577+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex 2023-10-21T04:16:21.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex started 2023-10-21T04:16:21.578+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13739166377675070800 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:21.578+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:21.578+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:21.581+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.581+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.586+05:30 [Info] switched currmeta from 97 -> 98 force true 2023-10-21T04:16:21.596+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13739166377675070800 PartitionId 0 Warmup took 30.431781ms 2023-10-21T04:16:21.597+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13739166377675070800 partitionId 0 WriterThreads 4 2023-10-21T04:16:21.601+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13739166377675070800 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:21.601+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13739166377675070800 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index Index: 13739166377675070800 Partition: 0 2023-10-21T04:16:21.601+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 13739166377675070800, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:16:21.601+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 13739166377675070800, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:16:21.601+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:21.602+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13739166377675070800 2023-10-21T04:16:21.602+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:21.602+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:21.602+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:21.602+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:21.602+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:21.603+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13739166377675070800 Partitions [0] 2023-10-21T04:16:21.603+05:30 [Info] StorageMgr::openSnapshot IndexInst:13739166377675070800 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:21.616+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.097542ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:21.617+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 67.556µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:21.631+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:21.631+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:21.634+05:30 [Info] switched currmeta from 98 -> 99 force true 2023-10-21T04:16:21.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6617339537546281, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:16:22.008+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.008+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.011+05:30 [Info] switched currmeta from 99 -> 100 force true 2023-10-21T04:16:22.053+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.054+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.056+05:30 [Info] switched currmeta from 100 -> 101 force true 2023-10-21T04:16:22.345+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14271070195512901425:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:22.345+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0073f3500 2023-10-21T04:16:22.346+05:30 [Info] handleUpdateIndexRState: Index instance 14271070195512901425 rstate moved to ACTIVE 2023-10-21T04:16:22.358+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.891986ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:22.358+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [573521267164877540 14292583894719865678 12338036531800130039] belonging to collectionId: 0, ttid: ShardTokena7:37:e9:2a:6a:83:12:d0. Initiating build 2023-10-21T04:16:22.368+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.368+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.373+05:30 [Info] switched currmeta from 101 -> 102 force true 2023-10-21T04:16:22.406+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.406+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.410+05:30 [Info] switched currmeta from 102 -> 103 force true 2023-10-21T04:16:22.430+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.436+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.439+05:30 [Info] switched currmeta from 103 -> 104 force true 2023-10-21T04:16:22.441+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [9912003609070511414 3100906924534408756 15667952562388529486] &{2} 2023-10-21T04:16:22.441+05:30 [Info] Indexer::handleBuildRecoveredIndexes [9912003609070511414 3100906924534408756 15667952562388529486] 2023-10-21T04:16:22.441+05:30 [Info] bulkUpdateRState: Index instance 9912003609070511414 rstate moved to PENDING 2023-10-21T04:16:22.441+05:30 [Info] bulkUpdateRState: Index instance 3100906924534408756 rstate moved to PENDING 2023-10-21T04:16:22.441+05:30 [Info] bulkUpdateRState: Index instance 15667952562388529486 rstate moved to PENDING 2023-10-21T04:16:22.441+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [9912003609070511414 3100906924534408756 15667952562388529486] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:22.441+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:22.441+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:22.442+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:22.461+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1 2023-10-21T04:16:22.461+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:22.461+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:16:22.461+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:22.461+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:22.461+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:22.462+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:16:22.462+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:16:22.462+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:16:22.462+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1 SessionId 42562 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:22.463+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.463+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.463+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_1. OSO not supported for default scope/collection. 2023-10-21T04:16:22.464+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:16:22.464+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream INIT_STREAM 2023-10-21T04:16:22.464+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_1:1]] 2023-10-21T04:16:22.464+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3100906924534408756 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:22.465+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_1:2]] 2023-10-21T04:16:22.465+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 9912003609070511414 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:22.465+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_1:3]] 2023-10-21T04:16:22.465+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15667952562388529486 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:22.466+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1 2023-10-21T04:16:22.466+05:30 [Info] switched currmeta from 104 -> 105 force true 2023-10-21T04:16:22.466+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 42562 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:16:22.467+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:16:22.467+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:16:22.468+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:22.469+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 27.790884ms 2023-10-21T04:16:22.469+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 27.748793ms 2023-10-21T04:16:22.469+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [9912003609070511414 3100906924534408756 15667952562388529486] 2023-10-21T04:16:22.469+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 107.232411ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:22.470+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[3100906924534408756:true 9912003609070511414:true 15667952562388529486:true] 2023-10-21T04:16:22.470+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:16:22.476+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1 Check pending stream merge. 2023-10-21T04:16:22.485+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.145722ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:22.508+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.680663ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:22.514+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.213045ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:22.521+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:47702" doAuth() ... 2023-10-21T04:16:22.525+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:47702" auth successful 2023-10-21T04:16:22.525+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:47702" +1 2023-10-21T04:16:22.547+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.548+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.554+05:30 [Info] switched currmeta from 105 -> 109 force true 2023-10-21T04:16:22.564+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.565+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.573+05:30 [Info] switched currmeta from 109 -> 109 force true 2023-10-21T04:16:22.592+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11050396066566371125:INDEX_STATE_RECOVERED 13739166377675070800:INDEX_STATE_RECOVERED 14579235200590321485:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:22.593+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 1844709904168202817, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:22.593+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 1844709904168202817, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:22.639+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4611895227247053723,realInstId: 0, indexDefn DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:22.639+05:30 [Info] Indexer::handleRecoverIndex InstId: 4611895227247053723 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:22.639+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 215.463µs 2023-10-21T04:16:22.639+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 297.705µs 2023-10-21T04:16:22.639+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4611895227247053723 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:22.640+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4611895227247053723 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:22.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:22.640+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:22.641+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:22.641+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4611895227247053723, indexDefn DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:22.641+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 48.137357ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:22.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 133 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:22.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:22.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:22.654+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:22.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex started 2023-10-21T04:16:22.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4611895227247053723 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:22.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4611895227247053723 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:22.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:16:22.654+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:16:22.654+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:22.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex stopped 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:22.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:22.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:22.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:22.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:22.665+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:22.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:16:22.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex closed 2023-10-21T04:16:22.665+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:22.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex ... 2023-10-21T04:16:22.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 133 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex ... 2023-10-21T04:16:22.669+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.669+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.670+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:22.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index destroyed 2023-10-21T04:16:22.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:22.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:22.676+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:22.676+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:22.681+05:30 [Info] switched currmeta from 109 -> 111 force true 2023-10-21T04:16:22.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 164 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:22.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:22.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:22.683+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:22.683+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:16:22.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex started 2023-10-21T04:16:22.684+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4611895227247053723 PartitionId 0 Warmup took 44.626606ms 2023-10-21T04:16:22.685+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4611895227247053723 partitionId 0 WriterThreads 4 2023-10-21T04:16:22.685+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4611895227247053723 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index Index: 4611895227247053723 Partition: 0 2023-10-21T04:16:22.685+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4611895227247053723, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:16:22.685+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4611895227247053723, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:16:22.685+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4611895227247053723 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:22.686+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4611895227247053723 2023-10-21T04:16:22.685+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:22.685+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:22.688+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4611895227247053723 Name: #primary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:22.688+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:22.689+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:22.689+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4611895227247053723 Name: #primary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:22.689+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4611895227247053723 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:22.689+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4611895227247053723 Partitions [0] 2023-10-21T04:16:22.690+05:30 [Info] StorageMgr::openSnapshot IndexInst:4611895227247053723 Partition:0 No Snapshot Found. 2023-10-21T04:16:22.690+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4611895227247053723 Adding Nil Snapshot. 2023-10-21T04:16:22.697+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.126311ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:22.697+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 75.966µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:22.704+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.704+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.707+05:30 [Info] switched currmeta from 111 -> 112 force true 2023-10-21T04:16:22.713+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:22.713+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:22.716+05:30 [Info] switched currmeta from 112 -> 112 force true 2023-10-21T04:16:22.990+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 0 186517626268899 55 42562 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:22.991+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 1 178840478062605 57 42562 172.31.5.112:9000. HWT [54-57,57]. 2023-10-21T04:16:22.994+05:30 [Info] NewTsVbuuidCached: bucket_1, 64 TsVbuuidPoolMap is updated from: map[] to: map[64:0xc00aaeefc0] 2023-10-21T04:16:22.994+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 2 148075163102199 54 42562 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:22.996+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 3 215374859564337 35 42562 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:16:23.000+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 6 242327631330054 62 42562 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:23.001+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 4 28605411869539 58 42562 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:23.001+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 5 39552632604610 52 42562 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:23.007+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 7 191903564840858 63 42562 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:23.009+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 8 50961953886801 45 42562 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:23.011+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 9 225280996447309 55 42562 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:16:23.035+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 10 274410200699189 54 42562 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:23.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 11 66881023717063 51 42562 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:23.042+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 13 183148377680315 46 42562 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:23.048+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 15 127185710837163 50 42562 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:23.048+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 14 226814108379280 72 42562 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:16:23.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 17 55290066231400 51 42562 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:23.051+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 18 242578104468179 63 42562 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:23.054+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 19 210306088158507 48 42562 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:16:23.058+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.058+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.059+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 21 209771427122473 48 42562 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:16:23.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 23 216462437971699 57 42562 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:23.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 12 4429827396609 42 42562 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:16:23.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 22 260036488774725 51 42562 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:23.064+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 16 257031483891250 58 42562 172.31.5.112:9000. HWT [55-58,58]. 2023-10-21T04:16:23.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 25 280874278672777 71 42562 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:16:23.071+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 26 219132834127088 52 42562 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:23.073+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 27 184056281482149 49 42562 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:23.077+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 29 45957426777107 57 42562 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:23.081+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 30 99433130332778 58 42562 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:23.086+05:30 [Info] switched currmeta from 112 -> 116 force true 2023-10-21T04:16:23.087+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 31 217381278987664 67 42562 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:23.087+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 20 230239359684242 50 42562 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:23.103+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.103+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.111+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 36 253845164956592 58 42562 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:23.111+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 24 160961818008256 59 42562 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:23.111+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 28 102578907139247 58 42562 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:23.112+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 33 136958029391999 65 42562 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:16:23.112+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 34 28417669987133 58 42562 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:23.112+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 35 70213628416850 47 42562 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:16:23.113+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 32 82469662608686 60 42562 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:23.117+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 47 271074273345339 53 42562 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:23.118+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 38 188107826286720 60 42562 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:23.118+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 39 222555662236724 67 42562 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:23.118+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 41 259381390638966 58 42562 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:23.118+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 42 252931432434966 46 42562 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:23.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 44 158545118983293 67 42562 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:23.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 48 227378777358533 59 42562 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:16:23.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 37 253494845579900 54 42562 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:23.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 40 5915864305121 60 42562 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:23.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 43 146267938201125 55 42562 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:23.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 45 12301831552869 66 42562 172.31.5.112:9000. HWT [63-66,66]. 2023-10-21T04:16:23.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 46 242994859770444 54 42562 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:23.122+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 50 207367350302991 64 42562 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:16:23.122+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 51 200436537417141 54 42562 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:23.122+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 53 204867593700407 56 42562 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:23.122+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 49 215467668600630 34 42562 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:16:23.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 54 279362078165491 55 42562 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:23.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 52 188718367400378 56 42562 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:23.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 55 170056388266420 70 42562 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:16:23.132+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 57 129411666769864 55 42562 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:23.132+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 56 264835102027079 53 42562 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:16:23.133+05:30 [Info] switched currmeta from 116 -> 116 force true 2023-10-21T04:16:23.143+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 58 16044440367455 53 42562 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:23.144+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 59 125009072873642 53 42562 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:23.146+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 60 75566139929862 56 42562 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:23.148+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 62 10527297481171 57 42562 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:23.149+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 61 106932309485550 53 42562 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:23.154+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1 InstanceIds [3100906924534408756 9912003609070511414 15667952562388529486] 2023-10-21T04:16:23.154+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1 Cid 0 SessionId 42562 2023-10-21T04:16:23.154+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 42562 2023-10-21T04:16:23.154+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:16:23.154+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 3100906924534408756 INDEX_STATE_CATCHUP 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 9912003609070511414 INDEX_STATE_CATCHUP 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 15667952562388529486 INDEX_STATE_CATCHUP 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 3100906924534408756 true 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 9912003609070511414 true 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 15667952562388529486 true 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 3100906924534408756 Stream: INIT_STREAM KeyspaceId: bucket_1 Session: 42562 BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1 2023-10-21T04:16:23.155+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1 2023-10-21T04:16:23.155+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 42562 STREAM_ACTIVE 2023-10-21T04:16:23.155+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1 Stream: INIT_STREAM SessionId: 42562 2023-10-21T04:16:23.155+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1 42562 2023-10-21T04:16:23.156+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:23.156+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:23.156+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:23.157+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:23.157+05:30 [Info] send progress stats to clients 2023-10-21T04:16:23.157+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 42562 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:23.157+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3100906924534408756 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:16:23.157+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 9912003609070511414 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:16:23.157+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15667952562388529486 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:16:23.158+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:16:23.158+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:23.158+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:23.158+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 42562 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:23.158+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:16:23.158+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0] 2023-10-21T04:16:23.158+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0] 2023-10-21T04:16:23.158+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:16:23.158+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:47702: use of closed network connection 2023-10-21T04:16:23.158+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:47702" exit: read tcp4 127.0.0.1:9121->127.0.0.1:47702: use of closed network connection 2023-10-21T04:16:23.158+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:47702" closed ! 2023-10-21T04:16:23.158+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:16:23.159+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:16:23.159+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:16:23.161+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1 2023-10-21T04:16:23.161+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1] 2023-10-21T04:16:23.181+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.245549ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.187+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1]. Err genServer.closed 2023-10-21T04:16:23.187+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:16:23.187+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Treating genServer.closed As Success 2023-10-21T04:16:23.187+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1 SessionId 42562 2023-10-21T04:16:23.187+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1 2023-10-21T04:16:23.187+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:16:23.187+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:23.187+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:16:23.187+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:16:23.187+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:23.187+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:23.187+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:16:23.187+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:16:23.187+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:16:23.188+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 28469 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:23.188+05:30 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2023-10-21T04:16:23.188+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:16:23.188+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:16:23.188+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:16:23.188+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:16:23.188+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:16:23.188+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 28469 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:16:23.189+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:16:23.189+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:16:23.199+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:16:23.204+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.407185ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.211+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.211+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.218+05:30 [Info] switched currmeta from 116 -> 118 force true 2023-10-21T04:16:23.219+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.883143ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.233+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.233+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.246+05:30 [Info] switched currmeta from 118 -> 120 force true 2023-10-21T04:16:23.274+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.274+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.286+05:30 [Info] switched currmeta from 120 -> 121 force true 2023-10-21T04:16:23.314+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.327+05:30 [Info] switched currmeta from 121 -> 122 force true 2023-10-21T04:16:23.333+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.334+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.337+05:30 [Info] switched currmeta from 122 -> 122 force true 2023-10-21T04:16:23.345+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:43508" doAuth() ... 2023-10-21T04:16:23.346+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:43508" auth successful 2023-10-21T04:16:23.348+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:43508" +1 2023-10-21T04:16:23.498+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00957b260 2023-10-21T04:16:23.499+05:30 [Info] handleUpdateIndexRState: Index instance 3100906924534408756 rstate moved to ACTIVE 2023-10-21T04:16:23.499+05:30 [Info] Indexer::handleMergePartition Source 8538178595515381903 Target 9912003609070511414 2023-10-21T04:16:23.499+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00ba2ae40 2023-10-21T04:16:23.500+05:30 [Info] MergePartition: sent async request to update index instance 9912003609070511414 rstate moved to ACTIVE 2023-10-21T04:16:23.510+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.714153ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.512+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00949bb60 2023-10-21T04:16:23.512+05:30 [Info] handleUpdateIndexRState: Index instance 15667952562388529486 rstate moved to ACTIVE 2023-10-21T04:16:23.531+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.870519ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.531+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 8538178595515381903, realInst: 9912003609070511414 2023-10-21T04:16:23.540+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.870046ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.540+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 7494249569853685720. Retrying... 2023-10-21T04:16:23.540+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 11790381330881939912. Retrying... 2023-10-21T04:16:23.541+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 7582778950442222621. Retrying... 2023-10-21T04:16:23.541+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 5004576769623010783. Retrying... 2023-10-21T04:16:23.541+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 12265503634969032942. Retrying... 2023-10-21T04:16:23.541+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[3100906924534408756:true 9912003609070511414:true 15667952562388529486:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:23.545+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 12833670934079048728, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:23.545+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 12833670934079048728, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:23.551+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.551+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.559+05:30 [Info] switched currmeta from 122 -> 125 force true 2023-10-21T04:16:23.565+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.565+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.568+05:30 [Info] switched currmeta from 125 -> 126 force true 2023-10-21T04:16:23.574+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.574+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.579+05:30 [Info] switched currmeta from 126 -> 126 force true 2023-10-21T04:16:23.582+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:23.609+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.609+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.614+05:30 [Info] switched currmeta from 126 -> 127 force true 2023-10-21T04:16:23.621+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.630+05:30 [Info] switched currmeta from 127 -> 127 force true 2023-10-21T04:16:23.637+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 5004576769623010783,realInstId: 0, indexDefn DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:23.642+05:30 [Info] Indexer::handleRecoverIndex InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:23.643+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 206.264µs 2023-10-21T04:16:23.643+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 254.676µs 2023-10-21T04:16:23.643+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 5004576769623010783, indexDefn DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:23.643+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 97.931402ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.643+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5004576769623010783 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:23.643+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5004576769623010783 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:23.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:23.646+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 64 2023-10-21T04:16:23.646+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 21 metaSz: 2548 rp [1] sn: 1080007 count: 21 metaSz: 2548 2023-10-21T04:16:23.649+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 98776 recovery UsedSpace 149292 2023-10-21T04:16:23.649+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4611895227247053723:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:23.649+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4611895227247053723 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009b76480 2023-10-21T04:16:23.649+05:30 [Info] handleUpdateIndexRState: Index instance 4611895227247053723 rstate moved to ACTIVE 2023-10-21T04:16:23.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:16:23.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:23.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:23.659+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:16:23.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex started 2023-10-21T04:16:23.660+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5004576769623010783 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:23.660+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:23.660+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:23.662+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.694068ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.662+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [15991242037348127852 14741949230702299196 7874378322029374196] belonging to collectionId: d, ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c. Initiating build 2023-10-21T04:16:23.666+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.670+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.697+05:30 [Info] switched currmeta from 127 -> 129 force true 2023-10-21T04:16:23.725+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5004576769623010783 PartitionId 0 Warmup took 81.794262ms 2023-10-21T04:16:23.726+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5004576769623010783 partitionId 0 WriterThreads 4 2023-10-21T04:16:23.726+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5004576769623010783 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index Index: 5004576769623010783 Partition: 0 2023-10-21T04:16:23.726+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 5004576769623010783, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:16:23.726+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 5004576769623010783, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:16:23.727+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:23.728+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 5004576769623010783 2023-10-21T04:16:23.728+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:23.728+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:23.728+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:23.731+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:23.731+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:23.732+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 5004576769623010783 Partitions [0] 2023-10-21T04:16:23.734+05:30 [Info] StorageMgr::openSnapshot IndexInst:5004576769623010783 Partition:0 Attempting to open snapshot (SnapshotInfo: count:21 committed:false) 2023-10-21T04:16:23.770+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [14579235200590321485 11050396066566371125 13739166377675070800] &{2} 2023-10-21T04:16:23.770+05:30 [Info] Indexer::handleBuildRecoveredIndexes [14579235200590321485 11050396066566371125 13739166377675070800] 2023-10-21T04:16:23.771+05:30 [Info] bulkUpdateRState: Index instance 14579235200590321485 rstate moved to PENDING 2023-10-21T04:16:23.771+05:30 [Info] bulkUpdateRState: Index instance 11050396066566371125 rstate moved to PENDING 2023-10-21T04:16:23.771+05:30 [Info] bulkUpdateRState: Index instance 13739166377675070800 rstate moved to PENDING 2023-10-21T04:16:23.771+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [14579235200590321485 11050396066566371125 13739166377675070800] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:23.771+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:23.772+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:23.772+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:23.786+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.787+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.807+05:30 [Info] switched currmeta from 129 -> 134 force true 2023-10-21T04:16:23.835+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.836+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.855+05:30 [Info] switched currmeta from 134 -> 134 force true 2023-10-21T04:16:23.855+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15667952562388529486:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:23.855+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 15667952562388529486:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:23.855+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3100906924534408756:[0] 9912003609070511414:[5 6 7 8 1 2 3 4] 15667952562388529486:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:23.886+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.886+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.889+05:30 [Info] switched currmeta from 134 -> 135 force true 2023-10-21T04:16:23.917+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.917+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.921+05:30 [Info] switched currmeta from 135 -> 136 force true 2023-10-21T04:16:23.944+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:16:23.945+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:23.945+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:23.945+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:23.945+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:16:23.945+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:16:23.945+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:23.945+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:23.946+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:16:23.946+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:23.946+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:23.947+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c2% reset bucket state to NIL 2023-10-21T04:16:23.951+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:16:23.955+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:23.955+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:23.958+05:30 [Info] switched currmeta from 136 -> 137 force true 2023-10-21T04:16:23.951+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% SessionId 88713 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:23.965+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:23.965+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:23.966+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c2% Added for Stream INIT_STREAM 2023-10-21T04:16:23.966+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[bucket_%2:_default:c2%:1]] 2023-10-21T04:16:23.966+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 14579235200590321485 stream INIT_STREAM keyspaceId bucket_%2:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:23.966+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[bucket_%2:_default:c2%:2]] 2023-10-21T04:16:23.966+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 11050396066566371125 stream INIT_STREAM keyspaceId bucket_%2:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:23.966+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[bucket_%2:_default:c2%:3]] 2023-10-21T04:16:23.966+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 13739166377675070800 stream INIT_STREAM keyspaceId bucket_%2:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:23.966+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:16:23.966+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 88713 CollectionId: d CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:16:23.967+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:16:23.967+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:16:23.969+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:23.970+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 199.20411ms 2023-10-21T04:16:23.970+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 199.266844ms 2023-10-21T04:16:23.970+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [14579235200590321485 11050396066566371125 13739166377675070800] 2023-10-21T04:16:23.970+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 305.230041ms len(expediates) 4 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.971+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[11050396066566371125:true 13739166377675070800:true 14579235200590321485:true] 2023-10-21T04:16:23.978+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c2% Check pending stream merge. 2023-10-21T04:16:23.980+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.044283ms len(expediates) 3 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:23.993+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.039688ms len(expediates) 2 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:24.011+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.716149ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:24.020+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.77164ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:24.020+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 81.945µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:24.020+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.020+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:24.024+05:30 [Info] switched currmeta from 137 -> 141 force true 2023-10-21T04:16:24.033+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:24.054+05:30 [Info] switched currmeta from 141 -> 141 force true 2023-10-21T04:16:24.159+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:16:24.221+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 186517626268899 55 28469 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:24.224+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 178840478062605 57 28469 172.31.5.112:9000. HWT [54-57,57]. 2023-10-21T04:16:24.224+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 148075163102199 54 28469 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:24.227+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 215374859564337 35 28469 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:16:24.230+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 28605411869539 58 28469 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:24.230+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 39552632604610 52 28469 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:24.231+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 242327631330054 62 28469 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:24.233+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 191903564840858 63 28469 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:24.236+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 225280996447309 55 28469 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:16:24.236+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 50961953886801 45 28469 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:24.238+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 274410200699189 54 28469 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:24.238+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 66881023717063 51 28469 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:24.241+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 183148377680315 46 28469 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:24.241+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 4429827396609 42 28469 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:16:24.245+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 127185710837163 50 28469 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:24.245+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 226814108379280 72 28469 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:16:24.249+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 55290066231400 51 28469 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:24.249+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 257031483891250 58 28469 172.31.5.112:9000. HWT [55-58,58]. 2023-10-21T04:16:24.250+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 242578104468179 63 28469 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:24.252+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 210306088158507 48 28469 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:16:24.255+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 230239359684242 50 28469 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:24.256+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 260036488774725 51 28469 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:24.256+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 209771427122473 48 28469 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:16:24.259+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 216462437971699 57 28469 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:24.261+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 160961818008256 59 28469 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:24.262+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 280874278672777 71 28469 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:16:24.264+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 184056281482149 49 28469 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:24.264+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 219132834127088 52 28469 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:24.267+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 45957426777107 57 28469 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:24.267+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 102578907139247 58 28469 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:24.269+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 99433130332778 58 28469 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:24.270+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 217381278987664 67 28469 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:24.272+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 136958029391999 65 28469 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:16:24.273+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 82469662608686 60 28469 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:24.275+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 28417669987133 58 28469 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:24.280+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 253845164956592 58 28469 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:24.280+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 70213628416850 47 28469 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:16:24.280+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 253494845579900 54 28469 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:24.282+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 188107826286720 60 28469 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:24.283+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 222555662236724 67 28469 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:24.288+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 252931432434966 46 28469 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:24.288+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 5915864305121 60 28469 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:24.288+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 146267938201125 55 28469 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:24.296+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 242994859770444 54 28469 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:24.296+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 259381390638966 58 28469 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:24.298+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 271074273345339 53 28469 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:24.303+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 158545118983293 67 28469 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:24.303+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 227378777358533 59 28469 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:16:24.303+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 12301831552869 66 28469 172.31.5.112:9000. HWT [63-66,66]. 2023-10-21T04:16:24.304+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 215467668600630 34 28469 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:16:24.307+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 207367350302991 64 28469 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:16:24.309+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 200436537417141 54 28469 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:24.315+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 279362078165491 55 28469 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:24.315+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 188718367400378 56 28469 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:24.318+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 204867593700407 56 28469 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:24.320+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 170056388266420 70 28469 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:16:24.336+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 264835102027079 53 28469 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:16:24.336+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 16044440367455 53 28469 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:24.339+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 129411666769864 55 28469 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:24.339+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 125009072873642 53 28469 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:24.340+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 75566139929862 56 28469 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:24.349+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 106932309485550 53 28469 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:24.349+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 10527297481171 57 28469 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:24.349+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 133748675726909 56 28469 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:24.350+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1 InstanceIds [9912003609070511414 3100906924534408756 15667952562388529486] 2023-10-21T04:16:24.350+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 28469 2023-10-21T04:16:24.350+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:16:24.350+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:16:24.350+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:16:24.350+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:16:24.350+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:16:24.350+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 28469 2023-10-21T04:16:24.350+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 STREAM_ACTIVE 2023-10-21T04:16:24.350+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:16:24.350+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:16:24.445+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.445+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:24.450+05:30 [Info] switched currmeta from 141 -> 143 force true 2023-10-21T04:16:24.459+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.459+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:24.465+05:30 [Info] switched currmeta from 143 -> 143 force true 2023-10-21T04:16:24.501+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.502+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:24.509+05:30 [Info] switched currmeta from 143 -> 144 force true 2023-10-21T04:16:24.673+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[5004576769623010783:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:24.673+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [12833670934079048728] belonging to collectionId: 9, ttid: ShardTokena7:37:e9:2a:6a:83:12:d0. Initiating build 2023-10-21T04:16:24.692+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [5004576769623010783] &{2} 2023-10-21T04:16:24.692+05:30 [Info] Indexer::handleBuildRecoveredIndexes [5004576769623010783] 2023-10-21T04:16:24.693+05:30 [Info] bulkUpdateRState: Index instance 5004576769623010783 rstate moved to PENDING 2023-10-21T04:16:24.693+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [5004576769623010783] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:24.693+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:24.693+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:24.693+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:24.731+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.731+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:24.734+05:30 [Info] switched currmeta from 144 -> 145 force true 2023-10-21T04:16:24.756+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:16:24.756+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:24.756+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:24.756+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:24.756+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:24.756+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:24.756+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:16:24.756+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:16:24.756+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:24.756+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:24.757+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c2%:{0xc009f69e00 }] 2023-10-21T04:16:24.757+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:24.757+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:24.757+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:24.757+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:24.757+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:16:24.757+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:16:24.757+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 2793 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:24.757+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:24.757+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:24.757+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:24.757+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:24.757+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:16:24.757+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[bucket_%2:_default:c2%:3 bucket_1:_system:_query:1]] 2023-10-21T04:16:24.757+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 5004576769623010783 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:24.757+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:16:24.757+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 2793 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:16:24.758+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c2%:{0xc009f69e00 } bucket_1:_system:_query:{0xc00c6fa3c0 }] 2023-10-21T04:16:24.759+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:24.759+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 66.774319ms 2023-10-21T04:16:24.759+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 66.814849ms 2023-10-21T04:16:24.759+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [5004576769623010783] 2023-10-21T04:16:24.759+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 85.82422ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:24.761+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[5004576769623010783:true] 2023-10-21T04:16:24.766+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.713359ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:24.770+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:16:24.783+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.784+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:24.787+05:30 [Info] switched currmeta from 145 -> 146 force true 2023-10-21T04:16:24.992+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9560656924598472, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:16:24.995+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:24.995+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:25.002+05:30 [Info] switched currmeta from 146 -> 148 force true 2023-10-21T04:16:25.023+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:25.023+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:25.032+05:30 [Info] switched currmeta from 148 -> 149 force true 2023-10-21T04:16:25.037+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:25.037+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:25.041+05:30 [Info] switched currmeta from 149 -> 149 force true 2023-10-21T04:16:25.351+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:16:25.351+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:16:25.762+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:16:25.776+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48014" doAuth() ... 2023-10-21T04:16:25.776+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48014" auth successful 2023-10-21T04:16:25.776+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:48014" +1 2023-10-21T04:16:25.937+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:25.937+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:25.943+05:30 [Info] switched currmeta from 149 -> 152 force true 2023-10-21T04:16:25.955+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:25.955+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:25.958+05:30 [Info] switched currmeta from 152 -> 152 force true 2023-10-21T04:16:25.971+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:25.971+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:25.975+05:30 [Info] switched currmeta from 152 -> 153 force true 2023-10-21T04:16:25.977+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c2% Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:16:25.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.889041131492128, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:16:26.002+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5852031406248543250:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:26.002+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5852031406248543250:[0] 8852459821271515836:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:26.002+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5852031406248543250:[0] 8852459821271515836:[0] 12510230312641977282:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:26.278+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:26.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:26.294+05:30 [Info] switched currmeta from 153 -> 154 force true 2023-10-21T04:16:26.305+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:26.305+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:26.314+05:30 [Info] switched currmeta from 154 -> 156 force true 2023-10-21T04:16:26.325+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:26.325+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:26.330+05:30 [Info] switched currmeta from 156 -> 157 force true 2023-10-21T04:16:26.335+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 0 186517626268899 55 2793 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.337+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 1 178840478062605 57 2793 172.31.5.112:9000. HWT [54-57,57]. 2023-10-21T04:16:26.339+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 4 28605411869539 58 2793 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.340+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 3 215374859564337 35 2793 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:16:26.342+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 5 39552632604610 52 2793 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:26.346+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 2 148075163102199 54 2793 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 7 191903564840858 63 2793 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:26.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 6 242327631330054 62 2793 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:26.348+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 8 50961953886801 45 2793 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:26.353+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 9 225280996447309 55 2793 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:16:26.355+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 11 66881023717063 51 2793 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:26.356+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 12 4429827396609 42 2793 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:16:26.359+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 13 183148377680315 46 2793 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:26.359+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:26.359+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:26.363+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 15 127185710837163 50 2793 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:26.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 16 257031483891250 58 2793 172.31.5.112:9000. HWT [55-58,58]. 2023-10-21T04:16:26.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 17 55290066231400 51 2793 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:26.369+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 19 210306088158507 48 2793 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:16:26.372+05:30 [Info] switched currmeta from 157 -> 157 force true 2023-10-21T04:16:26.373+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 20 230239359684242 50 2793 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:26.374+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 21 209771427122473 48 2793 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:16:26.381+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 25 280874278672777 71 2793 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:16:26.381+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 23 216462437971699 57 2793 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:26.381+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 24 160961818008256 59 2793 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:26.388+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 28 102578907139247 58 2793 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.388+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 27 184056281482149 49 2793 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:26.393+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 29 45957426777107 57 2793 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:26.394+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 31 217381278987664 67 2793 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:26.403+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 32 82469662608686 60 2793 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:26.404+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 33 136958029391999 65 2793 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:16:26.404+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 35 70213628416850 47 2793 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:16:26.404+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 36 253845164956592 58 2793 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.409+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 39 222555662236724 67 2793 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:26.410+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 37 253494845579900 54 2793 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.410+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 40 5915864305121 60 2793 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:26.420+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 22 260036488774725 51 2793 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:26.420+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 38 188107826286720 60 2793 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:26.420+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 44 158545118983293 67 2793 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:26.420+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 18 242578104468179 63 2793 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:26.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 26 219132834127088 52 2793 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:26.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 34 28417669987133 58 2793 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 30 99433130332778 58 2793 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 42 252931432434966 46 2793 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:26.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 10 274410200699189 54 2793 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 46 242994859770444 54 2793 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.422+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 14 226814108379280 72 2793 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:16:26.422+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 41 259381390638966 58 2793 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.436+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 48 227378777358533 59 2793 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:16:26.438+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 43 146267938201125 55 2793 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.441+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:26.441+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 49 215467668600630 34 2793 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:16:26.441+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:26.441+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 45 12301831552869 66 2793 172.31.5.112:9000. HWT [63-66,66]. 2023-10-21T04:16:26.441+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 47 271074273345339 53 2793 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:26.443+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 50 207367350302991 64 2793 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:16:26.443+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 52 188718367400378 56 2793 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:26.449+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 54 279362078165491 55 2793 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.453+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 51 200436537417141 54 2793 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.453+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 56 264835102027079 53 2793 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:16:26.456+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 53 204867593700407 56 2793 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:26.457+05:30 [Info] switched currmeta from 157 -> 159 force true 2023-10-21T04:16:26.460+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 58 16044440367455 53 2793 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:26.460+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 60 75566139929862 56 2793 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:26.461+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 55 170056388266420 70 2793 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:16:26.463+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 57 129411666769864 55 2793 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.463+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 62 10527297481171 57 2793 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:26.464+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 59 125009072873642 53 2793 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:26.470+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:26.470+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:26.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 61 106932309485550 53 2793 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:26.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 63 133748675726909 56 2793 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:26.475+05:30 [Info] switched currmeta from 159 -> 159 force true 2023-10-21T04:16:26.483+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_system:_query InstanceIds [5004576769623010783] 2023-10-21T04:16:26.483+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 2793 2023-10-21T04:16:26.484+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 2793 2023-10-21T04:16:26.484+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:16:26.484+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:16:26.484+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 5004576769623010783 INDEX_STATE_CATCHUP 2023-10-21T04:16:26.484+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 5004576769623010783 true 2023-10-21T04:16:26.485+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 5004576769623010783 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query Session: 2793 BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:16:26.485+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_system:_query, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:16:26.486+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:16:26.486+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:16:26.486+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 2793 STREAM_ACTIVE 2023-10-21T04:16:26.486+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:16:26.486+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=4,1=57,2=4,3=4,4=4,5=29,6=4,7=4,8=4,9=55,10=4,11=4,12=4,13=4,14=72,15=4,16=58,17=4,18=63,19=4,20=4,21=48,22=4,23=20,24=37,25=4,26=4,27=4,28=4,29=57,30=4,31=4,32=4,33=25,34=4,35=4,36=4,37=4,38=4,39=4,40=60,41=4,42=4,43=35,44=4,45=66,46=24,47=4,48=59,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=53,57=4,58=53,59=4,60=4,61=21,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:26.487+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 2793 2023-10-21T04:16:26.487+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_system:_query 2793 2023-10-21T04:16:26.487+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:26.491+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:26.495+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:26.498+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:26.498+05:30 [Info] send progress stats to clients 2023-10-21T04:16:26.499+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:16:26.499+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:16:26.500+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:4] INIT_STREAM:map[bucket_%2:_default:c2%:3 bucket_1:_system:_query:1]] 2023-10-21T04:16:26.500+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 2793 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:16:26.520+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5642987086862451891:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:26.520+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[864958057973929476:[0] 5642987086862451891:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:26.520+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[864958057973929476:[0] 5642987086862451891:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:26.520+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[864958057973929476:[0] 5642987086862451891:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:26.520+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[864958057973929476:[0] 5642987086862451891:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:26.520+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[864958057973929476:[0] 5642987086862451891:[2 3 4 5 6 7 8 1] 8060737836868569583:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:26.527+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:16:26.538+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 39.719197ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:26.575+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 2023-10-21T04:16:26.577+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 2793 2023-10-21T04:16:26.577+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 2793 2023-10-21T04:16:26.577+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:16:26.577+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:16:26.577+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 5004576769623010783 false 2023-10-21T04:16:26.577+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_system:_query 5004576769623010783 2023-10-21T04:16:26.577+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_system:_query. 2023-10-21T04:16:26.577+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 2793 2023-10-21T04:16:26.589+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:26.590+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:26.592+05:30 [Info] switched currmeta from 159 -> 160 force true 2023-10-21T04:16:26.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:26.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:26.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:26.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:26.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:26.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:26.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:16:26.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 1 176639880977228 61 88713 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:26.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 0 263268055970459 52 88713 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:26.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 2 35403626585217 50 88713 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:26.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 5 3874519919278 54 88713 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 4 159113185399719 58 88713 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 3 215749515705641 45 88713 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:26.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 6 261493429196201 61 88713 172.31.5.112:9000. HWT [60-61,61]. 2023-10-21T04:16:26.831+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 7 170288557775498 52 88713 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:26.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 10 206533515052028 50 88713 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:26.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 11 234758907181738 56 88713 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:26.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 9 236398693961229 62 88713 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:26.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 14 140236896721046 60 88713 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:26.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 8 264975169139761 56 88713 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:26.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 13 102350060311343 53 88713 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:26.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 12 116104661063478 55 88713 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 15 162158720181435 67 88713 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:26.853+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 18 207134287529256 46 88713 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:26.853+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 17 98820809009123 44 88713 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:16:26.853+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 16 200752780087280 54 88713 172.31.5.112:9000. HWT [49-54,54]. 2023-10-21T04:16:26.855+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 19 208836052671833 45 88713 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:26.857+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 22 192215088742243 55 88713 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.857+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 21 27979933591532 54 88713 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.858+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 23 154934968850942 51 88713 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:26.862+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 25 168740244208559 61 88713 172.31.5.112:9000. HWT [59-61,61]. 2023-10-21T04:16:26.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 26 120989102157727 59 88713 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:26.866+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 27 5715487545729 54 88713 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.871+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 29 130231725986856 49 88713 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:26.871+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 28 236953356346174 54 88713 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.871+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 20 92956357968546 50 88713 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:26.872+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 24 271667630317209 52 88713 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:26.872+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 30 60545336618717 46 88713 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:26.873+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 31 73432771436458 60 88713 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:26.875+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 32 67362620867705 67 88713 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:26.876+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 33 278426329668882 54 88713 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 35 24368176313355 50 88713 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:26.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 34 51776324008509 60 88713 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:26.882+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 37 83647976044922 59 88713 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:26.882+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 36 43594649874772 52 88713 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:26.884+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 38 181267443431019 53 88713 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:26.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 39 100659545510465 49 88713 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:16:26.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 40 153478150002150 68 88713 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:16:26.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 41 116785907755771 55 88713 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 42 45479338375128 55 88713 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.896+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 44 166675095997741 57 88713 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:26.896+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 43 249919763471169 66 88713 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:26.899+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 46 124194574848641 54 88713 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.899+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 45 49796766757346 61 88713 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:26.901+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 47 149022976193937 55 88713 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.903+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 48 19274425379955 55 88713 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 49 184487117398647 55 88713 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:26.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 50 199728167534433 66 88713 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:26.908+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 51 196777743741606 51 88713 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:26.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 52 158076890855634 54 88713 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:26.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 53 138935694816848 61 88713 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:26.915+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 55 61798873988101 67 88713 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:26.915+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 54 24856128127173 60 88713 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:26.917+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 56 29071415952852 61 88713 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:26.919+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 57 191719959781498 63 88713 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:26.929+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 58 103336286678362 46 88713 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:26.929+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 59 140942236111856 63 88713 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:26.937+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 60 56626109228904 46 88713 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:26.937+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 61 119871837460216 39 88713 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:16:26.939+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 62 177105814270799 58 88713 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:26.940+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 63 252537041863196 73 88713 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:16:26.943+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c2% InstanceIds [14579235200590321485 11050396066566371125 13739166377675070800 5004576769623010783] 2023-10-21T04:16:26.943+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% Cid d SessionId 88713 2023-10-21T04:16:26.943+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 88713 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 14579235200590321485 INDEX_STATE_CATCHUP 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 11050396066566371125 INDEX_STATE_CATCHUP 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 13739166377675070800 INDEX_STATE_CATCHUP 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 14579235200590321485 true 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 11050396066566371125 true 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 13739166377675070800 true 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 14579235200590321485 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% Session: 88713 BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c2%, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:16:26.943+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:16:26.944+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 88713 STREAM_ACTIVE 2023-10-21T04:16:26.946+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c2% Stream: INIT_STREAM SessionId: 88713 2023-10-21T04:16:26.946+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2:_default:c2% 88713 2023-10-21T04:16:26.950+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:26.950+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:26.950+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:26.950+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:26.951+05:30 [Info] send progress stats to clients 2023-10-21T04:16:26.951+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 88713 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:26.951+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 13739166377675070800 from stream INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:16:26.951+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 14579235200590321485 from stream INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:16:26.951+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 11050396066566371125 from stream INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:16:26.951+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:16:26.951+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:16:26.951+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 0 New 0 2023-10-21T04:16:26.951+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 88713 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:26.952+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_system:_query:{0xc00c6fa3c0 }] 2023-10-21T04:16:26.952+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.952+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:26.952+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.952+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:26.953+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.953+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:26.954+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.954+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:26.956+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2:_default:c2% 2023-10-21T04:16:26.958+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c2%] 2023-10-21T04:16:26.959+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.797367ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:26.961+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 1 176639880977228 61 2023-10-21T04:16:26.961+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:26.961+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 2 35403626585217 50 2023-10-21T04:16:26.961+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:26.961+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 0 263268055970459 52 2023-10-21T04:16:26.961+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:26.968+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.623408ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:26.971+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c2%] 2023-10-21T04:16:26.971+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 88713 2023-10-21T04:16:26.971+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:16:26.971+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:16:26.972+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.972+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:26.972+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.972+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:26.972+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:26.972+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:16:26.972+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:16:26.972+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:26.972+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:26.972+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc009f69440 }] 2023-10-21T04:16:26.972+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.972+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:26.972+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.972+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:26.972+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:16:26.972+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:16:26.972+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 32478 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:26.972+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.972+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:26.972+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:26.972+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:26.972+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:16:26.973+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1 bucket_1:4] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:16:26.973+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:4] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:16:26.973+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:16:26.973+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:16:26.974+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.380775ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:26.973+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: true SessionId: 32478 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:16:27.001+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc00e1af380 } bucket_1:{0xc009f69440 }] 2023-10-21T04:16:27.001+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 27 5715487545729 54 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 5 3874519919278 54 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 6 261493429196201 61 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 9 236398693961229 62 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 11 234758907181738 56 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 4 159113185399719 58 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 7 170288557775498 52 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 8 264975169139761 56 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 10 206533515052028 50 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.002+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 3 215749515705641 45 2023-10-21T04:16:27.008+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:16:27.002+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 16 200752780087280 54 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 17 98820809009123 44 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 18 207134287529256 46 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 19 208836052671833 45 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 20 92956357968546 50 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 12 116104661063478 55 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 13 102350060311343 53 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 14 140236896721046 60 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 15 162158720181435 67 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.013+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 29 130231725986856 49 2023-10-21T04:16:27.013+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 25 168740244208559 61 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 26 120989102157727 59 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 28 236953356346174 54 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 21 27979933591532 54 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 23 154934968850942 51 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 24 271667630317209 52 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 22 192215088742243 55 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 31 73432771436458 60 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 34 51776324008509 60 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 32 67362620867705 67 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 33 278426329668882 54 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 35 24368176313355 50 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 30 60545336618717 46 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 42 45479338375128 55 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 57 191719959781498 63 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 44 166675095997741 57 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 39 100659545510465 49 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 40 153478150002150 68 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 46 124194574848641 54 2023-10-21T04:16:27.014+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.014+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 51 196777743741606 51 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 52 158076890855634 54 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 53 138935694816848 61 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 54 24856128127173 60 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 36 43594649874772 52 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 41 116785907755771 55 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 45 49796766757346 61 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 49 184487117398647 55 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 37 83647976044922 59 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 48 19274425379955 55 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 43 249919763471169 66 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 55 61798873988101 67 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 47 149022976193937 55 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 50 199728167534433 66 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 56 29071415952852 61 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 38 181267443431019 53 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 59 140942236111856 63 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 60 56626109228904 46 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 61 119871837460216 39 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.015+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 62 177105814270799 58 2023-10-21T04:16:27.015+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.016+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 63 252537041863196 73 2023-10-21T04:16:27.016+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.016+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c2% 58 103336286678362 46 2023-10-21T04:16:27.016+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c2% Stream INIT_STREAM. Ignored. 2023-10-21T04:16:27.021+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0001163c0 2023-10-21T04:16:27.021+05:30 [Info] handleUpdateIndexRState: Index instance 11050396066566371125 rstate moved to ACTIVE 2023-10-21T04:16:27.021+05:30 [Info] Indexer::handleMergePartition Source 6291351234483181222 Target 14579235200590321485 2023-10-21T04:16:27.021+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc004de80c0 2023-10-21T04:16:27.021+05:30 [Info] MergePartition: sent async request to update index instance 14579235200590321485 rstate moved to ACTIVE 2023-10-21T04:16:27.024+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.024+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.027+05:30 [Info] switched currmeta from 160 -> 163 force true 2023-10-21T04:16:27.031+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.115869ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.031+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006938ba0 2023-10-21T04:16:27.042+05:30 [Info] handleUpdateIndexRState: Index instance 13739166377675070800 rstate moved to ACTIVE 2023-10-21T04:16:27.048+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.47737ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.050+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 6291351234483181222, realInst: 14579235200590321485 2023-10-21T04:16:27.051+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.051+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.056+05:30 [Info] switched currmeta from 163 -> 165 force true 2023-10-21T04:16:27.056+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.973931ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.056+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 2296526085582506469. Retrying... 2023-10-21T04:16:27.056+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 1265729768899070816. Retrying... 2023-10-21T04:16:27.056+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 4992762551574016313. Retrying... 2023-10-21T04:16:27.056+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 18415077564616412601. Retrying... 2023-10-21T04:16:27.056+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 16733302726553098964. Retrying... 2023-10-21T04:16:27.056+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[11050396066566371125:true 13739166377675070800:true 14579235200590321485:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:27.060+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12027054384338714804, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:27.061+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12027054384338714804, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:27.069+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.069+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.077+05:30 [Info] switched currmeta from 165 -> 166 force true 2023-10-21T04:16:27.089+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.089+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.092+05:30 [Info] switched currmeta from 166 -> 166 force true 2023-10-21T04:16:27.096+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 9720186692804226822,realInstId: 0, indexDefn DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:27.096+05:30 [Info] Indexer::handleRecoverIndex InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:27.096+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 324.4µs 2023-10-21T04:16:27.097+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 389.282µs 2023-10-21T04:16:27.097+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9720186692804226822 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:27.097+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9720186692804226822 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:27.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:27.097+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1056230 memSz 2202 2023-10-21T04:16:27.097+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 1000 metaSz: 2576 rp [1] sn: 720006 count: 1000 metaSz: 2576 rp [2] sn: 720007 count: 1000 metaSz: 2575 rp [3] sn: 1080007 count: 1000 metaSz: 2576 2023-10-21T04:16:27.098+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 251735 recovery UsedSpace 186942 2023-10-21T04:16:27.101+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 9720186692804226822, indexDefn DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:27.101+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 41.289988ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:27.106+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1056524 memSz 670 2023-10-21T04:16:27.106+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 1000 metaSz: 2575 rp [1] sn: 1080007 count: 1000 metaSz: 2576 2023-10-21T04:16:27.107+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 228425 recovery UsedSpace 168184 2023-10-21T04:16:27.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.120+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex 2023-10-21T04:16:27.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:27.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:27.130+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex 2023-10-21T04:16:27.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex started 2023-10-21T04:16:27.130+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:27.131+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:27.157+05:30 [Info] switched currmeta from 166 -> 168 force true 2023-10-21T04:16:27.183+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.183+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.187+05:30 [Info] switched currmeta from 168 -> 168 force true 2023-10-21T04:16:27.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 1 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex 2023-10-21T04:16:27.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:27.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:27.193+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex 2023-10-21T04:16:27.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex started 2023-10-21T04:16:27.193+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:27.193+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:27.193+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9720186692804226822 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:27.230+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9720186692804226822 PartitionId 0 Warmup took 133.308841ms 2023-10-21T04:16:27.231+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9720186692804226822 partitionId 0 WriterThreads 4 2023-10-21T04:16:27.231+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9720186692804226822 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:27.231+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9720186692804226822 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index Index: 9720186692804226822 Partition: 0 2023-10-21T04:16:27.231+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 9720186692804226822, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:27.231+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 9720186692804226822, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:27.231+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:27.234+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 9720186692804226822 2023-10-21T04:16:27.234+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:27.234+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:27.234+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:27.234+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:27.234+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:27.234+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 9720186692804226822 Partitions [0] 2023-10-21T04:16:27.235+05:30 [Info] StorageMgr::openSnapshot IndexInst:9720186692804226822 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:27.240+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.581491ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.240+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 68.587µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:27.246+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.246+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.249+05:30 [Info] switched currmeta from 168 -> 169 force true 2023-10-21T04:16:27.487+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:16:27.487+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_system:_query 2023-10-21T04:16:27.487+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 5004576769623010783 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId: 2793, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:27.487+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 5004576769623010783 INDEX_STATE_ACTIVE 2023-10-21T04:16:27.487+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_system:_query State Changed to INACTIVE 2023-10-21T04:16:27.487+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:16:27.487+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:16:27.487+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:27.487+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 2793 2023-10-21T04:16:27.488+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:27.488+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:27.488+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:27.488+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 2793 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:27.488+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 5004576769623010783 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:16:27.488+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:16:27.488+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:16:27.488+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:16:27.488+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 2793 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:27.488+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:16:27.489+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:27.489+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:27.489+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:16:27.489+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:48014: use of closed network connection 2023-10-21T04:16:27.489+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:48014" exit: read tcp4 127.0.0.1:9121->127.0.0.1:48014: use of closed network connection 2023-10-21T04:16:27.489+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48014" closed ! 2023-10-21T04:16:27.489+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:16:27.489+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:48014" already gone 2023-10-21T04:16:27.489+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:27.489+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:27.489+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:27.489+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:27.489+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:16:27.489+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:27.490+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId 2793 2023-10-21T04:16:27.491+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:16:27.503+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.503+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.513+05:30 [Info] switched currmeta from 169 -> 170 force true 2023-10-21T04:16:27.519+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_system:_query]. Err genServer.closed 2023-10-21T04:16:27.519+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:16:27.519+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Treating genServer.closed As Success 2023-10-21T04:16:27.519+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 2793 2023-10-21T04:16:27.519+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 2793 2023-10-21T04:16:27.519+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:16:27.526+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 36.744109ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.558+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.558+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.564+05:30 [Info] switched currmeta from 170 -> 172 force true 2023-10-21T04:16:27.594+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.594+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.612+05:30 [Info] switched currmeta from 172 -> 172 force true 2023-10-21T04:16:27.779+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0089ba300 2023-10-21T04:16:27.779+05:30 [Info] handleUpdateIndexRState: Index instance 5004576769623010783 rstate moved to ACTIVE 2023-10-21T04:16:27.783+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 263268055970459 52 32478 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:27.786+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 35403626585217 50 32478 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:27.788+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.783819ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.788+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 12265503634969032942. Retrying... 2023-10-21T04:16:27.788+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[5004576769623010783:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:27.790+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16013578430151540990, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:27.790+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16013578430151540990, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:27.799+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 176639880977228 61 32478 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:27.799+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 159113185399719 58 32478 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:27.799+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 3874519919278 54 32478 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:27.799+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 261493429196201 61 32478 172.31.5.112:9000. HWT [60-61,61]. 2023-10-21T04:16:27.800+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 215749515705641 45 32478 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:27.803+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 264975169139761 56 32478 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:27.806+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 236398693961229 62 32478 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:27.806+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 206533515052028 50 32478 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:27.809+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.809+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.814+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 116104661063478 55 32478 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:27.814+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 234758907181738 56 32478 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:27.814+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 170288557775498 52 32478 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:27.815+05:30 [Info] switched currmeta from 172 -> 173 force true 2023-10-21T04:16:27.815+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14579235200590321485:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:27.816+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11050396066566371125:[0] 14579235200590321485:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:27.816+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11050396066566371125:[0] 14579235200590321485:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:27.816+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11050396066566371125:[0] 13739166377675070800:[0] 14579235200590321485:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:27.816+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11050396066566371125:[0] 13739166377675070800:[0] 14579235200590321485:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:27.816+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11050396066566371125:[0] 13739166377675070800:[0] 14579235200590321485:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:27.823+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 102350060311343 53 32478 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:27.823+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 140236896721046 60 32478 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:27.823+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 162158720181435 67 32478 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:27.824+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 200752780087280 54 32478 172.31.5.112:9000. HWT [49-54,54]. 2023-10-21T04:16:27.824+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 98820809009123 44 32478 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:16:27.827+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 207134287529256 46 32478 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:27.831+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 208836052671833 45 32478 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:27.832+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 92956357968546 50 32478 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:27.831+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4440544141961093904,realInstId: 0, indexDefn DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:27.834+05:30 [Info] Indexer::handleRecoverIndex InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:27.835+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 1.075188ms 2023-10-21T04:16:27.835+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4440544141961093904, indexDefn DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:27.836+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 46.11437ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:27.835+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 1.514779ms 2023-10-21T04:16:27.836+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:27.838+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 27979933591532 54 32478 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:27.842+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.841+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:27.851+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 192215088742243 55 32478 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:27.851+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 154934968850942 51 32478 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:27.842+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:27.852+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 258 2023-10-21T04:16:27.852+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 139 metaSz: 2576 rp [1] sn: 360005 count: 139 metaSz: 2577 rp [2] sn: 720006 count: 139 metaSz: 2577 rp [3] sn: 1080007 count: 139 metaSz: 2577 2023-10-21T04:16:27.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:27.852+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:27.852+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 139 metaSz: 2577 rp [1] sn: 1080007 count: 139 metaSz: 2577 2023-10-21T04:16:27.852+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 271667630317209 52 32478 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:27.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 132246 recovery UsedSpace 141484 2023-10-21T04:16:27.854+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 116000 recovery UsedSpace 149002 2023-10-21T04:16:27.858+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 5715487545729 54 32478 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:27.858+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 120989102157727 59 32478 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:27.860+05:30 [Info] switched currmeta from 173 -> 174 force true 2023-10-21T04:16:27.864+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 60545336618717 46 32478 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:27.866+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 73432771436458 60 32478 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:27.875+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 236953356346174 54 32478 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:27.875+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 67362620867705 67 32478 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:27.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex 2023-10-21T04:16:27.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:27.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:27.877+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex 2023-10-21T04:16:27.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex started 2023-10-21T04:16:27.878+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:27.879+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:27.881+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 83647976044922 59 32478 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:27.881+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 51776324008509 60 32478 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:27.881+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 24368176313355 50 32478 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:27.881+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 43594649874772 52 32478 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:27.882+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 278426329668882 54 32478 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:27.882+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 168740244208559 61 32478 172.31.5.112:9000. HWT [59-61,61]. 2023-10-21T04:16:27.882+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 130231725986856 49 32478 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:27.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex 2023-10-21T04:16:27.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:27.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:27.884+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex 2023-10-21T04:16:27.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex started 2023-10-21T04:16:27.885+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:27.886+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:27.885+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:27.892+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 181267443431019 53 32478 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:27.892+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 100659545510465 49 32478 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:16:27.892+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 153478150002150 68 32478 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:16:27.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:27.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:27.899+05:30 [Info] switched currmeta from 174 -> 175 force true 2023-10-21T04:16:27.901+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 249919763471169 66 32478 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:27.901+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 45479338375128 55 32478 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:27.902+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 116785907755771 55 32478 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:27.915+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 124194574848641 54 32478 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:27.919+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 166675095997741 57 32478 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:27.919+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 149022976193937 55 32478 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:27.919+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 19274425379955 55 32478 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:27.919+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 158076890855634 54 32478 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:27.919+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 196777743741606 51 32478 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:27.920+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 199728167534433 66 32478 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:27.923+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 184487117398647 55 32478 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:27.923+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 49796766757346 61 32478 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:27.926+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 24856128127173 60 32478 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:27.928+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 61798873988101 67 32478 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:27.934+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 3 Warmup took 83.713157ms 2023-10-21T04:16:27.935+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 3 WriterThreads 1 2023-10-21T04:16:27.935+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index Index: 4440544141961093904 Partition: 3 2023-10-21T04:16:27.936+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:27.938+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:27.939+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 56626109228904 46 32478 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:27.939+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 29071415952852 61 32478 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:27.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:27.939+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:27.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:27.939+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 112 metaSz: 2576 rp [1] sn: 1080007 count: 112 metaSz: 2576 2023-10-21T04:16:27.939+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:27.939+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 112 metaSz: 2576 rp [1] sn: 360005 count: 112 metaSz: 2576 rp [2] sn: 720006 count: 112 metaSz: 2576 rp [3] sn: 1080007 count: 112 metaSz: 2576 2023-10-21T04:16:27.939+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 112215 recovery UsedSpace 149002 2023-10-21T04:16:27.941+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 124713 recovery UsedSpace 140766 2023-10-21T04:16:27.942+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 140942236111856 63 32478 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:27.943+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 177105814270799 58 32478 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:27.944+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 138935694816848 61 32478 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:27.945+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:16:27.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex 2023-10-21T04:16:27.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:27.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:27.948+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex 2023-10-21T04:16:27.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex started 2023-10-21T04:16:27.948+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:27.948+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:27.946+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 103336286678362 46 32478 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:27.949+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2 InstanceIds [15667952562388529486 9912003609070511414 3100906924534408756 5004576769623010783 13739166377675070800 14579235200590321485 11050396066566371125] 2023-10-21T04:16:27.949+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 32478 2023-10-21T04:16:27.949+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 252537041863196 73 32478 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:16:27.950+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 191719959781498 63 32478 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:27.951+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:16:27.952+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:16:27.952+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:16:27.952+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:16:27.952+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:16:27.953+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:16:27.953+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 32478 2023-10-21T04:16:27.953+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 STREAM_ACTIVE 2023-10-21T04:16:27.953+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 119871837460216 39 32478 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:16:27.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex 2023-10-21T04:16:27.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:27.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:27.967+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex 2023-10-21T04:16:27.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex started 2023-10-21T04:16:27.969+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:27.969+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:27.969+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.008+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 4 Warmup took 69.955668ms 2023-10-21T04:16:28.010+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 4 WriterThreads 1 2023-10-21T04:16:28.010+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index Index: 4440544141961093904 Partition: 4 2023-10-21T04:16:28.011+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:28.011+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:28.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:28.012+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:28.012+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 133 metaSz: 2577 rp [1] sn: 1080007 count: 133 metaSz: 2577 2023-10-21T04:16:28.013+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 115046 recovery UsedSpace 149002 2023-10-21T04:16:28.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:28.015+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:16:28.015+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 133 metaSz: 2576 rp [1] sn: 360005 count: 133 metaSz: 2577 rp [2] sn: 720006 count: 133 metaSz: 2577 rp [3] sn: 1080007 count: 133 metaSz: 2577 2023-10-21T04:16:28.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex 2023-10-21T04:16:28.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:28.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:28.019+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex 2023-10-21T04:16:28.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex started 2023-10-21T04:16:28.019+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.019+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.021+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 131433 recovery UsedSpace 141484 2023-10-21T04:16:28.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex 2023-10-21T04:16:28.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:28.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:28.032+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex 2023-10-21T04:16:28.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex started 2023-10-21T04:16:28.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.034+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.035+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:28.056+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 5 Warmup took 44.538908ms 2023-10-21T04:16:28.056+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 5 WriterThreads 1 2023-10-21T04:16:28.056+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index Index: 4440544141961093904 Partition: 5 2023-10-21T04:16:28.056+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:28.057+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:28.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:28.057+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:16:28.057+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 121 metaSz: 2576 rp [1] sn: 360005 count: 121 metaSz: 2577 rp [2] sn: 720006 count: 121 metaSz: 2577 rp [3] sn: 1080007 count: 121 metaSz: 2577 2023-10-21T04:16:28.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:28.057+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:28.057+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 121 metaSz: 2577 rp [1] sn: 1080007 count: 121 metaSz: 2577 2023-10-21T04:16:28.058+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 125813 recovery UsedSpace 140766 2023-10-21T04:16:28.059+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 113638 recovery UsedSpace 149002 2023-10-21T04:16:28.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex 2023-10-21T04:16:28.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:28.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:28.063+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex 2023-10-21T04:16:28.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex started 2023-10-21T04:16:28.063+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.063+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex 2023-10-21T04:16:28.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:28.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:28.067+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex 2023-10-21T04:16:28.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex started 2023-10-21T04:16:28.068+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.068+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.068+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:28.106+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[9720186692804226822:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:28.107+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 18046371353728341033, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:28.107+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 18046371353728341033, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:28.129+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 6 Warmup took 72.04355ms 2023-10-21T04:16:28.129+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 6 WriterThreads 1 2023-10-21T04:16:28.129+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index Index: 4440544141961093904 Partition: 6 2023-10-21T04:16:28.129+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:28.129+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:28.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:28.130+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:28.130+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 119 metaSz: 2576 rp [1] sn: 360005 count: 119 metaSz: 2576 rp [2] sn: 720006 count: 119 metaSz: 2576 rp [3] sn: 1080007 count: 119 metaSz: 2576 2023-10-21T04:16:28.130+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 125606 recovery UsedSpace 140766 2023-10-21T04:16:28.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:28.131+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:28.131+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 119 metaSz: 2576 rp [1] sn: 1080007 count: 119 metaSz: 2576 2023-10-21T04:16:28.131+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 113123 recovery UsedSpace 149002 2023-10-21T04:16:28.136+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 13624500265195250810,realInstId: 0, indexDefn DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:28.136+05:30 [Info] Indexer::handleRecoverIndex InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:28.137+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 237.977µs 2023-10-21T04:16:28.137+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 281.557µs 2023-10-21T04:16:28.137+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13624500265195250810 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:28.138+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13624500265195250810 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:28.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex 2023-10-21T04:16:28.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:28.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:28.139+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex 2023-10-21T04:16:28.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex started 2023-10-21T04:16:28.139+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.139+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.140+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 13624500265195250810, indexDefn DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:28.140+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 33.193409ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:28.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:28.142+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:28.143+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:28.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:28.143+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:28.144+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:28.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex 2023-10-21T04:16:28.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:28.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:28.145+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex 2023-10-21T04:16:28.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex started 2023-10-21T04:16:28.145+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:28.148+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.148+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 128 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:28.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:28.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:28.166+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:28.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex started 2023-10-21T04:16:28.166+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.166+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 142 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:28.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:28.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:28.175+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:28.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex started 2023-10-21T04:16:28.175+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13624500265195250810 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:28.175+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13624500265195250810 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:28.175+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:28.175+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.175+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.175+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:28.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:16:28.175+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:28.175+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:16:28.175+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:16:28.175+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:28.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex stopped 2023-10-21T04:16:28.176+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:28.176+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:28.176+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:28.176+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:28.176+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 7 Warmup took 46.385131ms 2023-10-21T04:16:28.176+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 7 WriterThreads 1 2023-10-21T04:16:28.176+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index Index: 4440544141961093904 Partition: 7 2023-10-21T04:16:28.176+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:28.176+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:28.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:28.177+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:28.177+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 122 metaSz: 2576 rp [1] sn: 360005 count: 122 metaSz: 2577 rp [2] sn: 720006 count: 122 metaSz: 2577 rp [3] sn: 1080007 count: 122 metaSz: 2577 2023-10-21T04:16:28.177+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 125973 recovery UsedSpace 140766 2023-10-21T04:16:28.177+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:28.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:28.178+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:28.178+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 122 metaSz: 2577 rp [1] sn: 1080007 count: 122 metaSz: 2577 2023-10-21T04:16:28.178+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 113746 recovery UsedSpace 149002 2023-10-21T04:16:28.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex 2023-10-21T04:16:28.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:28.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:28.186+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex 2023-10-21T04:16:28.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex started 2023-10-21T04:16:28.186+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.186+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:28.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:16:28.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex closed 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:28.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:16:28.188+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:16:28.188+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:28.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex stopped 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:28.188+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:28.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex 2023-10-21T04:16:28.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:28.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:28.189+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex 2023-10-21T04:16:28.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex started 2023-10-21T04:16:28.189+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:28.190+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.190+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.225+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:28.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:16:28.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex closed 2023-10-21T04:16:28.236+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:28.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex ... 2023-10-21T04:16:28.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 128 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex ... 2023-10-21T04:16:28.242+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.242+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.244+05:30 [Info] switched currmeta from 175 -> 178 force true 2023-10-21T04:16:28.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:28.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex sucessfully destroyed 2023-10-21T04:16:28.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex ... 2023-10-21T04:16:28.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 142 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex ... 2023-10-21T04:16:28.248+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 8 Warmup took 71.120634ms 2023-10-21T04:16:28.248+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 8 WriterThreads 1 2023-10-21T04:16:28.248+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index Index: 4440544141961093904 Partition: 8 2023-10-21T04:16:28.248+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:28.250+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:28.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:28.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index destroyed 2023-10-21T04:16:28.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:28.252+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:16:28.252+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 134 metaSz: 2576 rp [1] sn: 360005 count: 134 metaSz: 2577 rp [2] sn: 720006 count: 134 metaSz: 2577 rp [3] sn: 1080007 count: 134 metaSz: 2577 2023-10-21T04:16:28.252+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 131760 recovery UsedSpace 141484 2023-10-21T04:16:28.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:28.253+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:28.253+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 134 metaSz: 2577 rp [1] sn: 1080007 count: 134 metaSz: 2577 2023-10-21T04:16:28.253+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 115438 recovery UsedSpace 153098 2023-10-21T04:16:28.260+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.260+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.262+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:16:28.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:28.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:28.263+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:28.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:28.267+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:28.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:28.267+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:28.267+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:28.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex 2023-10-21T04:16:28.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex 2023-10-21T04:16:28.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:28.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:28.268+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex 2023-10-21T04:16:28.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex started 2023-10-21T04:16:28.268+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.269+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:28.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:28.269+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex 2023-10-21T04:16:28.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex started 2023-10-21T04:16:28.271+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.271+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:28.271+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 148 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:28.274+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:28.274+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex started 2023-10-21T04:16:28.274+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.274+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 165 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:28.274+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:28.274+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:16:28.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex started 2023-10-21T04:16:28.274+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13624500265195250810 PartitionId 0 Warmup took 136.702968ms 2023-10-21T04:16:28.275+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13624500265195250810 partitionId 0 WriterThreads 4 2023-10-21T04:16:28.275+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13624500265195250810 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index Index: 13624500265195250810 Partition: 0 2023-10-21T04:16:28.275+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 13624500265195250810, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:28.275+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 13624500265195250810, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:28.275+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:28.275+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13624500265195250810 2023-10-21T04:16:28.276+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:28.276+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:28.276+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:28.276+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:28.276+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:28.277+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13624500265195250810 Partitions [0] 2023-10-21T04:16:28.277+05:30 [Info] StorageMgr::openSnapshot IndexInst:13624500265195250810 Partition:0 No Snapshot Found. 2023-10-21T04:16:28.277+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13624500265195250810 Adding Nil Snapshot. 2023-10-21T04:16:28.277+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.277+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.285+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.692411ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:28.285+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 80.824µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:28.303+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 1 Warmup took 52.560402ms 2023-10-21T04:16:28.304+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 1 WriterThreads 1 2023-10-21T04:16:28.304+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index Index: 4440544141961093904 Partition: 1 2023-10-21T04:16:28.304+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4440544141961093904 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:28.305+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4440544141961093904 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:28.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:28.305+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:16:28.305+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 5 count: 120 metaSz: 2576 rp [1] sn: 360005 count: 120 metaSz: 2577 rp [2] sn: 720006 count: 120 metaSz: 2577 rp [3] sn: 1080007 count: 120 metaSz: 2577 2023-10-21T04:16:28.306+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.306+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:28.306+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:28.306+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 120 metaSz: 2577 rp [1] sn: 1080007 count: 120 metaSz: 2577 2023-10-21T04:16:28.308+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 125946 recovery UsedSpace 140766 2023-10-21T04:16:28.310+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 113338 recovery UsedSpace 149002 2023-10-21T04:16:28.310+05:30 [Info] switched currmeta from 178 -> 179 force true 2023-10-21T04:16:28.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex 2023-10-21T04:16:28.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:28.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:28.326+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex 2023-10-21T04:16:28.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex started 2023-10-21T04:16:28.327+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.327+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex 2023-10-21T04:16:28.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:28.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:28.335+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex 2023-10-21T04:16:28.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex started 2023-10-21T04:16:28.336+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:28.336+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.336+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.346+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.346+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.349+05:30 [Info] switched currmeta from 179 -> 180 force true 2023-10-21T04:16:28.353+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4440544141961093904 PartitionId 2 Warmup took 48.420089ms 2023-10-21T04:16:28.353+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4440544141961093904 partitionId 2 WriterThreads 1 2023-10-21T04:16:28.353+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4440544141961093904 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index Index: 4440544141961093904 Partition: 2 2023-10-21T04:16:28.353+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4440544141961093904, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:28.354+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4440544141961093904, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:28.354+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:28.354+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4440544141961093904 2023-10-21T04:16:28.354+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:28.354+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:28.354+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:28.355+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:28.355+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:28.355+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4440544141961093904 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:16:28.356+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:5 Attempting to open snapshot (SnapshotInfo: count:133 committed:false) 2023-10-21T04:16:28.357+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:6 Attempting to open snapshot (SnapshotInfo: count:121 committed:false) 2023-10-21T04:16:28.358+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:7 Attempting to open snapshot (SnapshotInfo: count:119 committed:false) 2023-10-21T04:16:28.358+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:8 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:16:28.359+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:1 Attempting to open snapshot (SnapshotInfo: count:134 committed:false) 2023-10-21T04:16:28.360+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:2 Attempting to open snapshot (SnapshotInfo: count:120 committed:false) 2023-10-21T04:16:28.361+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:3 Attempting to open snapshot (SnapshotInfo: count:139 committed:false) 2023-10-21T04:16:28.362+05:30 [Info] StorageMgr::openSnapshot IndexInst:4440544141961093904 Partition:4 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:16:28.366+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.415999ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:28.367+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 123.726µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:28.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.391+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.394+05:30 [Info] switched currmeta from 180 -> 181 force true 2023-10-21T04:16:28.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.462+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.470+05:30 [Info] switched currmeta from 181 -> 182 force true 2023-10-21T04:16:28.488+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:16:28.622+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:28.634+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.635+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.637+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:16:28.794+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:28.794+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:28.794+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:28.794+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:28.849+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4440544141961093904:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:28.850+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 2048006213723315482, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:28.850+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 2048006213723315482, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:28.864+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14213410630307008054,realInstId: 0, indexDefn DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:28.870+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.870+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.866+05:30 [Info] Indexer::handleRecoverIndex InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:28.874+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 8.61759ms 2023-10-21T04:16:28.874+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 8.665149ms 2023-10-21T04:16:28.875+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14213410630307008054 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:28.875+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14213410630307008054 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:28.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:28.876+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1056050 memSz 2104 2023-10-21T04:16:28.876+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 1000 metaSz: 2514 rp [1] sn: 2 count: 1000 metaSz: 2580 2023-10-21T04:16:28.877+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 186085 recovery UsedSpace 24936 2023-10-21T04:16:28.877+05:30 [Info] switched currmeta from 182 -> 183 force true 2023-10-21T04:16:28.877+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14213410630307008054, indexDefn DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:28.878+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 27.930522ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:28.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:28.878+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1106 2023-10-21T04:16:28.878+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 1000 metaSz: 2514 rp [1] sn: 2 count: 1000 metaSz: 2580 2023-10-21T04:16:28.879+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 196492 recovery UsedSpace 23005 2023-10-21T04:16:28.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 138 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex 2023-10-21T04:16:28.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:28.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:28.893+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex 2023-10-21T04:16:28.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex started 2023-10-21T04:16:28.893+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.894+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 154 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex 2023-10-21T04:16:28.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:28.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:28.894+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex 2023-10-21T04:16:28.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex started 2023-10-21T04:16:28.894+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14213410630307008054 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:28.894+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:28.894+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:28.910+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14213410630307008054 PartitionId 0 Warmup took 34.833132ms 2023-10-21T04:16:28.910+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14213410630307008054 partitionId 0 WriterThreads 4 2023-10-21T04:16:28.910+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14213410630307008054 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index Index: 14213410630307008054 Partition: 0 2023-10-21T04:16:28.910+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14213410630307008054, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:28.910+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14213410630307008054, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:28.911+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:28.911+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14213410630307008054 2023-10-21T04:16:28.912+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:28.912+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:28.912+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:28.912+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:28.912+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:28.913+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14213410630307008054 Partitions [0] 2023-10-21T04:16:28.914+05:30 [Info] StorageMgr::openSnapshot IndexInst:14213410630307008054 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:28.918+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.82758ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:28.918+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 77.195µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:28.931+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:28.933+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:28.936+05:30 [Info] switched currmeta from 183 -> 184 force true 2023-10-21T04:16:28.954+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:16:28.954+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:16:29.144+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[13624500265195250810:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:29.144+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00690cba0 2023-10-21T04:16:29.145+05:30 [Info] handleUpdateIndexRState: Index instance 13624500265195250810 rstate moved to ACTIVE 2023-10-21T04:16:29.152+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.344431ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:29.156+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 15431181238515168303, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:29.156+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 15431181238515168303, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:29.173+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.173+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.178+05:30 [Info] switched currmeta from 184 -> 185 force true 2023-10-21T04:16:29.179+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 13987057765505463258,realInstId: 0, indexDefn DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:29.179+05:30 [Info] Indexer::handleRecoverIndex InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:29.180+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 429.946µs 2023-10-21T04:16:29.180+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 634.127µs 2023-10-21T04:16:29.180+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.181+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 13987057765505463258, indexDefn DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:29.181+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 25.404817ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:29.181+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.183+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.185+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.192+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 124 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:29.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.196+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:29.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex started 2023-10-21T04:16:29.196+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.196+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.201+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.201+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.204+05:30 [Info] switched currmeta from 185 -> 186 force true 2023-10-21T04:16:29.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 138 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:29.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.205+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:29.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex started 2023-10-21T04:16:29.206+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:29.206+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:16:29.206+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:16:29.206+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:29.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex stopped 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.206+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.212+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:16:29.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex closed 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:16:29.213+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:16:29.213+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:29.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex stopped 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.213+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.225+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:16:29.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex closed 2023-10-21T04:16:29.225+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:16:29.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 124 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:16:29.250+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.250+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:16:29.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 138 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:16:29.260+05:30 [Info] switched currmeta from 186 -> 187 force true 2023-10-21T04:16:29.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index destroyed 2023-10-21T04:16:29.268+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.268+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.270+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.271+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.276+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.276+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.278+05:30 [Info] switched currmeta from 187 -> 188 force true 2023-10-21T04:16:29.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 149 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:29.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.284+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:29.284+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:16:29.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex started 2023-10-21T04:16:29.284+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.284+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 166 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:29.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.289+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:29.289+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:16:29.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex started 2023-10-21T04:16:29.289+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Warmup took 107.517819ms 2023-10-21T04:16:29.289+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 3 WriterThreads 1 2023-10-21T04:16:29.289+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Stopped Billing WUs 2023-10-21T04:16:29.289+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.289+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.289+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index Index: 13987057765505463258 Partition: 3 2023-10-21T04:16:29.297+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.297+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.298+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.298+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.300+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.300+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 125 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:29.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.307+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:29.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex started 2023-10-21T04:16:29.308+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.308+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 139 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:29.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.316+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:29.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex started 2023-10-21T04:16:29.316+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:29.317+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:16:29.317+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:16:29.317+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:29.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex stopped 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.317+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.329+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:16:29.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex closed 2023-10-21T04:16:29.333+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.335+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.336+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.336+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:16:29.337+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:29.337+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:16:29.337+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:16:29.337+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:29.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex stopped 2023-10-21T04:16:29.338+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.338+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.338+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.338+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.342+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.343+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.346+05:30 [Info] switched currmeta from 188 -> 189 force true 2023-10-21T04:16:29.358+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:16:29.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex closed 2023-10-21T04:16:29.359+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:16:29.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 125 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:16:29.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:16:29.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 139 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:16:29.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index destroyed 2023-10-21T04:16:29.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.378+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.382+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.382+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 150 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:29.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.387+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:29.387+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:16:29.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex started 2023-10-21T04:16:29.387+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.388+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.399+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.400+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.403+05:30 [Info] switched currmeta from 189 -> 190 force true 2023-10-21T04:16:29.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 167 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:29.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.410+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:29.410+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:16:29.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex started 2023-10-21T04:16:29.411+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Warmup took 113.095949ms 2023-10-21T04:16:29.411+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 4 WriterThreads 1 2023-10-21T04:16:29.411+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Stopped Billing WUs 2023-10-21T04:16:29.411+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index Index: 13987057765505463258 Partition: 4 2023-10-21T04:16:29.411+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.411+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.411+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.412+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.413+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.413+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.413+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.413+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 126 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:29.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.424+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:29.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex started 2023-10-21T04:16:29.424+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.424+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 140 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:29.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.426+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:29.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex started 2023-10-21T04:16:29.426+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:29.426+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.426+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.426+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:16:29.427+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:16:29.427+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:29.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex stopped 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.433+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:16:29.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex closed 2023-10-21T04:16:29.434+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.434+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.434+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.434+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:16:29.434+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:29.435+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:16:29.435+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:16:29.435+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:29.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex stopped 2023-10-21T04:16:29.435+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.435+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.435+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.436+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.442+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:16:29.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex closed 2023-10-21T04:16:29.444+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:16:29.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 126 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:16:29.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:16:29.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 140 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:16:29.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index destroyed 2023-10-21T04:16:29.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.456+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.457+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.460+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.461+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 151 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:29.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.465+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:29.465+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:16:29.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex started 2023-10-21T04:16:29.466+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.466+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 168 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:29.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:29.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:16:29.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex started 2023-10-21T04:16:29.467+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Warmup took 55.85433ms 2023-10-21T04:16:29.467+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.467+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.467+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 5 WriterThreads 1 2023-10-21T04:16:29.467+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Stopped Billing WUs 2023-10-21T04:16:29.467+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index Index: 13987057765505463258 Partition: 5 2023-10-21T04:16:29.467+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.468+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.468+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.468+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.468+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.469+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 127 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:29.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.482+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:29.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex started 2023-10-21T04:16:29.482+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.482+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 141 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:29.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.486+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:29.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex started 2023-10-21T04:16:29.486+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:29.486+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.486+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.486+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.486+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.486+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.486+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:16:29.487+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:29.487+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:16:29.487+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:16:29.487+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:29.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex stopped 2023-10-21T04:16:29.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.487+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.500+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:16:29.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex closed 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:16:29.501+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:16:29.501+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:29.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex stopped 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.501+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:16:29.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex closed 2023-10-21T04:16:29.518+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:16:29.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 127 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:16:29.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:16:29.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 141 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:16:29.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index destroyed 2023-10-21T04:16:29.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.538+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.539+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.542+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.542+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 152 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:29.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.549+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:29.549+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:16:29.549+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex started 2023-10-21T04:16:29.549+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.549+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 169 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:29.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.550+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:29.550+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:16:29.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex started 2023-10-21T04:16:29.551+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Warmup took 82.976872ms 2023-10-21T04:16:29.551+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.551+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.551+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 6 WriterThreads 1 2023-10-21T04:16:29.551+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Stopped Billing WUs 2023-10-21T04:16:29.551+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index Index: 13987057765505463258 Partition: 6 2023-10-21T04:16:29.551+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.551+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.552+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.552+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.552+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 120 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:29.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.560+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:29.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex started 2023-10-21T04:16:29.560+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.560+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 134 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:29.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.561+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:29.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex started 2023-10-21T04:16:29.561+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:29.561+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:16:29.561+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:16:29.561+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:29.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex stopped 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.561+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.562+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:16:29.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex closed 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:16:29.568+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:16:29.568+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:29.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex stopped 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.568+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.579+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:16:29.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex closed 2023-10-21T04:16:29.580+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:16:29.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 120 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:16:29.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:16:29.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 134 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:16:29.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index destroyed 2023-10-21T04:16:29.599+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.599+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.600+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.601+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.603+05:30 [Info] switched currmeta from 190 -> 191 force true 2023-10-21T04:16:29.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.611+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.611+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 170 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.623+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:29.623+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex started 2023-10-21T04:16:29.623+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 153 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.623+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.623+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:29.623+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:16:29.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex started 2023-10-21T04:16:29.623+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.623+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Warmup took 72.086937ms 2023-10-21T04:16:29.624+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 7 WriterThreads 1 2023-10-21T04:16:29.624+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Stopped Billing WUs 2023-10-21T04:16:29.624+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index Index: 13987057765505463258 Partition: 7 2023-10-21T04:16:29.624+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.623+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.625+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.626+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.626+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.627+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.627+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 135 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.654+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex started 2023-10-21T04:16:29.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 121 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.654+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:29.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex started 2023-10-21T04:16:29.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:29.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:16:29.655+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:16:29.655+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:29.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex stopped 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.655+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:16:29.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex closed 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:16:29.678+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:16:29.678+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:29.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex stopped 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.678+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.689+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:16:29.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex closed 2023-10-21T04:16:29.690+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:16:29.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 121 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:16:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:16:29.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 135 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:16:29.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index destroyed 2023-10-21T04:16:29.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.706+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.707+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.712+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.712+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 154 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:29.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.721+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:29.721+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:16:29.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex started 2023-10-21T04:16:29.721+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 171 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:29.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.722+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:29.722+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:16:29.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex started 2023-10-21T04:16:29.722+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Warmup took 97.072648ms 2023-10-21T04:16:29.722+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 8 WriterThreads 1 2023-10-21T04:16:29.722+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Stopped Billing WUs 2023-10-21T04:16:29.722+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index Index: 13987057765505463258 Partition: 8 2023-10-21T04:16:29.722+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.722+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.723+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.723+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.722+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.723+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.723+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.724+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.724+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 122 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:29.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.732+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:29.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex started 2023-10-21T04:16:29.732+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.732+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.734+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.734+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 136 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:29.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.734+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:29.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex started 2023-10-21T04:16:29.734+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:29.734+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:16:29.734+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:16:29.734+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:29.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex stopped 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.734+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.735+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.738+05:30 [Info] switched currmeta from 191 -> 192 force true 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:16:29.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex closed 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:16:29.742+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:16:29.742+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:29.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex stopped 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.742+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.749+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:16:29.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex closed 2023-10-21T04:16:29.750+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:16:29.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 122 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:16:29.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:16:29.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 136 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:16:29.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index destroyed 2023-10-21T04:16:29.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.774+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.780+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.780+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 172 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:29.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:29.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 155 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:29.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.790+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:29.790+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:16:29.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex started 2023-10-21T04:16:29.790+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.790+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:16:29.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex started 2023-10-21T04:16:29.791+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Warmup took 68.552642ms 2023-10-21T04:16:29.792+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 1 WriterThreads 1 2023-10-21T04:16:29.792+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Stopped Billing WUs 2023-10-21T04:16:29.792+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index Index: 13987057765505463258 Partition: 1 2023-10-21T04:16:29.792+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:29.791+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.791+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.793+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:29.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:29.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.799+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.800+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 137 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:29.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.803+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:29.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex started 2023-10-21T04:16:29.804+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.805+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 123 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:29.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:29.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex started 2023-10-21T04:16:29.809+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.809+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.808+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:29.810+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.810+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.810+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:16:29.811+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:29.811+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:16:29.811+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:16:29.811+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:29.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex stopped 2023-10-21T04:16:29.811+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.811+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.812+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.812+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.810+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.820+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.820+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:16:29.820+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex closed 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:16:29.821+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:16:29.821+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:29.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex stopped 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.830+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:16:29.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex closed 2023-10-21T04:16:29.832+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:16:29.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 123 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:16:29.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:16:29.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 137 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:16:29.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index destroyed 2023-10-21T04:16:29.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:29.853+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.853+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:29.857+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.857+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 173 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:29.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:29.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:29.867+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:29.867+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:16:29.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex started 2023-10-21T04:16:29.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.873+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.873+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 156 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:29.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:29.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:29.873+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:29.873+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:16:29.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex started 2023-10-21T04:16:29.874+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Warmup took 80.497101ms 2023-10-21T04:16:29.874+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.874+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.876+05:30 [Info] switched currmeta from 192 -> 193 force true 2023-10-21T04:16:29.877+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 2 WriterThreads 1 2023-10-21T04:16:29.877+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Stopped Billing WUs 2023-10-21T04:16:29.877+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index Index: 13987057765505463258 Partition: 2 2023-10-21T04:16:29.877+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 13987057765505463258, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:29.877+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 13987057765505463258, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:29.877+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:29.878+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13987057765505463258 2023-10-21T04:16:29.878+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13987057765505463258 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:29.878+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:29.878+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:29.878+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13987057765505463258 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:29.878+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:29.879+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13987057765505463258 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:29.879+05:30 [Info] StorageMgr::openSnapshot IndexInst:13987057765505463258 Partition:1 No Snapshot Found. 2023-10-21T04:16:29.879+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13987057765505463258 Adding Nil Snapshot. 2023-10-21T04:16:29.896+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.210692ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:29.896+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 113.682µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:29.901+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4440544141961093904:INDEX_STATE_RECOVERED 14213410630307008054:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:29.902+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 18323460706027642575, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:29.902+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 18323460706027642575, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:29.923+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.923+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.929+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 9112972261750078043,realInstId: 0, indexDefn DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:29.929+05:30 [Info] Indexer::handleRecoverIndex InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:29.930+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 392.917µs 2023-10-21T04:16:29.930+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 592.402µs 2023-10-21T04:16:29.931+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 9112972261750078043, indexDefn DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:29.932+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.760572ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:29.931+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9112972261750078043 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:29.933+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9112972261750078043 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:29.934+05:30 [Info] switched currmeta from 193 -> 194 force true 2023-10-21T04:16:29.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:29.935+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:29.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:29.937+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:29.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:29.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 142 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:29.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:29.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:29.943+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:29.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex started 2023-10-21T04:16:29.944+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.944+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 128 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:29.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:29.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:29.951+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:29.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex started 2023-10-21T04:16:29.952+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:29.952+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9112972261750078043 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:29.952+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9112972261750078043 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:29.953+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.953+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.952+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.955+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:16:29.955+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:16:29.955+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:29.955+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:16:29.956+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:16:29.958+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:29.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex stopped 2023-10-21T04:16:29.959+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.959+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.959+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.959+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.965+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:29.965+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:29.968+05:30 [Info] switched currmeta from 194 -> 195 force true 2023-10-21T04:16:29.969+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:16:29.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex closed 2023-10-21T04:16:29.970+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:29.971+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:29.971+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:29.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:16:29.971+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:29.971+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:16:29.971+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:16:29.971+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:29.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex stopped 2023-10-21T04:16:29.972+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:29.972+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:29.972+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:29.972+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:29.971+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:29.977+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:29.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:16:29.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex closed 2023-10-21T04:16:29.979+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:29.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex ... 2023-10-21T04:16:29.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 128 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex ... 2023-10-21T04:16:29.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex sucessfully destroyed 2023-10-21T04:16:29.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex ... 2023-10-21T04:16:29.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 142 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex ... 2023-10-21T04:16:29.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index destroyed 2023-10-21T04:16:29.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:29.989+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.989+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:29.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:29.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:29.993+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:29.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 148 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:29.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:29.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:29.997+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:29.997+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:16:29.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex started 2023-10-21T04:16:29.997+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:29.997+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 165 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:30.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:30.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:30.002+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:30.002+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:16:30.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex started 2023-10-21T04:16:30.002+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9112972261750078043 PartitionId 0 Warmup took 67.98915ms 2023-10-21T04:16:30.002+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.002+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9112972261750078043 partitionId 0 WriterThreads 4 2023-10-21T04:16:30.002+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9112972261750078043 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:30.002+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9112972261750078043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index Index: 9112972261750078043 Partition: 0 2023-10-21T04:16:30.002+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 9112972261750078043, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:30.002+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 9112972261750078043, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:30.002+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:30.002+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:30.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:30.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:30.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:30.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:30.007+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 9112972261750078043 2023-10-21T04:16:30.007+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:30.007+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:30.008+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:30.008+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:30.008+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:30.010+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 9112972261750078043 Partitions [0] 2023-10-21T04:16:30.010+05:30 [Info] StorageMgr::openSnapshot IndexInst:9112972261750078043 Partition:0 No Snapshot Found. 2023-10-21T04:16:30.010+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 9112972261750078043 Adding Nil Snapshot. 2023-10-21T04:16:30.013+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.243561ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:30.013+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 79.107µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:30.021+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.021+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.026+05:30 [Info] switched currmeta from 195 -> 196 force true 2023-10-21T04:16:30.198+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[13987057765505463258:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:30.198+05:30 [Info] Indexer::handleMergePartition Source 14888890778837396466 Target 13987057765505463258 2023-10-21T04:16:30.198+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00c6c8180 2023-10-21T04:16:30.198+05:30 [Info] MergePartition: sent async request to update index instance 13987057765505463258 rstate moved to ACTIVE 2023-10-21T04:16:30.204+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.593644ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:30.204+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 14888890778837396466, realInst: 13987057765505463258 2023-10-21T04:16:30.205+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16162012140118277065, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:30.205+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16162012140118277065, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:30.213+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.219+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.226+05:30 [Info] switched currmeta from 196 -> 197 force true 2023-10-21T04:16:30.236+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11460826864060385936,realInstId: 0, indexDefn DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:30.238+05:30 [Info] Indexer::handleRecoverIndex InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:30.240+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 2.180586ms 2023-10-21T04:16:30.240+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.241+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11460826864060385936, indexDefn DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:30.246+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 40.710628ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:30.241+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.048138ms 2023-10-21T04:16:30.252+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.254+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.254+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.255+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:30.255+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 110 metaSz: 2575 rp [1] sn: 1080007 count: 110 metaSz: 2575 2023-10-21T04:16:30.257+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 122033 recovery UsedSpace 165572 2023-10-21T04:16:30.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.262+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:30.262+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 110 metaSz: 2575 rp [1] sn: 720006 count: 110 metaSz: 2575 rp [2] sn: 720007 count: 110 metaSz: 2575 rp [3] sn: 1080007 count: 110 metaSz: 2575 2023-10-21T04:16:30.263+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 144138 recovery UsedSpace 177816 2023-10-21T04:16:30.263+05:30 [Info] switched currmeta from 197 -> 198 force true 2023-10-21T04:16:30.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:16:30.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.269+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:16:30.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex started 2023-10-21T04:16:30.270+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.270+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:16:30.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.281+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:16:30.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex started 2023-10-21T04:16:30.282+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.282+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.283+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:30.337+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 2 Warmup took 82.218245ms 2023-10-21T04:16:30.337+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 2 WriterThreads 1 2023-10-21T04:16:30.337+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index Index: 11460826864060385936 Partition: 2 2023-10-21T04:16:30.337+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.337+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.338+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 258 2023-10-21T04:16:30.338+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 129 metaSz: 2575 rp [1] sn: 720006 count: 129 metaSz: 2575 rp [2] sn: 720007 count: 129 metaSz: 2575 rp [3] sn: 1080007 count: 129 metaSz: 2576 2023-10-21T04:16:30.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.338+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:30.338+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 129 metaSz: 2575 rp [1] sn: 1080007 count: 129 metaSz: 2576 2023-10-21T04:16:30.340+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.340+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.341+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 152030 recovery UsedSpace 178534 2023-10-21T04:16:30.342+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 125183 recovery UsedSpace 165572 2023-10-21T04:16:30.344+05:30 [Info] switched currmeta from 198 -> 199 force true 2023-10-21T04:16:30.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:16:30.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.357+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:16:30.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex started 2023-10-21T04:16:30.357+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:16:30.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.357+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:16:30.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex started 2023-10-21T04:16:30.357+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.357+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.358+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.358+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:30.393+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 3 Warmup took 55.994805ms 2023-10-21T04:16:30.395+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 3 WriterThreads 1 2023-10-21T04:16:30.395+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index Index: 11460826864060385936 Partition: 3 2023-10-21T04:16:30.395+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.396+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.397+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:30.398+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 140 metaSz: 2575 rp [1] sn: 720006 count: 140 metaSz: 2576 rp [2] sn: 720007 count: 140 metaSz: 2575 rp [3] sn: 1080007 count: 140 metaSz: 2576 2023-10-21T04:16:30.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.401+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:30.402+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 720006 count: 140 metaSz: 2576 rp [1] sn: 720007 count: 140 metaSz: 2575 rp [2] sn: 1080007 count: 140 metaSz: 2576 2023-10-21T04:16:30.405+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 126850 recovery UsedSpace 157358 2023-10-21T04:16:30.402+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 153692 recovery UsedSpace 178524 2023-10-21T04:16:30.407+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.407+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.411+05:30 [Info] switched currmeta from 199 -> 200 force true 2023-10-21T04:16:30.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:16:30.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.418+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:16:30.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex started 2023-10-21T04:16:30.418+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.418+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:16:30.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.420+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:16:30.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex started 2023-10-21T04:16:30.420+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:30.420+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.420+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.454+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 4 Warmup took 57.331437ms 2023-10-21T04:16:30.454+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 4 WriterThreads 1 2023-10-21T04:16:30.454+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index Index: 11460826864060385936 Partition: 4 2023-10-21T04:16:30.454+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.454+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:30.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 116 metaSz: 2575 rp [1] sn: 1080007 count: 116 metaSz: 2576 2023-10-21T04:16:30.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.455+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:30.455+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 116 metaSz: 2575 rp [1] sn: 1080007 count: 116 metaSz: 2576 2023-10-21T04:16:30.455+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 113700 recovery UsedSpace 173764 2023-10-21T04:16:30.457+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 123133 recovery UsedSpace 165572 2023-10-21T04:16:30.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:16:30.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:16:30.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex started 2023-10-21T04:16:30.467+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.467+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:16:30.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:16:30.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex started 2023-10-21T04:16:30.470+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:30.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.471+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.490+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 5 Warmup took 35.947596ms 2023-10-21T04:16:30.491+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 5 WriterThreads 1 2023-10-21T04:16:30.491+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index Index: 11460826864060385936 Partition: 5 2023-10-21T04:16:30.491+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.491+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.492+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:30.492+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 127 metaSz: 2575 rp [1] sn: 720006 count: 127 metaSz: 2575 rp [2] sn: 720007 count: 127 metaSz: 2575 rp [3] sn: 1080007 count: 127 metaSz: 2576 2023-10-21T04:16:30.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.492+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:30.492+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 127 metaSz: 2575 rp [1] sn: 1080007 count: 127 metaSz: 2576 2023-10-21T04:16:30.493+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 151446 recovery UsedSpace 178524 2023-10-21T04:16:30.493+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 124875 recovery UsedSpace 165572 2023-10-21T04:16:30.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:16:30.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.499+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:16:30.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex started 2023-10-21T04:16:30.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.500+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:16:30.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.503+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:16:30.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex started 2023-10-21T04:16:30.503+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:30.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.506+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.520+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 6 Warmup took 28.791634ms 2023-10-21T04:16:30.521+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 6 WriterThreads 1 2023-10-21T04:16:30.521+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index Index: 11460826864060385936 Partition: 6 2023-10-21T04:16:30.521+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.521+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.522+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:30.522+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 125 metaSz: 2575 rp [1] sn: 1080007 count: 125 metaSz: 2576 2023-10-21T04:16:30.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.522+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 240 2023-10-21T04:16:30.522+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 720006 count: 125 metaSz: 2575 rp [1] sn: 720007 count: 125 metaSz: 2575 rp [2] sn: 1080007 count: 125 metaSz: 2576 2023-10-21T04:16:30.522+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 124883 recovery UsedSpace 165572 2023-10-21T04:16:30.523+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 118607 recovery UsedSpace 182454 2023-10-21T04:16:30.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:16:30.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.530+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:16:30.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex started 2023-10-21T04:16:30.530+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.530+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:16:30.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.531+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:16:30.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex started 2023-10-21T04:16:30.532+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:30.532+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.532+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.549+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 7 Warmup took 27.809867ms 2023-10-21T04:16:30.549+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 7 WriterThreads 1 2023-10-21T04:16:30.549+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index Index: 11460826864060385936 Partition: 7 2023-10-21T04:16:30.549+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.550+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.551+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:30.551+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 124 metaSz: 2575 rp [1] sn: 1080007 count: 124 metaSz: 2576 2023-10-21T04:16:30.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:30.551+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 124 metaSz: 2575 rp [1] sn: 720006 count: 124 metaSz: 2575 rp [2] sn: 720007 count: 124 metaSz: 2575 rp [3] sn: 1080007 count: 124 metaSz: 2576 2023-10-21T04:16:30.552+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 124525 recovery UsedSpace 165572 2023-10-21T04:16:30.552+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 146176 recovery UsedSpace 157336 2023-10-21T04:16:30.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:16:30.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.563+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:16:30.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex started 2023-10-21T04:16:30.563+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:16:30.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.563+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:16:30.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex started 2023-10-21T04:16:30.563+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:30.563+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.563+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.563+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.612+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 8 Warmup took 62.057015ms 2023-10-21T04:16:30.613+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 8 WriterThreads 1 2023-10-21T04:16:30.613+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index Index: 11460826864060385936 Partition: 8 2023-10-21T04:16:30.613+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:30.614+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:30.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:30.615+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:30.615+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 129 metaSz: 2575 rp [1] sn: 1080007 count: 129 metaSz: 2576 2023-10-21T04:16:30.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:30.615+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:30.615+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 129 metaSz: 2575 rp [1] sn: 720006 count: 129 metaSz: 2575 rp [2] sn: 720007 count: 129 metaSz: 2575 rp [3] sn: 1080007 count: 129 metaSz: 2576 2023-10-21T04:16:30.615+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 125041 recovery UsedSpace 165572 2023-10-21T04:16:30.618+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 151715 recovery UsedSpace 178524 2023-10-21T04:16:30.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:16:30.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:30.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:30.625+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:16:30.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex started 2023-10-21T04:16:30.625+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.625+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:16:30.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:30.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:30.628+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:16:30.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex started 2023-10-21T04:16:30.628+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:30.628+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:30.629+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:30.648+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 1 Warmup took 33.342087ms 2023-10-21T04:16:30.648+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 1 WriterThreads 1 2023-10-21T04:16:30.648+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index Index: 11460826864060385936 Partition: 1 2023-10-21T04:16:30.648+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11460826864060385936, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:30.648+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11460826864060385936, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:30.648+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:30.649+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11460826864060385936 2023-10-21T04:16:30.649+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:30.649+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:30.649+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:30.650+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:30.650+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:30.650+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11460826864060385936 Partitions [4 5 6 7 8 1 2 3] 2023-10-21T04:16:30.651+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:5 Attempting to open snapshot (SnapshotInfo: count:116 committed:false) 2023-10-21T04:16:30.652+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:16:30.653+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:7 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:16:30.654+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:8 Attempting to open snapshot (SnapshotInfo: count:124 committed:false) 2023-10-21T04:16:30.655+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.289849ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:30.655+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 101.043µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:30.655+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:1 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:16:30.664+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.664+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.667+05:30 [Info] switched currmeta from 200 -> 201 force true 2023-10-21T04:16:30.668+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:2 Attempting to open snapshot (SnapshotInfo: count:110 committed:false) 2023-10-21T04:16:30.670+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:3 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:16:30.676+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:4 Attempting to open snapshot (SnapshotInfo: count:140 committed:false) 2023-10-21T04:16:30.766+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.766+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.770+05:30 [Info] switched currmeta from 201 -> 202 force true 2023-10-21T04:16:30.781+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.783+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.789+05:30 [Info] switched currmeta from 202 -> 203 force true 2023-10-21T04:16:30.937+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[9112972261750078043:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:30.937+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007479320 2023-10-21T04:16:30.937+05:30 [Info] handleUpdateIndexRState: Index instance 9112972261750078043 rstate moved to ACTIVE 2023-10-21T04:16:30.958+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.375382ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:30.958+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 2872726210871813325, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:30.958+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 2872726210871813325, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:30.966+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:30.966+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:30.968+05:30 [Info] switched currmeta from 203 -> 204 force true 2023-10-21T04:16:30.996+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11375521620401506748,realInstId: 0, indexDefn DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:30.999+05:30 [Info] Indexer::handleRecoverIndex InstId: 11375521620401506748 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:31.000+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 1.082981ms 2023-10-21T04:16:31.000+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 1.134852ms 2023-10-21T04:16:31.000+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.001+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.001+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11375521620401506748, indexDefn DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:31.001+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 42.707922ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:31.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.004+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.004+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.005+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.006+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:31.009+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.009+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.016+05:30 [Info] switched currmeta from 204 -> 205 force true 2023-10-21T04:16:31.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 123 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:31.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 137 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:31.023+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:31.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex started 2023-10-21T04:16:31.028+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.029+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.032+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:31.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex started 2023-10-21T04:16:31.033+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.033+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.033+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:31.036+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.036+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.036+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.036+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:16:31.036+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:31.037+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:16:31.037+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:16:31.037+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex stopped 2023-10-21T04:16:31.037+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.037+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.037+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.037+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.044+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:16:31.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex closed 2023-10-21T04:16:31.046+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.046+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.046+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.046+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:16:31.046+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:31.046+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:16:31.046+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:16:31.046+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:31.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex stopped 2023-10-21T04:16:31.046+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.046+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.046+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.047+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.067+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:16:31.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex closed 2023-10-21T04:16:31.067+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex ... 2023-10-21T04:16:31.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 123 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex ... 2023-10-21T04:16:31.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex ... 2023-10-21T04:16:31.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 137 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex ... 2023-10-21T04:16:31.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index destroyed 2023-10-21T04:16:31.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.095+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.105+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.105+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 149 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:31.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.112+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:31.112+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:16:31.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex started 2023-10-21T04:16:31.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 166 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:31.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.113+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:31.113+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:16:31.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex started 2023-10-21T04:16:31.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.114+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 1 Warmup took 112.599255ms 2023-10-21T04:16:31.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.114+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 1 WriterThreads 1 2023-10-21T04:16:31.114+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index Index: 11375521620401506748 Partition: 1 2023-10-21T04:16:31.114+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.114+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.114+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.115+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.115+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.116+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:31.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 124 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:31.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:31.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex started 2023-10-21T04:16:31.122+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.122+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 138 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex started 2023-10-21T04:16:31.126+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.126+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:16:31.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:31.126+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:16:31.126+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:16:31.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex stopped 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.126+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.132+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:16:31.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex closed 2023-10-21T04:16:31.132+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.132+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.132+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.132+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:16:31.132+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:31.133+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:16:31.133+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:16:31.133+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:31.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex stopped 2023-10-21T04:16:31.133+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.133+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.133+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.133+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.143+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:16:31.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex closed 2023-10-21T04:16:31.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex ... 2023-10-21T04:16:31.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 124 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex ... 2023-10-21T04:16:31.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex ... 2023-10-21T04:16:31.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 138 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex ... 2023-10-21T04:16:31.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index destroyed 2023-10-21T04:16:31.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.159+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.159+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.164+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 150 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:31.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.165+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:31.165+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:16:31.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex started 2023-10-21T04:16:31.166+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.166+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 167 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:31.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.183+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:31.183+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:16:31.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex started 2023-10-21T04:16:31.183+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.183+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.183+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 2 Warmup took 69.105856ms 2023-10-21T04:16:31.184+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 2 WriterThreads 1 2023-10-21T04:16:31.184+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index Index: 11375521620401506748 Partition: 2 2023-10-21T04:16:31.184+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.184+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.184+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.184+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.184+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.184+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.185+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.185+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 125 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.196+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex started 2023-10-21T04:16:31.197+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.197+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 139 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:31.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.197+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:31.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex started 2023-10-21T04:16:31.197+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:31.197+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.197+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.197+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.197+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.197+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:16:31.197+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:31.197+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:16:31.197+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:16:31.198+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.198+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:31.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex stopped 2023-10-21T04:16:31.198+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.198+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.198+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.198+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.215+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:16:31.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex closed 2023-10-21T04:16:31.215+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.215+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.216+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.216+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:16:31.216+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:31.216+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:16:31.216+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:16:31.216+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:31.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex stopped 2023-10-21T04:16:31.216+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.216+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.216+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.216+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.234+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:16:31.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex closed 2023-10-21T04:16:31.235+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex ... 2023-10-21T04:16:31.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 125 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex ... 2023-10-21T04:16:31.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex ... 2023-10-21T04:16:31.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 139 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex ... 2023-10-21T04:16:31.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index destroyed 2023-10-21T04:16:31.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.252+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.252+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.256+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.256+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.260+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[9720186692804226822:INDEX_STATE_RECOVERED 11460826864060385936:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:31.260+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 8857269605176042330, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:31.261+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 8857269605176042330, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:31.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 168 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:31.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.262+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:31.262+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:16:31.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex started 2023-10-21T04:16:31.263+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.263+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 151 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:31.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.267+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:31.267+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:16:31.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex started 2023-10-21T04:16:31.267+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 3 Warmup took 82.860596ms 2023-10-21T04:16:31.267+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 3 WriterThreads 1 2023-10-21T04:16:31.267+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index Index: 11375521620401506748 Partition: 3 2023-10-21T04:16:31.267+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.267+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.268+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.268+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.268+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.268+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.268+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.269+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:31.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 126 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:31.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.283+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:31.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex started 2023-10-21T04:16:31.283+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.283+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 140 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:31.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.287+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:31.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex started 2023-10-21T04:16:31.287+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:31.287+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.287+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2296526085582506469,realInstId: 0, indexDefn DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:31.287+05:30 [Info] Indexer::handleRecoverIndex InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:16:31.287+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:31.287+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:16:31.287+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 143.746µs 2023-10-21T04:16:31.287+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 183.83µs 2023-10-21T04:16:31.287+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:16:31.287+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:31.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex stopped 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.287+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2296526085582506469 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:31.288+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2296526085582506469 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:31.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:31.288+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1054936 memSz 772 2023-10-21T04:16:31.288+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 1000 metaSz: 2554 rp [1] sn: 1080007 count: 1000 metaSz: 2554 2023-10-21T04:16:31.288+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2296526085582506469, indexDefn DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:31.288+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 27.684607ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:31.288+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 209333 recovery UsedSpace 168569 2023-10-21T04:16:31.290+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.298+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.298+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.303+05:30 [Info] switched currmeta from 205 -> 206 force true 2023-10-21T04:16:31.309+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:16:31.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex closed 2023-10-21T04:16:31.309+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:16:31.310+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:31.310+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:16:31.310+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:16:31.310+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:31.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex stopped 2023-10-21T04:16:31.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.310+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:16:31.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:31.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:31.314+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:16:31.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex started 2023-10-21T04:16:31.314+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2296526085582506469 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:31.314+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.314+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.329+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:16:31.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex closed 2023-10-21T04:16:31.329+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex ... 2023-10-21T04:16:31.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 126 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex ... 2023-10-21T04:16:31.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex ... 2023-10-21T04:16:31.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 140 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex ... 2023-10-21T04:16:31.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index destroyed 2023-10-21T04:16:31.343+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2296526085582506469 PartitionId 0 Warmup took 55.420514ms 2023-10-21T04:16:31.343+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2296526085582506469 partitionId 0 WriterThreads 4 2023-10-21T04:16:31.343+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2296526085582506469 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index Index: 2296526085582506469 Partition: 0 2023-10-21T04:16:31.344+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2296526085582506469, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:16:31.344+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2296526085582506469, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:16:31.344+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:31.346+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2296526085582506469 2023-10-21T04:16:31.347+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:31.347+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:31.348+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:31.348+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:31.348+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:31.348+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2296526085582506469 Partitions [0] 2023-10-21T04:16:31.349+05:30 [Info] StorageMgr::openSnapshot IndexInst:2296526085582506469 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:31.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.350+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.353+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.793304ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:31.353+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 74.438µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:31.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.363+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.363+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 169 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:31.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.364+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:31.364+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:16:31.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex started 2023-10-21T04:16:31.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.376+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.376+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 152 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:31.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.398+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:31.398+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:16:31.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex started 2023-10-21T04:16:31.398+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.398+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.398+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 4 Warmup took 130.939852ms 2023-10-21T04:16:31.399+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 4 WriterThreads 1 2023-10-21T04:16:31.399+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index Index: 11375521620401506748 Partition: 4 2023-10-21T04:16:31.399+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.399+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.400+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.400+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.400+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.402+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:31.410+05:30 [Info] switched currmeta from 206 -> 208 force true 2023-10-21T04:16:31.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 127 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:31.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.417+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:31.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex started 2023-10-21T04:16:31.417+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.417+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 141 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:31.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.418+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:31.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex started 2023-10-21T04:16:31.419+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.419+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.419+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:31.420+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.419+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:16:31.420+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:31.420+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:16:31.419+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.420+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:16:31.420+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:31.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex stopped 2023-10-21T04:16:31.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.422+05:30 [Info] switched currmeta from 208 -> 208 force true 2023-10-21T04:16:31.430+05:30 [Error] ScanCoordinator: Unable to compute index items_count for bucket_1/idx_partitioned_defer/11375521620401506748 state 0 (Index not found) 2023-10-21T04:16:31.446+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:16:31.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex closed 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:16:31.447+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:31.447+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:16:31.447+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:16:31.447+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:31.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex stopped 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.447+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.468+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:16:31.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex closed 2023-10-21T04:16:31.468+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex ... 2023-10-21T04:16:31.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 127 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex ... 2023-10-21T04:16:31.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex ... 2023-10-21T04:16:31.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 141 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex ... 2023-10-21T04:16:31.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index destroyed 2023-10-21T04:16:31.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.494+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.494+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.509+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.509+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.513+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.513+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.516+05:30 [Info] switched currmeta from 208 -> 209 force true 2023-10-21T04:16:31.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 170 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:31.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.519+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:31.519+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:16:31.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex started 2023-10-21T04:16:31.519+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.519+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 153 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:31.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.527+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:31.527+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:16:31.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex started 2023-10-21T04:16:31.528+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 5 Warmup took 128.107844ms 2023-10-21T04:16:31.528+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 5 WriterThreads 1 2023-10-21T04:16:31.528+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index Index: 11375521620401506748 Partition: 5 2023-10-21T04:16:31.528+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.528+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.528+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.528+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.528+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.529+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.529+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.532+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:31.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 120 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:31.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.539+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:31.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex started 2023-10-21T04:16:31.539+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.539+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 134 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:31.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.547+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:31.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex started 2023-10-21T04:16:31.547+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:31.547+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:16:31.548+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:31.549+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:16:31.549+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:16:31.550+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:31.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex stopped 2023-10-21T04:16:31.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.550+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.549+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.567+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:16:31.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex closed 2023-10-21T04:16:31.568+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.568+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.569+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.569+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:16:31.569+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:31.569+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:16:31.569+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:16:31.569+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:31.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex stopped 2023-10-21T04:16:31.570+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.570+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.570+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.570+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.584+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:16:31.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex closed 2023-10-21T04:16:31.585+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex ... 2023-10-21T04:16:31.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 120 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex ... 2023-10-21T04:16:31.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex ... 2023-10-21T04:16:31.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 134 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex ... 2023-10-21T04:16:31.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index destroyed 2023-10-21T04:16:31.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.608+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.609+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.613+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.613+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 154 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:31.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.620+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:31.620+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:16:31.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex started 2023-10-21T04:16:31.620+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.621+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 171 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:31.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.624+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:31.625+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:16:31.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex started 2023-10-21T04:16:31.625+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 6 Warmup took 96.725171ms 2023-10-21T04:16:31.625+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 6 WriterThreads 1 2023-10-21T04:16:31.625+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index Index: 11375521620401506748 Partition: 6 2023-10-21T04:16:31.625+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.625+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.625+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.625+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.626+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.626+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:31.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.626+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.627+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 135 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:31.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.635+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:31.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex started 2023-10-21T04:16:31.635+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.635+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 121 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:31.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.636+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:31.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex started 2023-10-21T04:16:31.636+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:31.636+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.636+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.636+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.636+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:16:31.636+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:31.636+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:16:31.636+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:16:31.637+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:31.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex stopped 2023-10-21T04:16:31.637+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.637+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.637+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.637+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.637+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.637+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:16:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex closed 2023-10-21T04:16:31.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.644+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.644+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.644+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:16:31.644+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:31.644+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:16:31.644+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:16:31.644+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex stopped 2023-10-21T04:16:31.644+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.645+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.645+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.645+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.654+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:16:31.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex closed 2023-10-21T04:16:31.654+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex ... 2023-10-21T04:16:31.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 121 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex ... 2023-10-21T04:16:31.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex ... 2023-10-21T04:16:31.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 135 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex ... 2023-10-21T04:16:31.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index destroyed 2023-10-21T04:16:31.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.667+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.667+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.675+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.675+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 155 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:31.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.686+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:31.686+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:16:31.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex started 2023-10-21T04:16:31.686+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.686+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 172 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:31.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.695+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:31.695+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:16:31.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex started 2023-10-21T04:16:31.696+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 7 Warmup took 70.347094ms 2023-10-21T04:16:31.696+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.696+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.696+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 7 WriterThreads 1 2023-10-21T04:16:31.696+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index Index: 11375521620401506748 Partition: 7 2023-10-21T04:16:31.697+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11375521620401506748 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:31.698+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11375521620401506748 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:31.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.698+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.698+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:31.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.698+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:31.699+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:31.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 122 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:31.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.717+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex started 2023-10-21T04:16:31.717+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.717+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 136 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.717+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex started 2023-10-21T04:16:31.717+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.718+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:31.718+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:16:31.718+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:31.718+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:16:31.718+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:16:31.718+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:31.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex stopped 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.718+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.727+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:16:31.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex closed 2023-10-21T04:16:31.729+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:31.729+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:31.729+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:31.730+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:31.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:16:31.730+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:31.730+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:16:31.731+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:16:31.731+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:31.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex stopped 2023-10-21T04:16:31.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:31.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:31.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:31.731+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:31.738+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:31.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:16:31.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex closed 2023-10-21T04:16:31.740+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:31.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex ... 2023-10-21T04:16:31.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 122 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex ... 2023-10-21T04:16:31.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex sucessfully destroyed 2023-10-21T04:16:31.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex ... 2023-10-21T04:16:31.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 136 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex ... 2023-10-21T04:16:31.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index destroyed 2023-10-21T04:16:31.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:31.757+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:31.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:31.765+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:31.765+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:31.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 156 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:31.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:31.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:31.783+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:31.783+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:16:31.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex started 2023-10-21T04:16:31.783+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.783+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 173 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:31.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:31.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:31.785+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:31.785+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:16:31.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex started 2023-10-21T04:16:31.785+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11375521620401506748 PartitionId 8 Warmup took 87.660378ms 2023-10-21T04:16:31.785+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:31.786+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11375521620401506748 partitionId 8 WriterThreads 1 2023-10-21T04:16:31.785+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:31.786+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11375521620401506748 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index Index: 11375521620401506748 Partition: 8 2023-10-21T04:16:31.786+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11375521620401506748, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:31.786+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11375521620401506748, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:31.786+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11375521620401506748 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:31.787+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11375521620401506748 2023-10-21T04:16:31.787+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11375521620401506748 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:31.787+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:31.787+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:31.787+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11375521620401506748 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:31.787+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11375521620401506748 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:31.788+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11375521620401506748 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:16:31.788+05:30 [Info] StorageMgr::openSnapshot IndexInst:11375521620401506748 Partition:2 No Snapshot Found. 2023-10-21T04:16:31.788+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11375521620401506748 Adding Nil Snapshot. 2023-10-21T04:16:31.798+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.228126ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:31.799+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 99.863µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:31.848+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.848+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.851+05:30 [Info] switched currmeta from 209 -> 211 force true 2023-10-21T04:16:31.863+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.863+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.866+05:30 [Info] switched currmeta from 211 -> 213 force true 2023-10-21T04:16:31.893+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:31.893+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:31.900+05:30 [Info] switched currmeta from 213 -> 213 force true 2023-10-21T04:16:32.007+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11375521620401506748:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:32.007+05:30 [Info] Indexer::handleMergePartition Source 4412153410740056345 Target 11375521620401506748 2023-10-21T04:16:32.007+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11375521620401506748 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00c46f9e0 2023-10-21T04:16:32.008+05:30 [Info] MergePartition: sent async request to update index instance 11375521620401506748 rstate moved to ACTIVE 2023-10-21T04:16:32.018+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.191183ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:32.018+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 4412153410740056345, realInst: 11375521620401506748 2023-10-21T04:16:32.021+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 1571318351652255745, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:32.021+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 1571318351652255745, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:32.027+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.028+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.033+05:30 [Info] switched currmeta from 213 -> 214 force true 2023-10-21T04:16:32.046+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7494249569853685720,realInstId: 0, indexDefn DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:32.046+05:30 [Info] Indexer::handleRecoverIndex InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:32.046+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 235.245µs 2023-10-21T04:16:32.046+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 284.971µs 2023-10-21T04:16:32.046+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7494249569853685720 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:32.046+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7494249569853685720 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:32.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:32.047+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1054936 memSz 790 2023-10-21T04:16:32.047+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 720006 count: 1000 metaSz: 2553 rp [1] sn: 1080007 count: 1000 metaSz: 2553 2023-10-21T04:16:32.047+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7494249569853685720, indexDefn DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:32.047+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 26.582298ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:32.047+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 201800 recovery UsedSpace 151843 2023-10-21T04:16:32.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex 2023-10-21T04:16:32.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:32.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:32.067+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex 2023-10-21T04:16:32.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex started 2023-10-21T04:16:32.067+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7494249569853685720 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:32.068+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:32.068+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:32.070+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.075+05:30 [Info] switched currmeta from 214 -> 215 force true 2023-10-21T04:16:32.099+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7494249569853685720 PartitionId 0 Warmup took 51.941845ms 2023-10-21T04:16:32.099+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7494249569853685720 partitionId 0 WriterThreads 4 2023-10-21T04:16:32.099+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7494249569853685720 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:32.099+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7494249569853685720 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index Index: 7494249569853685720 Partition: 0 2023-10-21T04:16:32.099+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7494249569853685720, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:16:32.099+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7494249569853685720, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:16:32.099+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:32.101+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7494249569853685720 2023-10-21T04:16:32.101+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:32.101+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:32.102+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:32.103+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:32.104+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:32.104+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7494249569853685720 Partitions [0] 2023-10-21T04:16:32.108+05:30 [Info] StorageMgr::openSnapshot IndexInst:7494249569853685720 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:32.116+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.771325ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:32.117+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 69.419µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:32.129+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.132+05:30 [Info] switched currmeta from 215 -> 216 force true 2023-10-21T04:16:32.298+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2296526085582506469:INDEX_STATE_RECOVERED 9720186692804226822:INDEX_STATE_RECOVERED 11460826864060385936:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:32.299+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 13856704970496601958, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:32.299+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 13856704970496601958, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:32.322+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 16733302726553098964,realInstId: 0, indexDefn DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:32.322+05:30 [Info] Indexer::handleRecoverIndex InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:32.322+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 204.493µs 2023-10-21T04:16:32.322+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 251.504µs 2023-10-21T04:16:32.322+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16733302726553098964 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:32.322+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16733302726553098964 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:32.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:32.323+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:32.323+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 16733302726553098964, indexDefn DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:32.323+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 24.020561ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:32.323+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:32.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 143 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:32.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:32.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:32.334+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:32.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex started 2023-10-21T04:16:32.335+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16733302726553098964 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:32.335+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16733302726553098964 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:32.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:16:32.335+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:16:32.335+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:32.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex stopped 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:32.335+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:32.341+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:32.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:16:32.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex closed 2023-10-21T04:16:32.344+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:32.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:16:32.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 143 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:16:32.348+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.348+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:32.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index destroyed 2023-10-21T04:16:32.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:32.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:32.354+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:32.354+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:32.354+05:30 [Info] switched currmeta from 216 -> 218 force true 2023-10-21T04:16:32.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 174 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:32.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:32.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:32.362+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:32.362+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:16:32.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex started 2023-10-21T04:16:32.363+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16733302726553098964 PartitionId 0 Warmup took 39.984089ms 2023-10-21T04:16:32.363+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16733302726553098964 partitionId 0 WriterThreads 4 2023-10-21T04:16:32.363+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16733302726553098964 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index Index: 16733302726553098964 Partition: 0 2023-10-21T04:16:32.363+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 16733302726553098964, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:16:32.363+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 16733302726553098964, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:16:32.364+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:32.365+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 16733302726553098964 2023-10-21T04:16:32.365+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16733302726553098964 Name: #primary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:32.366+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:32.366+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:32.364+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:32.364+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:32.367+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.368+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.368+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16733302726553098964 Name: #primary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:32.369+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:32.369+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 16733302726553098964 Partitions [0] 2023-10-21T04:16:32.369+05:30 [Info] StorageMgr::openSnapshot IndexInst:16733302726553098964 Partition:0 No Snapshot Found. 2023-10-21T04:16:32.369+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 16733302726553098964 Adding Nil Snapshot. 2023-10-21T04:16:32.372+05:30 [Info] switched currmeta from 218 -> 218 force true 2023-10-21T04:16:32.374+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.601961ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:32.374+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 128.313µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:32.395+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.395+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.407+05:30 [Info] switched currmeta from 218 -> 219 force true 2023-10-21T04:16:32.427+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.427+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.445+05:30 [Info] switched currmeta from 219 -> 221 force true 2023-10-21T04:16:32.478+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.478+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.487+05:30 [Info] switched currmeta from 221 -> 223 force true 2023-10-21T04:16:32.501+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.501+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.513+05:30 [Info] switched currmeta from 223 -> 223 force true 2023-10-21T04:16:32.545+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.545+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.561+05:30 [Info] switched currmeta from 223 -> 226 force true 2023-10-21T04:16:32.580+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.580+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.588+05:30 [Info] switched currmeta from 226 -> 226 force true 2023-10-21T04:16:32.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:32.944+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:32.949+05:30 [Info] switched currmeta from 226 -> 227 force true 2023-10-21T04:16:33.012+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.012+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.017+05:30 [Info] switched currmeta from 227 -> 228 force true 2023-10-21T04:16:33.062+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4440544141961093904:INDEX_STATE_RECOVERED 7494249569853685720:INDEX_STATE_RECOVERED 14213410630307008054:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:33.063+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 17489537288579938718, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:33.063+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 17489537288579938718, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:33.090+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11790381330881939912,realInstId: 0, indexDefn DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:33.091+05:30 [Info] Indexer::handleRecoverIndex InstId: 11790381330881939912 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:33.091+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 165.386µs 2023-10-21T04:16:33.091+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 211.472µs 2023-10-21T04:16:33.091+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11790381330881939912 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:33.091+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11790381330881939912, indexDefn DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:33.091+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 27.954385ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.091+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11790381330881939912 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:33.091+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:33.091+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:33.092+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:33.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 143 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:33.099+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:33.099+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:33.100+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:33.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex started 2023-10-21T04:16:33.102+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11790381330881939912 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:33.103+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11790381330881939912 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:33.104+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:33.102+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:33.103+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:33.106+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:33.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:16:33.107+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:33.108+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:16:33.108+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:16:33.109+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:33.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex stopped 2023-10-21T04:16:33.109+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:33.109+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:33.109+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:33.107+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:33.110+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:33.114+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:33.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:16:33.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex closed 2023-10-21T04:16:33.123+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:33.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex ... 2023-10-21T04:16:33.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 143 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex ... 2023-10-21T04:16:33.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:33.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index destroyed 2023-10-21T04:16:33.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:33.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:33.136+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:33.136+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:33.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 174 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:33.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:33.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:33.141+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:33.141+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:16:33.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex started 2023-10-21T04:16:33.141+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11790381330881939912 PartitionId 0 Warmup took 49.533947ms 2023-10-21T04:16:33.141+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11790381330881939912 partitionId 0 WriterThreads 4 2023-10-21T04:16:33.141+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11790381330881939912 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:33.141+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:33.141+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11790381330881939912 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index Index: 11790381330881939912 Partition: 0 2023-10-21T04:16:33.141+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:33.141+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11790381330881939912, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:16:33.141+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11790381330881939912, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:16:33.141+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11790381330881939912 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:33.142+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11790381330881939912 2023-10-21T04:16:33.142+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11790381330881939912 Name: #primary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:33.142+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:33.142+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:33.142+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11790381330881939912 Name: #primary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:33.142+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11790381330881939912 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:33.142+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11790381330881939912 Partitions [0] 2023-10-21T04:16:33.143+05:30 [Info] StorageMgr::openSnapshot IndexInst:11790381330881939912 Partition:0 No Snapshot Found. 2023-10-21T04:16:33.143+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11790381330881939912 Adding Nil Snapshot. 2023-10-21T04:16:33.152+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.152+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.156+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.236495ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.157+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 108.537µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.160+05:30 [Info] switched currmeta from 228 -> 233 force true 2023-10-21T04:16:33.187+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.187+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.191+05:30 [Info] switched currmeta from 233 -> 233 force true 2023-10-21T04:16:33.333+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[16733302726553098964:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:33.334+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00683d0e0 2023-10-21T04:16:33.334+05:30 [Info] handleUpdateIndexRState: Index instance 16733302726553098964 rstate moved to ACTIVE 2023-10-21T04:16:33.341+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.846942ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.341+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [12027054384338714804 16162012140118277065 8857269605176042330] belonging to collectionId: 0, ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c. Initiating build 2023-10-21T04:16:33.355+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.355+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.358+05:30 [Info] switched currmeta from 233 -> 234 force true 2023-10-21T04:16:33.374+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.376+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.386+05:30 [Info] switched currmeta from 234 -> 235 force true 2023-10-21T04:16:33.405+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.405+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.409+05:30 [Info] switched currmeta from 235 -> 236 force true 2023-10-21T04:16:33.412+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [9720186692804226822 11460826864060385936 2296526085582506469] &{2} 2023-10-21T04:16:33.412+05:30 [Info] Indexer::handleBuildRecoveredIndexes [9720186692804226822 11460826864060385936 2296526085582506469] 2023-10-21T04:16:33.412+05:30 [Info] bulkUpdateRState: Index instance 9720186692804226822 rstate moved to PENDING 2023-10-21T04:16:33.412+05:30 [Info] bulkUpdateRState: Index instance 11460826864060385936 rstate moved to PENDING 2023-10-21T04:16:33.412+05:30 [Info] bulkUpdateRState: Index instance 2296526085582506469 rstate moved to PENDING 2023-10-21T04:16:33.412+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [9720186692804226822 11460826864060385936 2296526085582506469] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:33.412+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:33.413+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:33.413+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:33.421+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.423+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.426+05:30 [Info] switched currmeta from 236 -> 237 force true 2023-10-21T04:16:33.433+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2 2023-10-21T04:16:33.433+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:33.433+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:33.433+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:33.433+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:16:33.433+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:16:33.433+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:33.433+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:33.434+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:16:33.434+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:33.434+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:33.434+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:16:33.434+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2 SessionId 84972 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:33.434+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_%2. OSO not supported for default scope/collection. 2023-10-21T04:16:33.434+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:33.434+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:33.434+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream INIT_STREAM 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_%2:1]] 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 9720186692804226822 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_%2:2]] 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 2296526085582506469 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 11460826864060385936 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:33.434+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2 2023-10-21T04:16:33.434+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: true SessionId: 84972 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:16:33.436+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:16:33.436+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:16:33.436+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:33.437+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 24.995966ms 2023-10-21T04:16:33.437+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 25.035065ms 2023-10-21T04:16:33.437+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [9720186692804226822 11460826864060385936 2296526085582506469] 2023-10-21T04:16:33.437+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 92.900548ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.440+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[2296526085582506469:true 9720186692804226822:true 11460826864060385936:true] 2023-10-21T04:16:33.443+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:16:33.445+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2 Check pending stream merge. 2023-10-21T04:16:33.449+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.995701ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.466+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.100205ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.479+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.934417ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:33.485+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.488+05:30 [Info] switched currmeta from 237 -> 240 force true 2023-10-21T04:16:33.496+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48380" doAuth() ... 2023-10-21T04:16:33.496+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48380" auth successful 2023-10-21T04:16:33.497+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:48380" +1 2023-10-21T04:16:33.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.500+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.504+05:30 [Info] switched currmeta from 240 -> 240 force true 2023-10-21T04:16:33.666+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:33.674+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.674+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.676+05:30 [Info] switched currmeta from 240 -> 240 force true 2023-10-21T04:16:33.855+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 0 263268055970459 52 84972 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:33.857+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 1 176639880977228 61 84972 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:33.862+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 3 215749515705641 45 84972 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:33.865+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 2 35403626585217 50 84972 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:33.865+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 4 159113185399719 58 84972 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:33.870+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 7 170288557775498 52 84972 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:33.870+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 6 261493429196201 61 84972 172.31.5.112:9000. HWT [60-61,61]. 2023-10-21T04:16:33.870+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 5 3874519919278 54 84972 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:33.877+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 8 264975169139761 56 84972 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:33.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 9 236398693961229 62 84972 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:33.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 10 206533515052028 50 84972 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:33.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 11 234758907181738 56 84972 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:33.885+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 12 116104661063478 55 84972 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:33.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 15 162158720181435 67 84972 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:33.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 13 102350060311343 53 84972 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:33.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 14 140236896721046 60 84972 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:33.893+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 16 200752780087280 54 84972 172.31.5.112:9000. HWT [49-54,54]. 2023-10-21T04:16:33.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 17 98820809009123 44 84972 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:16:33.897+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 19 208836052671833 45 84972 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:33.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 18 207134287529256 46 84972 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:33.899+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 20 92956357968546 50 84972 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:33.902+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 22 192215088742243 55 84972 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:33.903+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 21 27979933591532 54 84972 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:33.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 23 154934968850942 51 84972 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:33.908+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 24 271667630317209 52 84972 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:33.910+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 25 168740244208559 61 84972 172.31.5.112:9000. HWT [59-61,61]. 2023-10-21T04:16:33.914+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 27 5715487545729 54 84972 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:33.914+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 26 120989102157727 59 84972 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:33.917+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 28 236953356346174 54 84972 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:33.928+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 29 130231725986856 49 84972 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:33.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 33 278426329668882 54 84972 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:33.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 34 51776324008509 60 84972 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:33.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 30 60545336618717 46 84972 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:33.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 35 24368176313355 50 84972 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:33.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 31 73432771436458 60 84972 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:33.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 32 67362620867705 67 84972 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:33.952+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 36 43594649874772 52 84972 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:33.966+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 40 153478150002150 68 84972 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:16:33.966+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 39 100659545510465 49 84972 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:16:33.966+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 37 83647976044922 59 84972 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:33.970+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 43 249919763471169 66 84972 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:33.972+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 44 166675095997741 57 84972 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:33.979+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 47 149022976193937 55 84972 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:33.981+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 48 19274425379955 55 84972 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:33.985+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:33.985+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:33.986+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 51 196777743741606 51 84972 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:33.987+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 38 181267443431019 53 84972 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:33.987+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 52 158076890855634 54 84972 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:33.988+05:30 [Info] switched currmeta from 240 -> 241 force true 2023-10-21T04:16:33.990+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 42 45479338375128 55 84972 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:33.992+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 49 184487117398647 55 84972 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:33.992+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 45 49796766757346 61 84972 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:33.992+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 41 116785907755771 55 84972 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:33.992+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 50 199728167534433 66 84972 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:33.993+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 46 124194574848641 54 84972 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:33.993+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 55 61798873988101 67 84972 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:33.998+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 54 24856128127173 60 84972 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:33.998+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 56 29071415952852 61 84972 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:33.998+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 53 138935694816848 61 84972 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:34.007+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 59 140942236111856 63 84972 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:34.007+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 58 103336286678362 46 84972 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:34.007+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 57 191719959781498 63 84972 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:34.008+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 60 56626109228904 46 84972 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:34.012+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 61 119871837460216 39 84972 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:16:34.015+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 63 252537041863196 73 84972 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:16:34.016+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 62 177105814270799 58 84972 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:34.016+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2 InstanceIds [9720186692804226822 2296526085582506469 11460826864060385936] 2023-10-21T04:16:34.017+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2 Cid 0 SessionId 84972 2023-10-21T04:16:34.018+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 84972 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 9720186692804226822 INDEX_STATE_CATCHUP 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 2296526085582506469 INDEX_STATE_CATCHUP 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 11460826864060385936 INDEX_STATE_CATCHUP 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 9720186692804226822 true 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 2296526085582506469 true 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 11460826864060385936 true 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 9720186692804226822 Stream: INIT_STREAM KeyspaceId: bucket_%2 Session: 84972 BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:16:34.018+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:16:34.019+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:16:34.019+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:16:34.019+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:16:34.019+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 84972 STREAM_ACTIVE 2023-10-21T04:16:34.019+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [0=19,1=19,2=19,3=18,4=19,5=19,6=25,7=19,8=25,9=26,10=19,11=23,12=22,13=22,14=27,15=26,16=20,17=12,18=21,19=17,20=20,21=19,22=23,23=19,24=15,25=22,26=22,27=29,28=18,29=19,30=20,31=22,32=26,33=22,34=26,35=18,36=21,37=20,38=25,39=19,40=27,41=25,42=26,43=26,44=18,45=18,46=25,47=20,48=21,49=27,50=25,51=20,52=19,53=25,54=24,55=28,56=21,57=24,58=20,59=26,60=19,61=14,62=23,63=21] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:34.019+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2 Stream: INIT_STREAM SessionId: 84972 2023-10-21T04:16:34.019+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2 84972 2023-10-21T04:16:34.019+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:34.019+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:34.020+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:34.020+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:34.020+05:30 [Info] send progress stats to clients 2023-10-21T04:16:34.020+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2 set minMergeTs 2023-10-21T04:16:34.020+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:16:34.021+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:4] INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:16:34.021+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:4] INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:16:34.021+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:4] INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:16:34.021+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 84972 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:16:34.026+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:16:34.032+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.936739ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.037+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 2023-10-21T04:16:34.037+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2 SessionId 84972 2023-10-21T04:16:34.037+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 84972 2023-10-21T04:16:34.037+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2 set minMergeTs 2023-10-21T04:16:34.037+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:16:34.037+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 9720186692804226822 false 2023-10-21T04:16:34.037+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 2296526085582506469 false 2023-10-21T04:16:34.037+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 11460826864060385936 false 2023-10-21T04:16:34.037+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2 2296526085582506469 2023-10-21T04:16:34.037+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2 11460826864060385936 2023-10-21T04:16:34.038+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2 9720186692804226822 2023-10-21T04:16:34.038+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2. 2023-10-21T04:16:34.038+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 84972 2023-10-21T04:16:34.038+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.038+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.041+05:30 [Info] switched currmeta from 241 -> 243 force true 2023-10-21T04:16:34.063+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 31.074386ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.069+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.069+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.085+05:30 [Info] switched currmeta from 243 -> 244 force true 2023-10-21T04:16:34.096+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 32.004087ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.108+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.109+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.110+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11790381330881939912:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:34.110+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11790381330881939912 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0026f7560 2023-10-21T04:16:34.111+05:30 [Info] handleUpdateIndexRState: Index instance 11790381330881939912 rstate moved to ACTIVE 2023-10-21T04:16:34.121+05:30 [Info] switched currmeta from 244 -> 246 force true 2023-10-21T04:16:34.148+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.149+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.152+05:30 [Info] switched currmeta from 246 -> 248 force true 2023-10-21T04:16:34.163+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.163+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.174+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 63.543452ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.174+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [16013578430151540990 2048006213723315482 1571318351652255745] belonging to collectionId: c, ttid: ShardTokena7:37:e9:2a:6a:83:12:d0. Initiating build 2023-10-21T04:16:34.188+05:30 [Info] switched currmeta from 248 -> 248 force true 2023-10-21T04:16:34.221+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.221+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.225+05:30 [Info] switched currmeta from 248 -> 250 force true 2023-10-21T04:16:34.241+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.243+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.254+05:30 [Info] switched currmeta from 250 -> 251 force true 2023-10-21T04:16:34.256+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [4440544141961093904 14213410630307008054 7494249569853685720] &{2} 2023-10-21T04:16:34.256+05:30 [Info] Indexer::handleBuildRecoveredIndexes [4440544141961093904 14213410630307008054 7494249569853685720] 2023-10-21T04:16:34.256+05:30 [Info] bulkUpdateRState: Index instance 4440544141961093904 rstate moved to PENDING 2023-10-21T04:16:34.256+05:30 [Info] bulkUpdateRState: Index instance 14213410630307008054 rstate moved to PENDING 2023-10-21T04:16:34.256+05:30 [Info] bulkUpdateRState: Index instance 7494249569853685720 rstate moved to PENDING 2023-10-21T04:16:34.256+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [4440544141961093904 14213410630307008054 7494249569853685720] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:34.257+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:34.257+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:34.257+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:34.273+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.274+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.280+05:30 [Info] switched currmeta from 251 -> 252 force true 2023-10-21T04:16:34.299+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.299+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.306+05:30 [Info] switched currmeta from 252 -> 252 force true 2023-10-21T04:16:34.315+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:16:34.317+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:34.317+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:34.317+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:34.317+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:34.317+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:34.317+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:16:34.317+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:34.317+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:34.317+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:34.317+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc004b32480 }] 2023-10-21T04:16:34.318+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:34.318+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:34.318+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:34.318+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:34.318+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:16:34.318+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId 65895 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:34.318+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:34.318+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:34.318+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:34.318+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:34.318+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:4] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c1:1]] 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 7494249569853685720 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:4] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c1:2]] 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4440544141961093904 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:4] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c1:3]] 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 14213410630307008054 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:34.318+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:16:34.318+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 65895 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:16:34.319+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc004b32480 } bucket_1:_default:c1:{0xc00042da40 }] 2023-10-21T04:16:34.319+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:34.320+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 63.737997ms 2023-10-21T04:16:34.320+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 64.222736ms 2023-10-21T04:16:34.320+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [4440544141961093904 14213410630307008054 7494249569853685720] 2023-10-21T04:16:34.320+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 143.219073ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.324+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4440544141961093904:true 7494249569853685720:true 14213410630307008054:true] 2023-10-21T04:16:34.324+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:16:34.329+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:16:34.341+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.075605ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.353+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.353+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.356+05:30 [Info] switched currmeta from 252 -> 253 force true 2023-10-21T04:16:34.364+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.766482ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.382+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.382+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.386+05:30 [Info] switched currmeta from 253 -> 254 force true 2023-10-21T04:16:34.388+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.878611ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.396+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.397+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.401+05:30 [Info] switched currmeta from 254 -> 255 force true 2023-10-21T04:16:34.561+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.562+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.580+05:30 [Info] switched currmeta from 255 -> 258 force true 2023-10-21T04:16:34.599+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.600+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.605+05:30 [Info] switched currmeta from 258 -> 259 force true 2023-10-21T04:16:34.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.629+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.637+05:30 [Info] switched currmeta from 259 -> 259 force true 2023-10-21T04:16:34.785+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 178840478062605 57 65895 172.31.5.112:9000. HWT [54-57,57]. 2023-10-21T04:16:34.788+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 148075163102199 54 65895 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:34.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 39552632604610 52 65895 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:34.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 191903564840858 63 65895 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:34.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 242327631330054 62 65895 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:34.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 215374859564337 35 65895 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:16:34.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 66881023717063 51 65895 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:34.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 225280996447309 55 65895 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:16:34.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 274410200699189 54 65895 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:34.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 183148377680315 46 65895 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:34.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 186517626268899 55 65895 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:34.815+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 28605411869539 58 65895 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:34.815+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 127185710837163 50 65895 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:34.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 226814108379280 72 65895 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:16:34.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 4429827396609 42 65895 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:16:34.817+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 50961953886801 45 65895 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:34.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 242578104468179 63 65895 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:34.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 257031483891250 58 65895 172.31.5.112:9000. HWT [55-58,58]. 2023-10-21T04:16:34.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 55290066231400 51 65895 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:34.823+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 210306088158507 48 65895 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:16:34.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 230239359684242 50 65895 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:34.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 209771427122473 48 65895 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:16:34.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 260036488774725 51 65895 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:34.830+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 216462437971699 57 65895 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:34.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 160961818008256 59 65895 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:34.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 280874278672777 71 65895 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:16:34.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 219132834127088 52 65895 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:34.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 184056281482149 49 65895 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:34.844+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 45957426777107 57 65895 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:34.844+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 102578907139247 58 65895 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:34.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 99433130332778 58 65895 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:34.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 82469662608686 60 65895 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:34.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 217381278987664 67 65895 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:34.852+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 136958029391999 65 65895 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:16:34.854+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 28417669987133 58 65895 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:34.857+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 70213628416850 47 65895 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:16:34.860+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 253845164956592 58 65895 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:34.860+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 253494845579900 54 65895 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:34.865+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 222555662236724 67 65895 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:34.865+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 188107826286720 60 65895 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:34.867+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 5915864305121 60 65895 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:34.872+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 259381390638966 58 65895 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:34.872+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 252931432434966 46 65895 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:34.876+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 146267938201125 55 65895 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:34.877+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 158545118983293 67 65895 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:34.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 12301831552869 66 65895 172.31.5.112:9000. HWT [63-66,66]. 2023-10-21T04:16:34.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 227378777358533 59 65895 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:16:34.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 242994859770444 54 65895 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:34.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 271074273345339 53 65895 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:34.887+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 215467668600630 34 65895 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:16:34.894+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 207367350302991 64 65895 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:16:34.894+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 200436537417141 54 65895 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:34.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 204867593700407 56 65895 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:34.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 188718367400378 56 65895 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:34.904+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 129411666769864 55 65895 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:34.904+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 170056388266420 70 65895 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:16:34.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 279362078165491 55 65895 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:34.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 264835102027079 53 65895 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:16:34.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 125009072873642 53 65895 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:34.906+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 16044440367455 53 65895 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:34.919+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c1 InstanceIds [9720186692804226822 2296526085582506469 11460826864060385936 7494249569853685720 4440544141961093904 14213410630307008054] 2023-10-21T04:16:34.919+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 65895 2023-10-21T04:16:34.919+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65895 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 7494249569853685720 INDEX_STATE_CATCHUP 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 4440544141961093904 INDEX_STATE_CATCHUP 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 14213410630307008054 INDEX_STATE_CATCHUP 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 7494249569853685720 true 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 4440544141961093904 true 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 14213410630307008054 true 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 7494249569853685720 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 65895 BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:16:34.919+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:16:34.920+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:16:34.920+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:16:34.920+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65895 STREAM_ACTIVE 2023-10-21T04:16:34.920+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 65895 2023-10-21T04:16:34.920+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 65895 2023-10-21T04:16:34.920+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:34.921+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:34.921+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:34.921+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:34.922+05:30 [Info] send progress stats to clients 2023-10-21T04:16:34.922+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:16:34.922+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:16:34.922+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:5] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c1:3]] 2023-10-21T04:16:34.922+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:6] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c1:3]] 2023-10-21T04:16:34.922+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:7] INIT_STREAM:map[bucket_%2:3 bucket_1:_default:c1:3]] 2023-10-21T04:16:34.922+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 65895 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:16:34.925+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:16:34.928+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:34.930+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 75566139929862 56 65895 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:34.930+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 106932309485550 53 65895 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:34.931+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:16:34.935+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 10527297481171 57 65895 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:34.938+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.953148ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.961+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.343798ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.969+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.312983ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:34.972+05:30 [Info] switched currmeta from 259 -> 261 force true 2023-10-21T04:16:34.990+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:34.990+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:34.995+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 133748675726909 56 65895 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:34.996+05:30 [Info] switched currmeta from 261 -> 262 force true 2023-10-21T04:16:35.018+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.018+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.020+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:16:35.022+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:16:35.022+05:30 [Info] switched currmeta from 262 -> 262 force true 2023-10-21T04:16:35.027+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 2023-10-21T04:16:35.028+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65895 2023-10-21T04:16:35.028+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65895 2023-10-21T04:16:35.029+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:16:35.028+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 9720186692804226822 Stream: INIT_STREAM KeyspaceId: bucket_%2 SessionId: 84972, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:35.028+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.031+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:16:35.032+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 9720186692804226822 INDEX_STATE_ACTIVE 2023-10-21T04:16:35.033+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 2296526085582506469 INDEX_STATE_ACTIVE 2023-10-21T04:16:35.033+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 11460826864060385936 INDEX_STATE_ACTIVE 2023-10-21T04:16:35.032+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.034+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.034+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0] 6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.034+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0] 6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.034+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2 State Changed to INACTIVE 2023-10-21T04:16:35.040+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0] 6589365266016780433:[0] 6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.041+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0] 6589365266016780433:[0] 6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.049+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0] 6589365266016780433:[0] 6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.057+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:16:35.057+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:16:35.057+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:35.057+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0] 6589365266016780433:[0] 6919182200795062103:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:35.059+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 7494249569853685720 false 2023-10-21T04:16:35.060+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 4440544141961093904 false 2023-10-21T04:16:35.060+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 14213410630307008054 false 2023-10-21T04:16:35.060+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 7494249569853685720 2023-10-21T04:16:35.060+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 4440544141961093904 2023-10-21T04:16:35.061+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 14213410630307008054 2023-10-21T04:16:35.061+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:16:35.061+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 65895 2023-10-21T04:16:35.062+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2 Stream: INIT_STREAM SessionId: 84972 2023-10-21T04:16:35.063+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.063+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.066+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:35.067+05:30 [Info] switched currmeta from 262 -> 263 force true 2023-10-21T04:16:35.069+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:35.069+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:35.071+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 84972 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:35.071+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 9720186692804226822 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:16:35.071+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 2296526085582506469 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:16:35.071+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 11460826864060385936 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:16:35.071+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:16:35.072+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:16:35.072+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:35.072+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 84972 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:35.074+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_default:c1:{0xc00042da40 }] 2023-10-21T04:16:35.074+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:35.074+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:35.075+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:35.076+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:35.076+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:35.076+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:35.077+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:35.077+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:35.077+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:16:35.078+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:35.086+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2 Stream: INIT_STREAM SessionId 84972 2023-10-21T04:16:35.089+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.053057ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.087+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2] 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 5 3874519919278 54 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 6 261493429196201 61 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 7 170288557775498 52 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 0 263268055970459 52 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 1 176639880977228 61 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 2 35403626585217 50 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 3 215749515705641 45 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.097+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 4 159113185399719 58 2023-10-21T04:16:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 12 116104661063478 55 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 10 206533515052028 50 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 13 102350060311343 53 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 14 140236896721046 60 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 15 162158720181435 67 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 8 264975169139761 56 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 9 236398693961229 62 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 16 200752780087280 54 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.098+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 11 234758907181738 56 2023-10-21T04:16:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.100+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 29 130231725986856 49 2023-10-21T04:16:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.100+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 19 208836052671833 45 2023-10-21T04:16:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.100+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 22 192215088742243 55 2023-10-21T04:16:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.100+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 24 271667630317209 52 2023-10-21T04:16:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.100+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 26 120989102157727 59 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 31 73432771436458 60 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 32 67362620867705 67 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 17 98820809009123 44 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 20 92956357968546 50 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 25 168740244208559 61 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 18 207134287529256 46 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 27 5715487545729 54 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.101+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 30 60545336618717 46 2023-10-21T04:16:35.101+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.102+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 28 236953356346174 54 2023-10-21T04:16:35.102+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.102+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 23 154934968850942 51 2023-10-21T04:16:35.102+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.102+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 21 27979933591532 54 2023-10-21T04:16:35.104+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.104+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 52 158076890855634 54 2023-10-21T04:16:35.104+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.104+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 60 56626109228904 46 2023-10-21T04:16:35.104+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 55 61798873988101 67 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 56 29071415952852 61 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 34 51776324008509 60 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 38 181267443431019 53 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 42 45479338375128 55 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 33 278426329668882 54 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 43 249919763471169 66 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 45 49796766757346 61 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 48 19274425379955 55 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 50 199728167534433 66 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 49 184487117398647 55 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.105+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 51 196777743741606 51 2023-10-21T04:16:35.105+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 54 24856128127173 60 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 57 191719959781498 63 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 36 43594649874772 52 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 40 153478150002150 68 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 47 149022976193937 55 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 41 116785907755771 55 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 46 124194574848641 54 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 59 140942236111856 63 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 35 24368176313355 50 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 58 103336286678362 46 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.187348ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 53 138935694816848 61 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 37 83647976044922 59 2023-10-21T04:16:35.106+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.106+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 39 100659545510465 49 2023-10-21T04:16:35.107+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.107+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 44 166675095997741 57 2023-10-21T04:16:35.107+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.107+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.107+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.109+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 62 177105814270799 58 2023-10-21T04:16:35.109+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.109+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 61 119871837460216 39 2023-10-21T04:16:35.109+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.109+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2 63 252537041863196 73 2023-10-21T04:16:35.109+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:35.112+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2] 2023-10-21T04:16:35.113+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2 SessionId 84972 2023-10-21T04:16:35.115+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 84972 2023-10-21T04:16:35.113+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.385503ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.116+05:30 [Info] switched currmeta from 263 -> 266 force true 2023-10-21T04:16:35.117+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:16:35.158+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.159+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.161+05:30 [Info] switched currmeta from 266 -> 266 force true 2023-10-21T04:16:35.185+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.185+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.189+05:30 [Info] switched currmeta from 266 -> 267 force true 2023-10-21T04:16:35.251+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.251+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.257+05:30 [Info] switched currmeta from 267 -> 268 force true 2023-10-21T04:16:35.462+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0001170e0 2023-10-21T04:16:35.463+05:30 [Info] handleUpdateIndexRState: Index instance 9720186692804226822 rstate moved to ACTIVE 2023-10-21T04:16:35.473+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.902673ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.473+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00df6b5c0 2023-10-21T04:16:35.473+05:30 [Info] handleUpdateIndexRState: Index instance 2296526085582506469 rstate moved to ACTIVE 2023-10-21T04:16:35.473+05:30 [Info] Indexer::handleMergePartition Source 4087317880204280526 Target 11460826864060385936 2023-10-21T04:16:35.473+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007758660 2023-10-21T04:16:35.473+05:30 [Info] MergePartition: sent async request to update index instance 11460826864060385936 rstate moved to ACTIVE 2023-10-21T04:16:35.486+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.476651ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.487+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 1265729768899070816. Retrying... 2023-10-21T04:16:35.487+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 4992762551574016313. Retrying... 2023-10-21T04:16:35.487+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 18415077564616412601. Retrying... 2023-10-21T04:16:35.487+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[2296526085582506469:true 9720186692804226822:true 11460826864060385936:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:35.491+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.491+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.495+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.682176ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.495+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 4087317880204280526, realInst: 11460826864060385936 2023-10-21T04:16:35.497+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 249887442666643345, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:35.498+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 249887442666643345, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:35.498+05:30 [Info] switched currmeta from 268 -> 271 force true 2023-10-21T04:16:35.511+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.511+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.514+05:30 [Info] switched currmeta from 271 -> 271 force true 2023-10-21T04:16:35.521+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14975131556821987100,realInstId: 0, indexDefn DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:35.522+05:30 [Info] Indexer::handleRecoverIndex InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:35.522+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 353.229µs 2023-10-21T04:16:35.522+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 397.402µs 2023-10-21T04:16:35.522+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14975131556821987100 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:35.522+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14975131556821987100 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:35.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:35.523+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1056168 memSz 2332 2023-10-21T04:16:35.523+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 1000 metaSz: 2512 rp [1] sn: 2 count: 1000 metaSz: 2578 2023-10-21T04:16:35.523+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14975131556821987100, indexDefn DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:35.523+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 25.505649ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:35.523+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1130 2023-10-21T04:16:35.523+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 1 count: 1000 metaSz: 2512 rp [1] sn: 2 count: 1000 metaSz: 2578 2023-10-21T04:16:35.523+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 202223 recovery UsedSpace 25145 2023-10-21T04:16:35.524+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 211725 recovery UsedSpace 22875 2023-10-21T04:16:35.531+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.531+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.534+05:30 [Info] switched currmeta from 271 -> 272 force true 2023-10-21T04:16:35.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 154 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex 2023-10-21T04:16:35.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:35.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:35.535+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex 2023-10-21T04:16:35.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex started 2023-10-21T04:16:35.535+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:35.535+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:35.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 138 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex 2023-10-21T04:16:35.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:35.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:35.538+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex 2023-10-21T04:16:35.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex started 2023-10-21T04:16:35.538+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14975131556821987100 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:35.538+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:35.538+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:35.554+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14975131556821987100 PartitionId 0 Warmup took 31.556877ms 2023-10-21T04:16:35.554+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14975131556821987100 partitionId 0 WriterThreads 4 2023-10-21T04:16:35.554+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14975131556821987100 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index Index: 14975131556821987100 Partition: 0 2023-10-21T04:16:35.555+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14975131556821987100, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:35.555+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14975131556821987100, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:35.555+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:35.555+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14975131556821987100 2023-10-21T04:16:35.555+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:35.555+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:35.556+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:35.556+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:35.556+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:35.556+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14975131556821987100 Partitions [0] 2023-10-21T04:16:35.557+05:30 [Info] StorageMgr::openSnapshot IndexInst:14975131556821987100 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:35.571+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.119522ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.572+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 87.275µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:35.579+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.579+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.582+05:30 [Info] switched currmeta from 272 -> 273 force true 2023-10-21T04:16:35.677+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.677+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:35.680+05:30 [Info] switched currmeta from 273 -> 274 force true 2023-10-21T04:16:35.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 9720186692804226822:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 9720186692804226822:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 9720186692804226822:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 9720186692804226822:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:35.925+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 4440544141961093904 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 65895, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 7494249569853685720 INDEX_STATE_ACTIVE 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 4440544141961093904 INDEX_STATE_ACTIVE 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 14213410630307008054 INDEX_STATE_ACTIVE 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:16:35.925+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:35.925+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:35.926+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 65895 2023-10-21T04:16:35.926+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:35.926+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:35.926+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:35.927+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65895 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:35.927+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 7494249569853685720 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:16:35.927+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4440544141961093904 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:16:35.927+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 14213410630307008054 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:16:35.927+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:16:35.927+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:35.927+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:35.927+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65895 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:35.927+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:16:35.928+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:35.928+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:35.928+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:16:35.928+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:48380: use of closed network connection 2023-10-21T04:16:35.928+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:48380" exit: read tcp4 127.0.0.1:9121->127.0.0.1:48380: use of closed network connection 2023-10-21T04:16:35.928+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48380" closed ! 2023-10-21T04:16:35.928+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:16:35.928+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:35.928+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:35.929+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:35.929+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:35.929+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:16:35.930+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:35.931+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 65895 2023-10-21T04:16:35.931+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:16:35.950+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:16:35.950+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:16:35.950+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:16:35.950+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65895 2023-10-21T04:16:35.950+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65895 2023-10-21T04:16:35.950+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:16:35.953+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.742977ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.966+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.141286ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.975+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.140016ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:35.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9593557024085695, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:16:35.996+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:35.996+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.005+05:30 [Info] switched currmeta from 274 -> 277 force true 2023-10-21T04:16:36.025+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.025+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.034+05:30 [Info] switched currmeta from 277 -> 277 force true 2023-10-21T04:16:36.064+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.064+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.067+05:30 [Info] switched currmeta from 277 -> 278 force true 2023-10-21T04:16:36.114+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.114+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.121+05:30 [Info] switched currmeta from 278 -> 279 force true 2023-10-21T04:16:36.138+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.138+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.145+05:30 [Info] switched currmeta from 279 -> 280 force true 2023-10-21T04:16:36.165+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.165+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.183+05:30 [Info] switched currmeta from 280 -> 281 force true 2023-10-21T04:16:36.269+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.269+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.273+05:30 [Info] switched currmeta from 281 -> 282 force true 2023-10-21T04:16:36.305+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.305+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.321+05:30 [Info] switched currmeta from 282 -> 283 force true 2023-10-21T04:16:36.354+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00957a240 2023-10-21T04:16:36.355+05:30 [Info] handleUpdateIndexRState: Index instance 14213410630307008054 rstate moved to ACTIVE 2023-10-21T04:16:36.355+05:30 [Info] Indexer::handleMergePartition Source 7691103359834243171 Target 4440544141961093904 2023-10-21T04:16:36.355+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00efce1e0 2023-10-21T04:16:36.355+05:30 [Info] MergePartition: sent async request to update index instance 4440544141961093904 rstate moved to ACTIVE 2023-10-21T04:16:36.362+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.289063ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:36.364+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009c78300 2023-10-21T04:16:36.364+05:30 [Info] handleUpdateIndexRState: Index instance 7494249569853685720 rstate moved to ACTIVE 2023-10-21T04:16:36.372+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.067798ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:36.372+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 7691103359834243171, realInst: 4440544141961093904 2023-10-21T04:16:36.382+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.383+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.390+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.380112ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:36.391+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 7582778950442222621. Retrying... 2023-10-21T04:16:36.393+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 12265503634969032942. Retrying... 2023-10-21T04:16:36.393+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[4440544141961093904:true 7494249569853685720:true 14213410630307008054:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:36.396+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 14026476026078401783, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:36.396+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 14026476026078401783, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:36.411+05:30 [Info] switched currmeta from 283 -> 287 force true 2023-10-21T04:16:36.432+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.432+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.440+05:30 [Info] switched currmeta from 287 -> 287 force true 2023-10-21T04:16:36.469+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 5744899830895973442,realInstId: 0, indexDefn DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:36.470+05:30 [Info] Indexer::handleRecoverIndex InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:36.470+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 344.4µs 2023-10-21T04:16:36.470+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 390.522µs 2023-10-21T04:16:36.470+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.470+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 5744899830895973442, indexDefn DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:36.470+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 74.44304ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:36.470+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.471+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:36.471+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 128 metaSz: 2578 rp [1] sn: 1080007 count: 128 metaSz: 2578 2023-10-21T04:16:36.471+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 110014 recovery UsedSpace 137142 2023-10-21T04:16:36.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.472+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:36.472+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 128 metaSz: 2578 rp [1] sn: 1080007 count: 128 metaSz: 2578 2023-10-21T04:16:36.472+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 123700 recovery UsedSpace 136528 2023-10-21T04:16:36.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex 2023-10-21T04:16:36.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.490+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex 2023-10-21T04:16:36.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex started 2023-10-21T04:16:36.490+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.490+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex 2023-10-21T04:16:36.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.495+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex 2023-10-21T04:16:36.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex started 2023-10-21T04:16:36.494+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.497+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.496+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:36.496+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.496+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.502+05:30 [Info] switched currmeta from 287 -> 288 force true 2023-10-21T04:16:36.530+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14975131556821987100:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:36.531+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 15649171661394925140, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:36.532+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 15649171661394925140, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:36.561+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10477796393418268202,realInstId: 0, indexDefn DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:36.561+05:30 [Info] Indexer::handleRecoverIndex InstId: 10477796393418268202 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:36.561+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 469.537µs 2023-10-21T04:16:36.561+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 516.686µs 2023-10-21T04:16:36.562+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:36.562+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:36.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.562+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.563+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:36.563+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10477796393418268202, indexDefn DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:36.563+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 32.160428ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:36.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.564+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.564+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:36.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 133 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:36.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.580+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:36.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex started 2023-10-21T04:16:36.581+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.581+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.582+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 8 Warmup took 111.442186ms 2023-10-21T04:16:36.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 148 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:36.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.584+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:36.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex started 2023-10-21T04:16:36.584+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:36.584+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:36.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:16:36.584+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:36.584+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:16:36.584+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:16:36.585+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:36.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex stopped 2023-10-21T04:16:36.585+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.585+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.585+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.585+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.585+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.593+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:16:36.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex closed 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:16:36.594+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:16:36.594+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:36.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex stopped 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.594+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.596+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.596+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 8 WriterThreads 1 2023-10-21T04:16:36.601+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index Index: 5744899830895973442 Partition: 8 2023-10-21T04:16:36.601+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.602+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.603+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 258 2023-10-21T04:16:36.603+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 13 numRPs: 4 rp [0] sn: 2 count: 135 metaSz: 2577 rp [1] sn: 360005 count: 135 metaSz: 2578 rp [2] sn: 720006 count: 135 metaSz: 2578 rp [3] sn: 1080007 count: 135 metaSz: 2578 2023-10-21T04:16:36.605+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 140707 recovery UsedSpace 120818 2023-10-21T04:16:36.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.605+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:36.605+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 135 metaSz: 2578 rp [1] sn: 1080007 count: 135 metaSz: 2578 2023-10-21T04:16:36.606+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 124891 recovery UsedSpace 132432 2023-10-21T04:16:36.610+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:16:36.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex closed 2023-10-21T04:16:36.611+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:36.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex ... 2023-10-21T04:16:36.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 133 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex ... 2023-10-21T04:16:36.612+05:30 [Info] switched currmeta from 288 -> 290 force true 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.617+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex started 2023-10-21T04:16:36.617+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.617+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex sucessfully destroyed 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex ... 2023-10-21T04:16:36.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 148 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex ... 2023-10-21T04:16:36.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index destroyed 2023-10-21T04:16:36.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex 2023-10-21T04:16:36.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.620+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex 2023-10-21T04:16:36.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex started 2023-10-21T04:16:36.621+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:36.621+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.621+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.621+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.626+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.626+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.631+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.631+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.634+05:30 [Info] switched currmeta from 290 -> 290 force true 2023-10-21T04:16:36.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 157 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:36.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.634+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:36.634+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:16:36.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex started 2023-10-21T04:16:36.634+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.634+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 175 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:36.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.637+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:36.637+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:16:36.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex started 2023-10-21T04:16:36.637+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 5 Warmup took 74.94603ms 2023-10-21T04:16:36.637+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 5 WriterThreads 1 2023-10-21T04:16:36.637+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 5 Stopped Billing WUs 2023-10-21T04:16:36.637+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index Index: 10477796393418268202 Partition: 5 2023-10-21T04:16:36.638+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:36.638+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.638+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.638+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:36.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.638+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.639+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:36.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.639+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.639+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 1 Warmup took 37.30176ms 2023-10-21T04:16:36.640+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:36.640+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 1 WriterThreads 1 2023-10-21T04:16:36.640+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index Index: 5744899830895973442 Partition: 1 2023-10-21T04:16:36.640+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.640+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.640+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:36.640+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 13 numRPs: 4 rp [0] sn: 2 count: 107 metaSz: 2577 rp [1] sn: 360005 count: 107 metaSz: 2578 rp [2] sn: 720006 count: 107 metaSz: 2578 rp [3] sn: 1080007 count: 107 metaSz: 2578 2023-10-21T04:16:36.641+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 132721 recovery UsedSpace 120100 2023-10-21T04:16:36.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.641+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:36.641+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 107 metaSz: 2578 rp [1] sn: 1080007 count: 107 metaSz: 2578 2023-10-21T04:16:36.641+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 120700 recovery UsedSpace 132432 2023-10-21T04:16:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 134 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:36.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.647+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:36.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex started 2023-10-21T04:16:36.647+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 149 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:36.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.648+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:36.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex started 2023-10-21T04:16:36.648+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:36.649+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:36.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.647+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:16:36.650+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:36.650+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:16:36.650+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:16:36.651+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:36.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex 2023-10-21T04:16:36.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.650+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex stopped 2023-10-21T04:16:36.651+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.651+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.651+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.651+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex 2023-10-21T04:16:36.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.652+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex 2023-10-21T04:16:36.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex started 2023-10-21T04:16:36.652+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.652+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex 2023-10-21T04:16:36.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex started 2023-10-21T04:16:36.653+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:36.653+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.652+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.653+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:16:36.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex closed 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:16:36.671+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:16:36.671+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:36.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex stopped 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.671+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.689+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:16:36.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex closed 2023-10-21T04:16:36.689+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:36.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex ... 2023-10-21T04:16:36.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 134 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex ... 2023-10-21T04:16:36.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex sucessfully destroyed 2023-10-21T04:16:36.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex ... 2023-10-21T04:16:36.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 149 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex ... 2023-10-21T04:16:36.695+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 2 Warmup took 54.502105ms 2023-10-21T04:16:36.695+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 2 WriterThreads 1 2023-10-21T04:16:36.695+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index Index: 5744899830895973442 Partition: 2 2023-10-21T04:16:36.695+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.695+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index destroyed 2023-10-21T04:16:36.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.699+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:16:36.699+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 13 numRPs: 4 rp [0] sn: 2 count: 138 metaSz: 2577 rp [1] sn: 360005 count: 138 metaSz: 2578 rp [2] sn: 720006 count: 138 metaSz: 2578 rp [3] sn: 1080007 count: 138 metaSz: 2578 2023-10-21T04:16:36.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.705+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.705+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.713+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.713+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 176 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:36.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.724+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:36.724+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:16:36.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex started 2023-10-21T04:16:36.724+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.724+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.726+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:36.726+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 14 numRPs: 3 rp [0] sn: 360005 count: 138 metaSz: 2578 rp [1] sn: 720006 count: 138 metaSz: 2578 rp [2] sn: 1080007 count: 138 metaSz: 2578 2023-10-21T04:16:36.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 158 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:36.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.728+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:36.728+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:16:36.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex started 2023-10-21T04:16:36.728+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 6 Warmup took 90.199501ms 2023-10-21T04:16:36.728+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 6 WriterThreads 1 2023-10-21T04:16:36.728+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 6 Stopped Billing WUs 2023-10-21T04:16:36.728+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index Index: 10477796393418268202 Partition: 6 2023-10-21T04:16:36.728+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:36.729+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:36.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.729+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.729+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.729+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.729+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.735+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.735+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.736+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 141550 recovery UsedSpace 120818 2023-10-21T04:16:36.741+05:30 [Info] switched currmeta from 290 -> 292 force true 2023-10-21T04:16:36.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex 2023-10-21T04:16:36.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.742+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex 2023-10-21T04:16:36.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex started 2023-10-21T04:16:36.743+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.744+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.746+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 125358 recovery UsedSpace 124218 2023-10-21T04:16:36.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex 2023-10-21T04:16:36.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.752+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex 2023-10-21T04:16:36.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex started 2023-10-21T04:16:36.753+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:36.758+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:36.775+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.775+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.775+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.776+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.776+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:36.798+05:30 [Info] switched currmeta from 292 -> 292 force true 2023-10-21T04:16:36.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 150 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:36.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.804+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:36.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex started 2023-10-21T04:16:36.804+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.804+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 135 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:36.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:36.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex started 2023-10-21T04:16:36.808+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:36.808+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:16:36.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:16:36.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:36.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex stopped 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.808+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.809+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.809+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.826+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:16:36.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex closed 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:16:36.829+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:16:36.829+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:36.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex stopped 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.829+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.830+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 3 Warmup took 134.619018ms 2023-10-21T04:16:36.831+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 3 WriterThreads 1 2023-10-21T04:16:36.831+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index Index: 5744899830895973442 Partition: 3 2023-10-21T04:16:36.831+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.831+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.831+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 238 2023-10-21T04:16:36.831+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 13 numRPs: 4 rp [0] sn: 2 count: 137 metaSz: 2577 rp [1] sn: 360005 count: 137 metaSz: 2578 rp [2] sn: 720006 count: 137 metaSz: 2578 rp [3] sn: 1080007 count: 137 metaSz: 2578 2023-10-21T04:16:36.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.832+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 141100 recovery UsedSpace 120798 2023-10-21T04:16:36.832+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:36.832+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 137 metaSz: 2578 rp [1] sn: 1080007 count: 137 metaSz: 2578 2023-10-21T04:16:36.832+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 125216 recovery UsedSpace 132432 2023-10-21T04:16:36.836+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:36.836+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:36.839+05:30 [Info] switched currmeta from 292 -> 293 force true 2023-10-21T04:16:36.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex 2023-10-21T04:16:36.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.856+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex 2023-10-21T04:16:36.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex started 2023-10-21T04:16:36.856+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.856+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.856+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:16:36.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex closed 2023-10-21T04:16:36.857+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:36.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex ... 2023-10-21T04:16:36.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 135 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex ... 2023-10-21T04:16:36.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex 2023-10-21T04:16:36.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.858+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex 2023-10-21T04:16:36.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex started 2023-10-21T04:16:36.858+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:36.859+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.859+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex sucessfully destroyed 2023-10-21T04:16:36.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex ... 2023-10-21T04:16:36.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 150 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex ... 2023-10-21T04:16:36.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index destroyed 2023-10-21T04:16:36.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.874+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.874+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.876+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 4 Warmup took 45.497974ms 2023-10-21T04:16:36.877+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 4 WriterThreads 1 2023-10-21T04:16:36.877+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index Index: 5744899830895973442 Partition: 4 2023-10-21T04:16:36.877+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.877+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.877+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:36.877+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 131 metaSz: 2578 rp [1] sn: 1080007 count: 131 metaSz: 2578 2023-10-21T04:16:36.878+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 124308 recovery UsedSpace 132432 2023-10-21T04:16:36.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.878+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 240 2023-10-21T04:16:36.878+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 13 numRPs: 4 rp [0] sn: 2 count: 131 metaSz: 2577 rp [1] sn: 360005 count: 131 metaSz: 2578 rp [2] sn: 720006 count: 131 metaSz: 2578 rp [3] sn: 1080007 count: 131 metaSz: 2578 2023-10-21T04:16:36.879+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 140028 recovery UsedSpace 120808 2023-10-21T04:16:36.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.880+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.880+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 159 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:36.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.887+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:36.887+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:16:36.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex started 2023-10-21T04:16:36.887+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.887+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex 2023-10-21T04:16:36.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.890+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex 2023-10-21T04:16:36.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex started 2023-10-21T04:16:36.890+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.891+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex 2023-10-21T04:16:36.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.891+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex 2023-10-21T04:16:36.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex started 2023-10-21T04:16:36.891+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:36.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 177 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:36.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.893+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:36.893+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:16:36.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex started 2023-10-21T04:16:36.893+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 7 Warmup took 164.583117ms 2023-10-21T04:16:36.893+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 7 WriterThreads 1 2023-10-21T04:16:36.894+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 7 Stopped Billing WUs 2023-10-21T04:16:36.894+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index Index: 10477796393418268202 Partition: 7 2023-10-21T04:16:36.894+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:36.894+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:36.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.894+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.895+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:36.895+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.895+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.895+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.896+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 151 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.905+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex started 2023-10-21T04:16:36.905+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.905+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 136 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.905+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex started 2023-10-21T04:16:36.905+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:36.905+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:36.905+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.905+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.905+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:16:36.905+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:36.905+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:16:36.905+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:16:36.905+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:36.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex stopped 2023-10-21T04:16:36.906+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.906+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.906+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.906+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.906+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.906+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.925+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 5 Warmup took 47.737513ms 2023-10-21T04:16:36.925+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 5 WriterThreads 1 2023-10-21T04:16:36.925+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index Index: 5744899830895973442 Partition: 5 2023-10-21T04:16:36.925+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.925+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:16:36.929+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.929+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:36.929+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 13 numRPs: 4 rp [0] sn: 2 count: 115 metaSz: 2577 rp [1] sn: 360005 count: 115 metaSz: 2578 rp [2] sn: 720006 count: 115 metaSz: 2578 rp [3] sn: 1080007 count: 115 metaSz: 2578 2023-10-21T04:16:36.930+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 133742 recovery UsedSpace 120100 2023-10-21T04:16:36.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.930+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:36.930+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 115 metaSz: 2578 rp [1] sn: 1080007 count: 115 metaSz: 2578 2023-10-21T04:16:36.930+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 121666 recovery UsedSpace 132432 2023-10-21T04:16:36.931+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:16:36.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex closed 2023-10-21T04:16:36.931+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:36.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:16:36.932+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:16:36.932+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:36.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex stopped 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:36.932+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:36.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex 2023-10-21T04:16:36.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.943+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex 2023-10-21T04:16:36.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex started 2023-10-21T04:16:36.943+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.943+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.945+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:36.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:16:36.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex closed 2023-10-21T04:16:36.945+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:36.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex ... 2023-10-21T04:16:36.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 136 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex ... 2023-10-21T04:16:36.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex 2023-10-21T04:16:36.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.950+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex 2023-10-21T04:16:36.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex started 2023-10-21T04:16:36.950+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.950+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:36.950+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex sucessfully destroyed 2023-10-21T04:16:36.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex ... 2023-10-21T04:16:36.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 151 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex ... 2023-10-21T04:16:36.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index destroyed 2023-10-21T04:16:36.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.970+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.971+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.973+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 6 Warmup took 44.075849ms 2023-10-21T04:16:36.975+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 6 WriterThreads 1 2023-10-21T04:16:36.975+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index Index: 5744899830895973442 Partition: 6 2023-10-21T04:16:36.975+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5744899830895973442 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:36.975+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5744899830895973442 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:36.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:36.976+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:36.976+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 13 numRPs: 4 rp [0] sn: 2 count: 109 metaSz: 2577 rp [1] sn: 360005 count: 109 metaSz: 2578 rp [2] sn: 720006 count: 109 metaSz: 2578 rp [3] sn: 1080007 count: 109 metaSz: 2578 2023-10-21T04:16:36.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:36.977+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:36.977+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 109 metaSz: 2578 rp [1] sn: 1080007 count: 109 metaSz: 2578 2023-10-21T04:16:36.978+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 120875 recovery UsedSpace 132432 2023-10-21T04:16:36.979+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 132700 recovery UsedSpace 120100 2023-10-21T04:16:36.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:36.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.980+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:36.980+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:36.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 178 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:36.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:36.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.985+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:36.986+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:16:36.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex started 2023-10-21T04:16:36.986+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex 2023-10-21T04:16:36.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:36.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 160 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:36.987+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:36.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.989+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:36.989+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:16:36.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex started 2023-10-21T04:16:36.989+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 8 Warmup took 94.828707ms 2023-10-21T04:16:36.989+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 8 WriterThreads 1 2023-10-21T04:16:36.989+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 8 Stopped Billing WUs 2023-10-21T04:16:36.989+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index Index: 10477796393418268202 Partition: 8 2023-10-21T04:16:36.989+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:36.989+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:36.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:36.989+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.990+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:36.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:36.990+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:36.991+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:36.988+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex 2023-10-21T04:16:36.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9838188326377354, throttleDelayMs (new, old, change): (4, 1, 3) 2023-10-21T04:16:36.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex started 2023-10-21T04:16:36.994+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:36.995+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex 2023-10-21T04:16:36.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:36.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:36.998+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex 2023-10-21T04:16:36.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex started 2023-10-21T04:16:36.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 144 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:36.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 129 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:36.999+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:36.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:36.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:36.999+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:37.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex started 2023-10-21T04:16:37.000+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.000+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:36.999+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:36.999+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.000+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:37.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex started 2023-10-21T04:16:37.001+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:37.001+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:37.002+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.001+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.001+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.004+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:16:37.005+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:37.005+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:16:37.005+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:16:37.005+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:37.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex stopped 2023-10-21T04:16:37.005+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.005+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.005+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.005+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.005+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.009+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:16:37.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex closed 2023-10-21T04:16:37.012+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:37.012+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.013+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.013+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:16:37.013+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:37.013+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:16:37.013+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:16:37.013+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:37.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex stopped 2023-10-21T04:16:37.014+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.014+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.014+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.014+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.032+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:16:37.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex closed 2023-10-21T04:16:37.033+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:37.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex ... 2023-10-21T04:16:37.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 129 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex ... 2023-10-21T04:16:37.034+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5744899830895973442 PartitionId 7 Warmup took 58.928067ms 2023-10-21T04:16:37.035+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5744899830895973442 partitionId 7 WriterThreads 1 2023-10-21T04:16:37.035+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5744899830895973442 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index Index: 5744899830895973442 Partition: 7 2023-10-21T04:16:37.036+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 5744899830895973442, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:37.036+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 5744899830895973442, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:37.036+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:37.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.038+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex sucessfully destroyed 2023-10-21T04:16:37.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex ... 2023-10-21T04:16:37.039+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 5744899830895973442 2023-10-21T04:16:37.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 144 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex ... 2023-10-21T04:16:37.041+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:37.042+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:37.042+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:37.043+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:37.043+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:37.043+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 5744899830895973442 Partitions [4 5 6 7 8 1 2 3] 2023-10-21T04:16:37.044+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:5 Attempting to open snapshot (SnapshotInfo: count:131 committed:false) 2023-10-21T04:16:37.045+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:6 Attempting to open snapshot (SnapshotInfo: count:115 committed:false) 2023-10-21T04:16:37.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index destroyed 2023-10-21T04:16:37.046+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:7 Attempting to open snapshot (SnapshotInfo: count:109 committed:false) 2023-10-21T04:16:37.050+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:8 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:16:37.051+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.185561ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:37.051+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 130.361µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:37.057+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:1 Attempting to open snapshot (SnapshotInfo: count:135 committed:false) 2023-10-21T04:16:37.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.061+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.062+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.062+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:2 Attempting to open snapshot (SnapshotInfo: count:107 committed:false) 2023-10-21T04:16:37.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.065+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.065+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 179 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:37.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.071+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:37.071+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:16:37.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex started 2023-10-21T04:16:37.071+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.071+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.084+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:3 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:16:37.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 161 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:37.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.088+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:37.088+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:16:37.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex started 2023-10-21T04:16:37.088+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 1 Warmup took 99.184082ms 2023-10-21T04:16:37.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.089+05:30 [Info] StorageMgr::openSnapshot IndexInst:5744899830895973442 Partition:4 Attempting to open snapshot (SnapshotInfo: count:137 committed:false) 2023-10-21T04:16:37.090+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 1 WriterThreads 1 2023-10-21T04:16:37.091+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 1 Stopped Billing WUs 2023-10-21T04:16:37.091+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index Index: 10477796393418268202 Partition: 1 2023-10-21T04:16:37.091+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.092+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.092+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.092+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.093+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:37.100+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:37.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 130 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:37.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.104+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:37.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex started 2023-10-21T04:16:37.105+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.105+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.105+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.105+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.111+05:30 [Info] switched currmeta from 293 -> 294 force true 2023-10-21T04:16:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 145 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.119+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex started 2023-10-21T04:16:37.119+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:37.119+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:37.119+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.119+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.119+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.119+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:16:37.119+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:37.119+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:16:37.119+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:16:37.119+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex stopped 2023-10-21T04:16:37.119+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.120+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.120+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.120+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.120+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.129+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:16:37.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex closed 2023-10-21T04:16:37.130+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:37.130+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.131+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:16:37.132+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:37.132+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:16:37.132+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:16:37.132+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:37.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex stopped 2023-10-21T04:16:37.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.132+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.133+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.139+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:16:37.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex closed 2023-10-21T04:16:37.140+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:37.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex ... 2023-10-21T04:16:37.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 130 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex ... 2023-10-21T04:16:37.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex sucessfully destroyed 2023-10-21T04:16:37.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex ... 2023-10-21T04:16:37.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 145 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex ... 2023-10-21T04:16:37.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index destroyed 2023-10-21T04:16:37.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.154+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.154+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.171+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 180 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.171+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:37.171+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:16:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex started 2023-10-21T04:16:37.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 162 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:37.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.188+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:37.188+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:16:37.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex started 2023-10-21T04:16:37.189+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 2 Warmup took 97.159696ms 2023-10-21T04:16:37.189+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.189+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.190+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 2 WriterThreads 1 2023-10-21T04:16:37.191+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 2 Stopped Billing WUs 2023-10-21T04:16:37.191+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index Index: 10477796393418268202 Partition: 2 2023-10-21T04:16:37.191+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.194+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.194+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.195+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:37.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.196+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.197+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:37.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 146 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:37.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.199+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:37.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex started 2023-10-21T04:16:37.199+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.199+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 131 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:37.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.204+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:37.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex started 2023-10-21T04:16:37.204+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.204+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.204+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:37.205+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:16:37.205+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:16:37.205+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:37.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex stopped 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.205+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:16:37.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex closed 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:16:37.215+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:16:37.215+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:37.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex stopped 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.215+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.224+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:16:37.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex closed 2023-10-21T04:16:37.224+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:37.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex ... 2023-10-21T04:16:37.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 131 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex ... 2023-10-21T04:16:37.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex sucessfully destroyed 2023-10-21T04:16:37.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex ... 2023-10-21T04:16:37.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 146 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex ... 2023-10-21T04:16:37.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index destroyed 2023-10-21T04:16:37.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.238+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.239+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.244+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.244+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 181 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:37.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.247+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:37.247+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:16:37.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex started 2023-10-21T04:16:37.247+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.247+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 163 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:37.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.250+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:37.250+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:16:37.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex started 2023-10-21T04:16:37.250+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.250+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 3 Warmup took 56.468467ms 2023-10-21T04:16:37.250+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.251+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 3 WriterThreads 1 2023-10-21T04:16:37.251+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 3 Stopped Billing WUs 2023-10-21T04:16:37.251+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index Index: 10477796393418268202 Partition: 3 2023-10-21T04:16:37.251+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10477796393418268202 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.251+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10477796393418268202 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.251+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.252+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:37.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.252+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.252+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:37.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 132 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:37.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.258+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:37.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex started 2023-10-21T04:16:37.258+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.258+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 147 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:37.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.259+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:37.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex started 2023-10-21T04:16:37.259+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:37.259+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.259+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:37.259+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.259+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.259+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:16:37.259+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:37.259+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:16:37.259+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:16:37.259+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:37.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex stopped 2023-10-21T04:16:37.260+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.260+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.260+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.260+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.260+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:16:37.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex closed 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:16:37.266+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:16:37.266+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:37.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex stopped 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.266+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.273+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:16:37.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex closed 2023-10-21T04:16:37.273+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:37.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex ... 2023-10-21T04:16:37.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 132 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex ... 2023-10-21T04:16:37.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex sucessfully destroyed 2023-10-21T04:16:37.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex ... 2023-10-21T04:16:37.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 147 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex ... 2023-10-21T04:16:37.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index destroyed 2023-10-21T04:16:37.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.286+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.287+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.289+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.289+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[7494249569853685720:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[5 6 7 8 1 2 3 4] 7494249569853685720:[0] 14213410630307008054:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:37.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 164 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:37.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.304+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:37.304+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:16:37.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex started 2023-10-21T04:16:37.306+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 182 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:37.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.306+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:37.306+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:16:37.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex started 2023-10-21T04:16:37.306+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10477796393418268202 PartitionId 4 Warmup took 55.05386ms 2023-10-21T04:16:37.306+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10477796393418268202 partitionId 4 WriterThreads 1 2023-10-21T04:16:37.306+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10477796393418268202 PartitionId 4 Stopped Billing WUs 2023-10-21T04:16:37.306+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10477796393418268202 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index Index: 10477796393418268202 Partition: 4 2023-10-21T04:16:37.306+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 10477796393418268202, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:37.307+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 10477796393418268202, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:37.307+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10477796393418268202 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:37.307+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10477796393418268202 2023-10-21T04:16:37.307+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10477796393418268202 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:37.308+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:37.308+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:37.306+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.308+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.308+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.308+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10477796393418268202 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:37.309+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 10477796393418268202 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:37.309+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10477796393418268202 Partitions [5 6 7 8 1 2 3 4] 2023-10-21T04:16:37.309+05:30 [Info] StorageMgr::openSnapshot IndexInst:10477796393418268202 Partition:5 No Snapshot Found. 2023-10-21T04:16:37.309+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10477796393418268202 Adding Nil Snapshot. 2023-10-21T04:16:37.327+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.026196ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:37.327+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 107.229µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:37.357+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.361+05:30 [Info] switched currmeta from 294 -> 295 force true 2023-10-21T04:16:37.432+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.432+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.437+05:30 [Info] switched currmeta from 295 -> 297 force true 2023-10-21T04:16:37.446+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.446+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.450+05:30 [Info] switched currmeta from 297 -> 297 force true 2023-10-21T04:16:37.468+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.468+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.486+05:30 [Info] switched currmeta from 297 -> 298 force true 2023-10-21T04:16:37.500+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[5744899830895973442:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:37.502+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 10585832761158485697, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:37.502+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 10585832761158485697, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:37.531+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 759049595436785640,realInstId: 0, indexDefn DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:37.531+05:30 [Info] Indexer::handleRecoverIndex InstId: 759049595436785640 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:37.543+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 12.002809ms 2023-10-21T04:16:37.544+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 12.318401ms 2023-10-21T04:16:37.544+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:37.544+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:37.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:37.545+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.547+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 759049595436785640, indexDefn DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:37.548+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 45.879412ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:37.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:37.548+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.550+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.550+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.551+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:37.554+05:30 [Info] switched currmeta from 298 -> 299 force true 2023-10-21T04:16:37.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 136 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:37.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:37.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:37.557+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:37.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex started 2023-10-21T04:16:37.558+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.557+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:37.558+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 151 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:37.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:37.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:37.562+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:37.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex started 2023-10-21T04:16:37.562+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:37.562+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:37.562+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.563+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.563+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.563+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.563+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:16:37.564+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:37.564+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:16:37.564+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:16:37.564+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:37.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex stopped 2023-10-21T04:16:37.565+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.566+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.566+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.566+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.576+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[10477796393418268202:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:37.577+05:30 [Info] Indexer::handleMergePartition Source 976404723154064977 Target 10477796393418268202 2023-10-21T04:16:37.577+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10477796393418268202 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00f42d6e0 2023-10-21T04:16:37.577+05:30 [Info] MergePartition: sent async request to update index instance 10477796393418268202 rstate moved to ACTIVE 2023-10-21T04:16:37.585+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:16:37.586+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.908944ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:37.586+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 976404723154064977, realInst: 10477796393418268202 2023-10-21T04:16:37.586+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 14340086159258068923, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:37.587+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 14340086159258068923, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:37.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex closed 2023-10-21T04:16:37.597+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:37.597+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.598+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.598+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:16:37.598+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:37.598+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:16:37.598+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:16:37.598+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:37.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex stopped 2023-10-21T04:16:37.598+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.598+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.598+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.598+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.602+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:16:37.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex closed 2023-10-21T04:16:37.603+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:37.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex ... 2023-10-21T04:16:37.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 136 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex ... 2023-10-21T04:16:37.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex sucessfully destroyed 2023-10-21T04:16:37.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex ... 2023-10-21T04:16:37.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 151 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex ... 2023-10-21T04:16:37.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.613+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index destroyed 2023-10-21T04:16:37.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:37.619+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.619+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:37.624+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.625+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.626+05:30 [Info] switched currmeta from 299 -> 300 force true 2023-10-21T04:16:37.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 175 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:37.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:37.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:37.630+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:37.630+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:16:37.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex started 2023-10-21T04:16:37.630+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.631+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.640+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17357232428819395534,realInstId: 0, indexDefn DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:16:37.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 157 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:37.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:37.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:37.641+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:37.641+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:16:37.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex started 2023-10-21T04:16:37.641+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 4 Warmup took 96.071621ms 2023-10-21T04:16:37.641+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 4 WriterThreads 1 2023-10-21T04:16:37.641+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 4 Stopped Billing WUs 2023-10-21T04:16:37.641+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index Index: 759049595436785640 Partition: 4 2023-10-21T04:16:37.641+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:37.641+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:37.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:37.642+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.642+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.642+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.642+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:37.640+05:30 [Info] Indexer::handleRecoverIndex InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:16:37.643+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 2.727283ms 2023-10-21T04:16:37.643+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 2.802837ms 2023-10-21T04:16:37.643+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.643+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17357232428819395534, indexDefn DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:37.643+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 57.009107ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:37.643+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:37.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 112 metaSz: 2572 rp [1] sn: 720006 count: 112 metaSz: 2572 rp [2] sn: 720007 count: 112 metaSz: 2572 rp [3] sn: 1080007 count: 112 metaSz: 2573 2023-10-21T04:16:37.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 143915 recovery UsedSpace 157336 2023-10-21T04:16:37.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:37.645+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.646+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:37.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.650+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:37.650+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 112 metaSz: 2572 rp [1] sn: 1080007 count: 112 metaSz: 2573 2023-10-21T04:16:37.650+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 132581 recovery UsedSpace 165572 2023-10-21T04:16:37.686+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.704+05:30 [Info] switched currmeta from 300 -> 301 force true 2023-10-21T04:16:37.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex 2023-10-21T04:16:37.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.706+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex 2023-10-21T04:16:37.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex started 2023-10-21T04:16:37.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex 2023-10-21T04:16:37.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.706+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex 2023-10-21T04:16:37.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex started 2023-10-21T04:16:37.706+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.707+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.707+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.716+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.716+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:37.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 129 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:37.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:37.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:37.717+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:37.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex started 2023-10-21T04:16:37.718+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.718+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.757+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 1 Warmup took 113.336992ms 2023-10-21T04:16:37.757+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 1 WriterThreads 1 2023-10-21T04:16:37.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 1 Stopped Billing WUs 2023-10-21T04:16:37.758+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index Index: 17357232428819395534 Partition: 1 2023-10-21T04:16:37.758+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.758+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:37.769+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 123 metaSz: 2572 rp [1] sn: 720006 count: 123 metaSz: 2572 rp [2] sn: 720007 count: 123 metaSz: 2572 rp [3] sn: 1080007 count: 123 metaSz: 2573 2023-10-21T04:16:37.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.770+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:37.770+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 123 metaSz: 2572 rp [1] sn: 1080007 count: 123 metaSz: 2573 2023-10-21T04:16:37.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 145023 recovery UsedSpace 177816 2023-10-21T04:16:37.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 134127 recovery UsedSpace 165572 2023-10-21T04:16:37.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex 2023-10-21T04:16:37.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.781+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex 2023-10-21T04:16:37.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex started 2023-10-21T04:16:37.781+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.781+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex 2023-10-21T04:16:37.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.781+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex 2023-10-21T04:16:37.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex started 2023-10-21T04:16:37.781+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:37.784+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.784+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 144 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:37.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:37.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:37.786+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:37.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex started 2023-10-21T04:16:37.788+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.788+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.789+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:37.796+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:37.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.798+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.799+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:16:37.816+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.816+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.820+05:30 [Info] switched currmeta from 301 -> 302 force true 2023-10-21T04:16:37.821+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:37.821+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:16:37.831+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:16:37.831+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:37.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex stopped 2023-10-21T04:16:37.831+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.831+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.831+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.831+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.851+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 2 Warmup took 92.872953ms 2023-10-21T04:16:37.852+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 2 WriterThreads 1 2023-10-21T04:16:37.852+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 2 Stopped Billing WUs 2023-10-21T04:16:37.852+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index Index: 17357232428819395534 Partition: 2 2023-10-21T04:16:37.852+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.854+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.855+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:37.855+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 123 metaSz: 2572 rp [1] sn: 720006 count: 123 metaSz: 2572 rp [2] sn: 720007 count: 123 metaSz: 2573 rp [3] sn: 1080007 count: 123 metaSz: 2574 2023-10-21T04:16:37.855+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 145146 recovery UsedSpace 177816 2023-10-21T04:16:37.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.856+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:37.856+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 123 metaSz: 2573 rp [1] sn: 1080007 count: 123 metaSz: 2574 2023-10-21T04:16:37.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 134372 recovery UsedSpace 165572 2023-10-21T04:16:37.870+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:16:37.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex closed 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:16:37.871+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:37.871+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:16:37.871+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:16:37.871+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:37.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex stopped 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.871+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.879+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex started 2023-10-21T04:16:37.879+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.879+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex 2023-10-21T04:16:37.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex started 2023-10-21T04:16:37.879+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:37.880+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.880+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.880+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.885+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:16:37.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex closed 2023-10-21T04:16:37.886+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:37.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex ... 2023-10-21T04:16:37.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 129 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex ... 2023-10-21T04:16:37.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex sucessfully destroyed 2023-10-21T04:16:37.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex ... 2023-10-21T04:16:37.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 144 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex ... 2023-10-21T04:16:37.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index destroyed 2023-10-21T04:16:37.905+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 3 Warmup took 50.886412ms 2023-10-21T04:16:37.906+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 3 WriterThreads 1 2023-10-21T04:16:37.906+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 3 Stopped Billing WUs 2023-10-21T04:16:37.906+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index Index: 17357232428819395534 Partition: 3 2023-10-21T04:16:37.906+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.906+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:37.907+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.907+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:37.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 138 metaSz: 2573 rp [1] sn: 720006 count: 138 metaSz: 2573 rp [2] sn: 720007 count: 138 metaSz: 2573 rp [3] sn: 1080007 count: 138 metaSz: 2574 2023-10-21T04:16:37.908+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 164783 recovery UsedSpace 178524 2023-10-21T04:16:37.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.908+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:37.908+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 138 metaSz: 2573 rp [1] sn: 1080007 count: 138 metaSz: 2574 2023-10-21T04:16:37.909+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 136663 recovery UsedSpace 169668 2023-10-21T04:16:37.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:37.912+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:37.912+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.920+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex started 2023-10-21T04:16:37.920+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.920+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex 2023-10-21T04:16:37.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex started 2023-10-21T04:16:37.920+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:16:37.920+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.920+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.920+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 158 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:37.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:37.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:37.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 176 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:37.922+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:37.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:37.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:37.922+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:37.922+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:16:37.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex started 2023-10-21T04:16:37.922+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.922+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.922+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:16:37.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex started 2023-10-21T04:16:37.922+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.922+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 5 Warmup took 280.56337ms 2023-10-21T04:16:37.922+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.922+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 5 WriterThreads 1 2023-10-21T04:16:37.922+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 5 Stopped Billing WUs 2023-10-21T04:16:37.922+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index Index: 759049595436785640 Partition: 5 2023-10-21T04:16:37.922+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:37.923+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:37.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:37.923+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.923+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:37.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:37.923+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:37.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:37.936+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:37.936+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:37.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 145 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:37.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:37.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:37.937+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:37.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex started 2023-10-21T04:16:37.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 130 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:37.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:37.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:37.938+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:37.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex started 2023-10-21T04:16:37.938+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:37.938+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:16:37.938+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:37.938+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:16:37.938+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:16:37.938+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:37.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex stopped 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.938+05:30 [Info] switched currmeta from 302 -> 303 force true 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.938+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.955+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 4 Warmup took 48.632159ms 2023-10-21T04:16:37.956+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.956+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 4 WriterThreads 1 2023-10-21T04:16:37.956+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 4 Stopped Billing WUs 2023-10-21T04:16:37.956+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index Index: 17357232428819395534 Partition: 4 2023-10-21T04:16:37.956+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:37.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:16:37.956+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:37.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex closed 2023-10-21T04:16:37.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:37.956+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:37.956+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 117 metaSz: 2572 rp [1] sn: 720006 count: 117 metaSz: 2572 rp [2] sn: 720007 count: 117 metaSz: 2573 rp [3] sn: 1080007 count: 117 metaSz: 2574 2023-10-21T04:16:37.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:37.957+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:37.957+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 117 metaSz: 2573 rp [1] sn: 1080007 count: 117 metaSz: 2574 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:37.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:16:37.957+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:37.957+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:16:37.957+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:16:37.957+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:37.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex stopped 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:37.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:37.957+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 156450 recovery UsedSpace 177816 2023-10-21T04:16:37.958+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 133100 recovery UsedSpace 165572 2023-10-21T04:16:37.978+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex closed 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:37.978+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex started 2023-10-21T04:16:37.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.978+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex ... 2023-10-21T04:16:37.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 130 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex ... 2023-10-21T04:16:37.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex 2023-10-21T04:16:37.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:37.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:37.982+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex 2023-10-21T04:16:37.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex started 2023-10-21T04:16:37.982+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:37.982+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:16:37.982+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:37.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:37.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex sucessfully destroyed 2023-10-21T04:16:37.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex ... 2023-10-21T04:16:37.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 145 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex ... 2023-10-21T04:16:37.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9790447611902976, throttleDelayMs (new, old, change): (7, 4, 3) 2023-10-21T04:16:38.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index destroyed 2023-10-21T04:16:38.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.008+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.008+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.015+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.015+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 159 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:38.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.019+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:38.019+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:16:38.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex started 2023-10-21T04:16:38.020+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.019+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 5 Warmup took 63.217077ms 2023-10-21T04:16:38.020+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 5 WriterThreads 1 2023-10-21T04:16:38.020+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 5 Stopped Billing WUs 2023-10-21T04:16:38.020+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index Index: 17357232428819395534 Partition: 5 2023-10-21T04:16:38.020+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:38.020+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:38.020+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:38.020+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:38.020+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 137 metaSz: 2573 rp [1] sn: 720006 count: 137 metaSz: 2573 rp [2] sn: 720007 count: 137 metaSz: 2573 rp [3] sn: 1080007 count: 137 metaSz: 2574 2023-10-21T04:16:38.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:38.021+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:38.021+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 137 metaSz: 2573 rp [1] sn: 1080007 count: 137 metaSz: 2574 2023-10-21T04:16:38.021+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 164566 recovery UsedSpace 178524 2023-10-21T04:16:38.022+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 136400 recovery UsedSpace 165572 2023-10-21T04:16:38.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 177 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:38.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.027+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:38.027+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:16:38.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex started 2023-10-21T04:16:38.027+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 6 Warmup took 104.793874ms 2023-10-21T04:16:38.028+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 6 WriterThreads 1 2023-10-21T04:16:38.028+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 6 Stopped Billing WUs 2023-10-21T04:16:38.028+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index Index: 759049595436785640 Partition: 6 2023-10-21T04:16:38.028+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:38.028+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.028+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex 2023-10-21T04:16:38.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:38.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:38.028+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex 2023-10-21T04:16:38.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex started 2023-10-21T04:16:38.029+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.029+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.029+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:38.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.029+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.029+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.030+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:38.030+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:38.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex 2023-10-21T04:16:38.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:38.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:38.032+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex 2023-10-21T04:16:38.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex started 2023-10-21T04:16:38.032+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.032+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.032+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:16:38.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 131 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:38.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.042+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:38.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex started 2023-10-21T04:16:38.042+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.042+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 146 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:38.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.043+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:38.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex started 2023-10-21T04:16:38.043+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:38.043+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:38.043+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.043+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.043+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.043+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:16:38.043+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:38.044+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:16:38.044+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:16:38.044+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:38.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex stopped 2023-10-21T04:16:38.044+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.044+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.044+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.044+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.044+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.053+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:16:38.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex closed 2023-10-21T04:16:38.053+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:38.053+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.053+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.054+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.054+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:16:38.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:38.054+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:16:38.054+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:16:38.054+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:38.054+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex stopped 2023-10-21T04:16:38.054+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.054+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.054+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.054+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.054+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 6 Warmup took 33.714472ms 2023-10-21T04:16:38.054+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 6 WriterThreads 1 2023-10-21T04:16:38.054+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 6 Stopped Billing WUs 2023-10-21T04:16:38.054+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index Index: 17357232428819395534 Partition: 6 2023-10-21T04:16:38.054+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:38.054+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:38.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:38.055+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:16:38.055+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 102 metaSz: 2571 rp [1] sn: 720006 count: 102 metaSz: 2571 rp [2] sn: 720007 count: 102 metaSz: 2571 rp [3] sn: 1080007 count: 102 metaSz: 2572 2023-10-21T04:16:38.055+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 154008 recovery UsedSpace 177816 2023-10-21T04:16:38.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:38.056+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:16:38.056+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 102 metaSz: 2571 rp [1] sn: 1080007 count: 102 metaSz: 2572 2023-10-21T04:16:38.056+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 130736 recovery UsedSpace 165572 2023-10-21T04:16:38.070+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:16:38.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex closed 2023-10-21T04:16:38.070+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:38.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex ... 2023-10-21T04:16:38.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 131 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex ... 2023-10-21T04:16:38.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex 2023-10-21T04:16:38.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex 2023-10-21T04:16:38.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:38.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:38.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:38.080+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex 2023-10-21T04:16:38.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex started 2023-10-21T04:16:38.081+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:38.081+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex 2023-10-21T04:16:38.081+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex started 2023-10-21T04:16:38.081+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:16:38.081+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.081+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex sucessfully destroyed 2023-10-21T04:16:38.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex ... 2023-10-21T04:16:38.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 146 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex ... 2023-10-21T04:16:38.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index destroyed 2023-10-21T04:16:38.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.098+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.098+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.106+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.106+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 160 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:38.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.113+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:38.113+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:16:38.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex started 2023-10-21T04:16:38.113+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.113+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.118+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 7 Warmup took 63.865748ms 2023-10-21T04:16:38.119+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 7 WriterThreads 1 2023-10-21T04:16:38.119+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 7 Stopped Billing WUs 2023-10-21T04:16:38.119+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index Index: 17357232428819395534 Partition: 7 2023-10-21T04:16:38.120+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17357232428819395534 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:38.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 178 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:38.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.120+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:38.120+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:16:38.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex started 2023-10-21T04:16:38.120+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 7 Warmup took 90.951467ms 2023-10-21T04:16:38.120+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 7 WriterThreads 1 2023-10-21T04:16:38.120+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 7 Stopped Billing WUs 2023-10-21T04:16:38.120+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index Index: 759049595436785640 Partition: 7 2023-10-21T04:16:38.120+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:38.120+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:38.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.121+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.121+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:16:38.122+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.122+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.122+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.122+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:38.125+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17357232428819395534 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:38.126+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:16:38.126+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 360005 count: 148 metaSz: 2573 rp [1] sn: 720006 count: 148 metaSz: 2573 rp [2] sn: 720007 count: 148 metaSz: 2573 rp [3] sn: 1080007 count: 148 metaSz: 2574 2023-10-21T04:16:38.127+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 166608 recovery UsedSpace 158044 2023-10-21T04:16:38.127+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:16:38.127+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 148 metaSz: 2573 rp [1] sn: 1080007 count: 148 metaSz: 2574 2023-10-21T04:16:38.128+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 138009 recovery UsedSpace 165572 2023-10-21T04:16:38.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 132 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:38.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.133+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:38.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex started 2023-10-21T04:16:38.133+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.133+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 147 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:38.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.134+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:38.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex started 2023-10-21T04:16:38.134+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:38.134+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:38.134+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.134+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.134+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:16:38.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.135+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:38.135+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:16:38.135+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:16:38.135+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:38.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex stopped 2023-10-21T04:16:38.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex 2023-10-21T04:16:38.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:38.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:38.137+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex 2023-10-21T04:16:38.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex started 2023-10-21T04:16:38.138+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.138+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex 2023-10-21T04:16:38.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:38.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:38.139+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex 2023-10-21T04:16:38.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex started 2023-10-21T04:16:38.139+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.139+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:16:38.139+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.143+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:16:38.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex closed 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:16:38.144+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:38.144+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:16:38.144+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:16:38.144+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:38.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex stopped 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.144+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:16:38.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex closed 2023-10-21T04:16:38.151+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:38.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex ... 2023-10-21T04:16:38.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 132 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex ... 2023-10-21T04:16:38.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex sucessfully destroyed 2023-10-21T04:16:38.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex ... 2023-10-21T04:16:38.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 147 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex ... 2023-10-21T04:16:38.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index destroyed 2023-10-21T04:16:38.164+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17357232428819395534 PartitionId 8 Warmup took 38.000509ms 2023-10-21T04:16:38.164+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17357232428819395534 partitionId 8 WriterThreads 1 2023-10-21T04:16:38.164+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17357232428819395534 PartitionId 8 Stopped Billing WUs 2023-10-21T04:16:38.164+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17357232428819395534 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index Index: 17357232428819395534 Partition: 8 2023-10-21T04:16:38.164+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17357232428819395534, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:38.164+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17357232428819395534, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:38.164+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.165+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17357232428819395534 2023-10-21T04:16:38.165+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:38.165+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:38.165+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:38.166+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:38.166+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.166+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17357232428819395534 Partitions [7 8 1 2 3 4 5 6] 2023-10-21T04:16:38.167+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:4 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:16:38.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.169+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:5 Attempting to open snapshot (SnapshotInfo: count:117 committed:false) 2023-10-21T04:16:38.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.170+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.170+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.170+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:6 Attempting to open snapshot (SnapshotInfo: count:137 committed:false) 2023-10-21T04:16:38.171+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:7 Attempting to open snapshot (SnapshotInfo: count:102 committed:false) 2023-10-21T04:16:38.172+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:8 Attempting to open snapshot (SnapshotInfo: count:148 committed:false) 2023-10-21T04:16:38.173+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:1 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:16:38.174+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:2 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:16:38.175+05:30 [Info] StorageMgr::openSnapshot IndexInst:17357232428819395534 Partition:3 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:16:38.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.178+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.189+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.907835ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.189+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 108.618µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:38.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 161 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:38.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:38.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:16:38.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex started 2023-10-21T04:16:38.194+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.194+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 179 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:38.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.202+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:38.202+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:16:38.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex started 2023-10-21T04:16:38.202+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 8 Warmup took 81.715802ms 2023-10-21T04:16:38.202+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 8 WriterThreads 1 2023-10-21T04:16:38.202+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 8 Stopped Billing WUs 2023-10-21T04:16:38.203+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index Index: 759049595436785640 Partition: 8 2023-10-21T04:16:38.203+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:38.203+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:38.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.203+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.204+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.204+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:38.208+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:38.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 133 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:38.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.214+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:38.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex started 2023-10-21T04:16:38.214+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.214+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.216+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.216+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.221+05:30 [Info] switched currmeta from 303 -> 304 force true 2023-10-21T04:16:38.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 148 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:38.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.229+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:38.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex started 2023-10-21T04:16:38.229+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:16:38.229+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:16:38.229+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:38.229+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:16:38.229+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:16:38.229+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:38.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex stopped 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.229+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.230+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.241+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:16:38.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex closed 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:16:38.242+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:38.242+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:16:38.242+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:16:38.242+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:38.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex stopped 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.242+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.250+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:16:38.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex closed 2023-10-21T04:16:38.250+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:38.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex ... 2023-10-21T04:16:38.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 133 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex ... 2023-10-21T04:16:38.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex sucessfully destroyed 2023-10-21T04:16:38.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex ... 2023-10-21T04:16:38.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 148 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex ... 2023-10-21T04:16:38.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index destroyed 2023-10-21T04:16:38.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.263+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.263+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.267+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.267+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 162 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:38.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.271+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:38.271+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:16:38.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex started 2023-10-21T04:16:38.271+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.271+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 180 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:38.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.274+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:38.274+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:16:38.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex started 2023-10-21T04:16:38.274+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 1 Warmup took 70.882348ms 2023-10-21T04:16:38.274+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 1 WriterThreads 1 2023-10-21T04:16:38.274+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 1 Stopped Billing WUs 2023-10-21T04:16:38.274+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index Index: 759049595436785640 Partition: 1 2023-10-21T04:16:38.274+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:38.274+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:38.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.275+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.275+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:38.276+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.276+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.276+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.277+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:38.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 134 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:38.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:38.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex started 2023-10-21T04:16:38.281+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.281+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 149 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:38.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.283+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:38.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex started 2023-10-21T04:16:38.283+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.283+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:16:38.283+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.283+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:38.283+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.283+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.283+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:16:38.284+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:38.284+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:16:38.284+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:16:38.284+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:38.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex stopped 2023-10-21T04:16:38.284+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.284+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.284+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.284+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.289+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:16:38.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex closed 2023-10-21T04:16:38.289+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:38.289+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.290+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.290+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:16:38.290+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:38.290+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:16:38.290+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:16:38.290+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:38.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex stopped 2023-10-21T04:16:38.290+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.290+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.290+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.290+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.295+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:16:38.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex closed 2023-10-21T04:16:38.296+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:38.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex ... 2023-10-21T04:16:38.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 134 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex ... 2023-10-21T04:16:38.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex sucessfully destroyed 2023-10-21T04:16:38.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex ... 2023-10-21T04:16:38.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 149 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex ... 2023-10-21T04:16:38.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index destroyed 2023-10-21T04:16:38.304+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.305+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.305+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.309+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.309+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 163 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:38.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.314+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:38.314+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:16:38.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex started 2023-10-21T04:16:38.314+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.314+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 181 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:38.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.319+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:38.319+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:16:38.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex started 2023-10-21T04:16:38.319+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 2 Warmup took 44.482836ms 2023-10-21T04:16:38.319+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.319+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.319+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 2 WriterThreads 1 2023-10-21T04:16:38.319+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 2 Stopped Billing WUs 2023-10-21T04:16:38.319+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index Index: 759049595436785640 Partition: 2 2023-10-21T04:16:38.319+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 759049595436785640 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:38.319+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 759049595436785640 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:38.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.320+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.320+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.320+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:38.321+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:38.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 135 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:38.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.330+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:38.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex started 2023-10-21T04:16:38.330+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.330+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 150 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:38.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.331+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:38.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex started 2023-10-21T04:16:38.331+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:16:38.331+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:38.331+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.331+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.331+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.331+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:16:38.331+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.333+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:38.333+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:16:38.333+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:16:38.333+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:38.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex stopped 2023-10-21T04:16:38.333+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.333+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.333+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.333+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.339+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:16:38.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex closed 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:16:38.340+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:38.340+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:16:38.340+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:16:38.340+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:38.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex stopped 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.340+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.347+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:16:38.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex closed 2023-10-21T04:16:38.347+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:38.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex ... 2023-10-21T04:16:38.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 135 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex ... 2023-10-21T04:16:38.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex sucessfully destroyed 2023-10-21T04:16:38.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex ... 2023-10-21T04:16:38.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 150 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex ... 2023-10-21T04:16:38.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index destroyed 2023-10-21T04:16:38.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.358+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.359+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.362+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.362+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 164 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:38.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.369+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:38.369+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:16:38.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex started 2023-10-21T04:16:38.369+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.369+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 182 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:38.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.370+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:38.370+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:16:38.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex started 2023-10-21T04:16:38.370+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 759049595436785640 PartitionId 3 Warmup took 50.531392ms 2023-10-21T04:16:38.370+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.370+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.370+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 759049595436785640 partitionId 3 WriterThreads 1 2023-10-21T04:16:38.370+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 759049595436785640 PartitionId 3 Stopped Billing WUs 2023-10-21T04:16:38.370+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 759049595436785640 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index Index: 759049595436785640 Partition: 3 2023-10-21T04:16:38.370+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 759049595436785640, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:38.370+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 759049595436785640, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:38.371+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 759049595436785640 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.371+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 759049595436785640 2023-10-21T04:16:38.372+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 759049595436785640 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:38.372+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:16:38.372+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:38.373+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 759049595436785640 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:38.373+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 759049595436785640 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.373+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 759049595436785640 Partitions [4 5 6 7 8 1 2 3] 2023-10-21T04:16:38.373+05:30 [Info] StorageMgr::openSnapshot IndexInst:759049595436785640 Partition:4 No Snapshot Found. 2023-10-21T04:16:38.373+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 759049595436785640 Adding Nil Snapshot. 2023-10-21T04:16:38.379+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.956752ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.379+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 109.965µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:38.399+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.399+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.413+05:30 [Info] switched currmeta from 304 -> 305 force true 2023-10-21T04:16:38.437+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.438+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.442+05:30 [Info] switched currmeta from 305 -> 306 force true 2023-10-21T04:16:38.464+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.464+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.468+05:30 [Info] switched currmeta from 306 -> 307 force true 2023-10-21T04:16:38.489+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.489+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.497+05:30 [Info] switched currmeta from 307 -> 308 force true 2023-10-21T04:16:38.561+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[759049595436785640:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:38.561+05:30 [Info] Indexer::handleMergePartition Source 3381906773469851 Target 759049595436785640 2023-10-21T04:16:38.561+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 759049595436785640 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00689b8c0 2023-10-21T04:16:38.561+05:30 [Info] MergePartition: sent async request to update index instance 759049595436785640 rstate moved to ACTIVE 2023-10-21T04:16:38.568+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.622037ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.569+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 3381906773469851, realInst: 759049595436785640 2023-10-21T04:16:38.576+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 6985049432225870291, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:38.576+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 6985049432225870291, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:38.589+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.589+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.594+05:30 [Info] switched currmeta from 308 -> 309 force true 2023-10-21T04:16:38.604+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4173530108555528677,realInstId: 0, indexDefn DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:38.615+05:30 [Info] Indexer::handleRecoverIndex InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:38.618+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 2.896572ms 2023-10-21T04:16:38.619+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.710898ms 2023-10-21T04:16:38.619+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4173530108555528677 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:38.623+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.623+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.619+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4173530108555528677, indexDefn DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:38.623+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 47.465523ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.623+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4173530108555528677 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:38.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.624+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.627+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:16:38.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.629+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.631+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:38.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 137 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:38.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.639+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:38.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex started 2023-10-21T04:16:38.639+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.639+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 152 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:38.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.642+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:38.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex started 2023-10-21T04:16:38.642+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4173530108555528677 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:38.642+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4173530108555528677 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:38.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:16:38.642+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:38.642+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:16:38.642+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:16:38.642+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:38.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex stopped 2023-10-21T04:16:38.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.643+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.643+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.644+05:30 [Info] switched currmeta from 309 -> 310 force true 2023-10-21T04:16:38.654+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:16:38.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex closed 2023-10-21T04:16:38.661+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:38.662+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.664+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.667+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:16:38.670+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:38.671+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:16:38.674+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:16:38.675+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:38.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex stopped 2023-10-21T04:16:38.676+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.675+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14975131556821987100:INDEX_STATE_RECOVERED 17357232428819395534:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:38.684+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.684+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.685+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.685+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 15968368593314043432, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:38.685+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 15968368593314043432, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:38.696+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:16:38.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex closed 2023-10-21T04:16:38.697+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:38.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex ... 2023-10-21T04:16:38.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 137 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex ... 2023-10-21T04:16:38.700+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8606677778651310689,realInstId: 0, indexDefn DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:38.700+05:30 [Info] Indexer::handleRecoverIndex InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:38.700+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 226.746µs 2023-10-21T04:16:38.700+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 268.332µs 2023-10-21T04:16:38.701+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8606677778651310689 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:38.701+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8606677778651310689 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:38.701+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8606677778651310689, indexDefn DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:38.702+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 16.701301ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex sucessfully destroyed 2023-10-21T04:16:38.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex ... 2023-10-21T04:16:38.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 152 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex ... 2023-10-21T04:16:38.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index destroyed 2023-10-21T04:16:38.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:38.709+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.710+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:38.713+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.714+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 137 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:38.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:38.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:38.718+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:38.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex started 2023-10-21T04:16:38.718+05:30 [Info] switched currmeta from 310 -> 311 force true 2023-10-21T04:16:38.718+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.718+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:38.722+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.722+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:38.723+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:38.726+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:38.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:38.729+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.729+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 165 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:38.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:38.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:38.730+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:38.730+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:16:38.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex started 2023-10-21T04:16:38.730+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.730+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 152 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:38.736+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex started 2023-10-21T04:16:38.736+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8606677778651310689 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:38.736+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8606677778651310689 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:16:38.736+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:16:38.736+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex stopped 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 183 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:38.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:38.737+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:38.737+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:16:38.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex started 2023-10-21T04:16:38.737+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4173530108555528677 PartitionId 0 Warmup took 113.480275ms 2023-10-21T04:16:38.737+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4173530108555528677 partitionId 0 WriterThreads 4 2023-10-21T04:16:38.737+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4173530108555528677 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index Index: 4173530108555528677 Partition: 0 2023-10-21T04:16:38.737+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4173530108555528677, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:38.737+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4173530108555528677, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:38.737+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.738+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4173530108555528677 2023-10-21T04:16:38.738+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:38.738+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:38.738+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:38.739+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:38.739+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.739+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4173530108555528677 Partitions [0] 2023-10-21T04:16:38.739+05:30 [Info] StorageMgr::openSnapshot IndexInst:4173530108555528677 Partition:0 No Snapshot Found. 2023-10-21T04:16:38.739+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4173530108555528677 Adding Nil Snapshot. 2023-10-21T04:16:38.739+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.739+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.740+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.758+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:16:38.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex closed 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:38.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:16:38.759+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:16:38.759+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:38.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex stopped 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:38.759+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:38.763+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.703274ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.763+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 105.721µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:38.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:38.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:16:38.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex closed 2023-10-21T04:16:38.772+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:38.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex ... 2023-10-21T04:16:38.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 137 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex ... 2023-10-21T04:16:38.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex sucessfully destroyed 2023-10-21T04:16:38.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex ... 2023-10-21T04:16:38.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 152 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex ... 2023-10-21T04:16:38.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index destroyed 2023-10-21T04:16:38.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:16:38.784+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:38.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:38.794+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:38.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:38.801+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:38.802+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.802+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.806+05:30 [Info] switched currmeta from 311 -> 312 force true 2023-10-21T04:16:38.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 165 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:38.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:16:38.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:16:38.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:38.808+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:16:38.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex started 2023-10-21T04:16:38.808+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.809+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 183 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:38.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:38.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:38.814+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:38.814+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:16:38.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex started 2023-10-21T04:16:38.814+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8606677778651310689 PartitionId 0 Warmup took 112.621923ms 2023-10-21T04:16:38.814+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8606677778651310689 partitionId 0 WriterThreads 4 2023-10-21T04:16:38.814+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:38.814+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8606677778651310689 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:38.814+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:38.814+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8606677778651310689 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index Index: 8606677778651310689 Partition: 0 2023-10-21T04:16:38.814+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8606677778651310689, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:16:38.814+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8606677778651310689, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:16:38.814+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.815+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8606677778651310689 2023-10-21T04:16:38.815+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:38.815+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:38.815+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:38.816+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:38.816+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:38.816+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8606677778651310689 Partitions [0] 2023-10-21T04:16:38.817+05:30 [Info] StorageMgr::openSnapshot IndexInst:8606677778651310689 Partition:0 No Snapshot Found. 2023-10-21T04:16:38.817+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8606677778651310689 Adding Nil Snapshot. 2023-10-21T04:16:38.824+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.57802ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.827+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 853.801µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:38.864+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.864+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.875+05:30 [Info] switched currmeta from 312 -> 315 force true 2023-10-21T04:16:38.892+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.892+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.898+05:30 [Info] switched currmeta from 315 -> 315 force true 2023-10-21T04:16:38.946+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:38.946+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:38.949+05:30 [Info] switched currmeta from 315 -> 316 force true 2023-10-21T04:16:38.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9198573541663124, throttleDelayMs (new, old, change): (4, 7, -3) 2023-10-21T04:16:39.250+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.250+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.253+05:30 [Info] switched currmeta from 316 -> 318 force true 2023-10-21T04:16:39.271+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.271+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.280+05:30 [Info] switched currmeta from 318 -> 319 force true 2023-10-21T04:16:39.290+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.292+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.300+05:30 [Info] switched currmeta from 319 -> 319 force true 2023-10-21T04:16:39.340+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.340+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.344+05:30 [Info] switched currmeta from 319 -> 320 force true 2023-10-21T04:16:39.380+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.382+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.387+05:30 [Info] switched currmeta from 320 -> 321 force true 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[2 3 4 5 6 7 8 1] 16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[2 3 4 5 6 7 8 1] 16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[2 3 4 5 6 7 8 1] 16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.507+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[2 3 4 5 6 7 8 1] 16422752179446142451:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:39.635+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4173530108555528677:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:39.635+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00ca7a060 2023-10-21T04:16:39.636+05:30 [Info] handleUpdateIndexRState: Index instance 4173530108555528677 rstate moved to ACTIVE 2023-10-21T04:16:39.648+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.559556ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:39.649+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12140458060295645238, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:39.649+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12140458060295645238, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:39.660+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.660+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.665+05:30 [Info] switched currmeta from 321 -> 322 force true 2023-10-21T04:16:39.680+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 962468716067699903,realInstId: 0, indexDefn DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:39.681+05:30 [Info] Indexer::handleRecoverIndex InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:39.681+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 284.223µs 2023-10-21T04:16:39.681+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 366.15µs 2023-10-21T04:16:39.681+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 962468716067699903 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:39.682+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 962468716067699903 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:39.682+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 962468716067699903, indexDefn DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:39.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:16:39.682+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 33.201449ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:39.682+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1056180 memSz 2060 2023-10-21T04:16:39.682+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 1000 metaSz: 2579 rp [1] sn: 1080007 count: 1000 metaSz: 2578 2023-10-21T04:16:39.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:39.682+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1056190 memSz 880 2023-10-21T04:16:39.682+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 1000 metaSz: 2579 rp [1] sn: 1080007 count: 1000 metaSz: 2578 2023-10-21T04:16:39.683+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 230815 recovery UsedSpace 165402 2023-10-21T04:16:39.692+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 227841 recovery UsedSpace 140033 2023-10-21T04:16:39.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.700+05:30 [Info] switched currmeta from 322 -> 323 force true 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex 2023-10-21T04:16:39.711+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex started 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:39.711+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex 2023-10-21T04:16:39.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex started 2023-10-21T04:16:39.711+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 962468716067699903 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:39.711+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:39.711+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:39.711+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:39.711+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:39.714+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8606677778651310689:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:39.715+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00e2a4d80 2023-10-21T04:16:39.715+05:30 [Info] handleUpdateIndexRState: Index instance 8606677778651310689 rstate moved to ACTIVE 2023-10-21T04:16:39.724+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.623779ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:39.724+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 5197198800322775808, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:39.724+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 5197198800322775808, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:39.744+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.744+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.749+05:30 [Info] switched currmeta from 323 -> 324 force true 2023-10-21T04:16:39.760+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 962468716067699903 PartitionId 0 Warmup took 78.701008ms 2023-10-21T04:16:39.761+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 962468716067699903 partitionId 0 WriterThreads 4 2023-10-21T04:16:39.761+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 962468716067699903 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:39.761+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 962468716067699903 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index Index: 962468716067699903 Partition: 0 2023-10-21T04:16:39.761+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 962468716067699903, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:16:39.761+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 962468716067699903, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:16:39.761+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:39.762+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 962468716067699903 2023-10-21T04:16:39.763+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:39.763+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:39.764+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:39.765+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:39.766+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:39.766+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 962468716067699903 Partitions [0] 2023-10-21T04:16:39.767+05:30 [Info] StorageMgr::openSnapshot IndexInst:962468716067699903 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:39.767+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 1265729768899070816,realInstId: 0, indexDefn DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:39.768+05:30 [Info] Indexer::handleRecoverIndex InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:39.768+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 1265729768899070816, indexDefn DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:39.768+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 43.899353ms len(expediates) 1 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:39.768+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 915.298µs 2023-10-21T04:16:39.768+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 840.042µs 2023-10-21T04:16:39.768+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1265729768899070816 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:39.769+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1265729768899070816 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:39.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:39.769+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1054618 memSz 948 2023-10-21T04:16:39.769+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 720007 count: 1000 metaSz: 2551 rp [1] sn: 1080007 count: 1000 metaSz: 2551 2023-10-21T04:16:39.770+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 227709 recovery UsedSpace 169313 2023-10-21T04:16:39.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex 2023-10-21T04:16:39.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:39.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:39.786+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex 2023-10-21T04:16:39.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex started 2023-10-21T04:16:39.786+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1265729768899070816 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:39.787+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:39.787+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:39.808+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 39.675272ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:39.808+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 84.984µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:39.818+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.818+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.828+05:30 [Info] switched currmeta from 324 -> 326 force true 2023-10-21T04:16:39.834+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.834+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.837+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1265729768899070816 PartitionId 0 Warmup took 67.976796ms 2023-10-21T04:16:39.837+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1265729768899070816 partitionId 0 WriterThreads 4 2023-10-21T04:16:39.837+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 1265729768899070816 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:39.838+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1265729768899070816 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index Index: 1265729768899070816 Partition: 0 2023-10-21T04:16:39.838+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 1265729768899070816, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:16:39.838+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 1265729768899070816, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:16:39.838+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:39.837+05:30 [Info] switched currmeta from 326 -> 326 force true 2023-10-21T04:16:39.840+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 1265729768899070816 2023-10-21T04:16:39.840+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:39.841+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:39.841+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:39.842+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:39.842+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:39.843+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 1265729768899070816 Partitions [0] 2023-10-21T04:16:39.844+05:30 [Info] StorageMgr::openSnapshot IndexInst:1265729768899070816 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:39.850+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.998178ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:39.853+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 2.984472ms len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:39.858+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.858+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.861+05:30 [Info] switched currmeta from 326 -> 327 force true 2023-10-21T04:16:39.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:39.944+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:39.948+05:30 [Info] switched currmeta from 327 -> 328 force true 2023-10-21T04:16:39.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7594545231797021, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:16:40.063+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:40.064+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:40.067+05:30 [Info] switched currmeta from 328 -> 329 force true 2023-10-21T04:16:40.366+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:40.366+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:40.370+05:30 [Info] switched currmeta from 329 -> 330 force true 2023-10-21T04:16:40.459+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:40.459+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:40.463+05:30 [Info] switched currmeta from 330 -> 331 force true 2023-10-21T04:16:40.699+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[962468716067699903:INDEX_STATE_RECOVERED 5744899830895973442:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:40.699+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 5103684059183126577, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:40.699+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 5103684059183126577, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:40.714+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7582778950442222621,realInstId: 0, indexDefn DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:40.714+05:30 [Info] Indexer::handleRecoverIndex InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:40.714+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 149.87µs 2023-10-21T04:16:40.714+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 200.068µs 2023-10-21T04:16:40.714+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7582778950442222621 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:40.715+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7582778950442222621 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:40.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:40.715+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1054956 memSz 804 2023-10-21T04:16:40.715+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 720006 count: 1000 metaSz: 2555 rp [1] sn: 1080007 count: 1000 metaSz: 2554 2023-10-21T04:16:40.715+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7582778950442222621, indexDefn DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:40.715+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 15.798832ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:40.719+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 204075 recovery UsedSpace 135429 2023-10-21T04:16:40.724+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:40.724+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:40.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex 2023-10-21T04:16:40.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:40.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:40.728+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex 2023-10-21T04:16:40.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex started 2023-10-21T04:16:40.728+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7582778950442222621 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:40.729+05:30 [Info] switched currmeta from 331 -> 332 force true 2023-10-21T04:16:40.730+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:40.730+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:40.741+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7582778950442222621 PartitionId 0 Warmup took 26.26186ms 2023-10-21T04:16:40.741+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7582778950442222621 partitionId 0 WriterThreads 4 2023-10-21T04:16:40.741+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7582778950442222621 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:40.741+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7582778950442222621 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index Index: 7582778950442222621 Partition: 0 2023-10-21T04:16:40.741+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7582778950442222621, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:16:40.741+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7582778950442222621, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:16:40.741+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:40.742+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7582778950442222621 2023-10-21T04:16:40.742+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:40.742+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:40.742+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:40.743+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:40.743+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:40.743+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7582778950442222621 Partitions [0] 2023-10-21T04:16:40.744+05:30 [Info] StorageMgr::openSnapshot IndexInst:7582778950442222621 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:16:40.749+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.565635ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:40.749+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 65.279µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:40.755+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:40.756+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:40.765+05:30 [Info] switched currmeta from 332 -> 333 force true 2023-10-21T04:16:40.821+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[1265729768899070816:INDEX_STATE_RECOVERED 14975131556821987100:INDEX_STATE_RECOVERED 17357232428819395534:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:40.822+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 17878371269610391239, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:40.822+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 17878371269610391239, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:16:40.841+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 18415077564616412601,realInstId: 0, indexDefn DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:40.844+05:30 [Info] Indexer::handleRecoverIndex InstId: 18415077564616412601 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:40.852+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 18415077564616412601 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:40.857+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 18415077564616412601 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:40.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:40.852+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 8.49358ms 2023-10-21T04:16:40.853+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 9.018541ms 2023-10-21T04:16:40.853+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 18415077564616412601, indexDefn DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:40.859+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:40.863+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:16:40.864+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 41.897234ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:40.875+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:40.875+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:40.880+05:30 [Info] switched currmeta from 333 -> 334 force true 2023-10-21T04:16:40.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 153 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:40.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:40.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:40.894+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:40.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex started 2023-10-21T04:16:40.896+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 18415077564616412601 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:40.896+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 18415077564616412601 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:40.897+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:40.897+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:40.897+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:40.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:40.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:40.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:16:40.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:40.900+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:16:40.900+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:16:40.901+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:40.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex stopped 2023-10-21T04:16:40.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:40.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:40.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:40.902+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:40.913+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:40.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:16:40.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex closed 2023-10-21T04:16:40.914+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:40.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex ... 2023-10-21T04:16:40.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 153 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex ... 2023-10-21T04:16:40.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:40.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index destroyed 2023-10-21T04:16:40.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:16:40.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:40.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:40.926+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:40.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 184 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:40.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:40.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:40.935+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:40.935+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:16:40.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex started 2023-10-21T04:16:40.935+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:40.935+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 18415077564616412601 PartitionId 0 Warmup took 77.518755ms 2023-10-21T04:16:40.935+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:40.935+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 18415077564616412601 partitionId 0 WriterThreads 4 2023-10-21T04:16:40.935+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 18415077564616412601 PartitionId 0 Stopped Billing WUs 2023-10-21T04:16:40.935+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 18415077564616412601 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index Index: 18415077564616412601 Partition: 0 2023-10-21T04:16:40.935+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 18415077564616412601, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:16:40.935+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 18415077564616412601, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:16:40.935+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18415077564616412601 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:40.936+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 18415077564616412601 2023-10-21T04:16:40.936+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 18415077564616412601 Name: #primary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:40.936+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:40.937+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:40.937+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 18415077564616412601 Name: #primary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:40.937+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 18415077564616412601 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:40.937+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 18415077564616412601 Partitions [0] 2023-10-21T04:16:40.937+05:30 [Info] StorageMgr::openSnapshot IndexInst:18415077564616412601 Partition:0 No Snapshot Found. 2023-10-21T04:16:40.937+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 18415077564616412601 Adding Nil Snapshot. 2023-10-21T04:16:40.947+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.056111ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:40.947+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 70.429µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:40.969+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:40.972+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:40.974+05:30 [Info] switched currmeta from 334 -> 335 force true 2023-10-21T04:16:41.016+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.016+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.021+05:30 [Info] switched currmeta from 335 -> 336 force true 2023-10-21T04:16:41.038+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.039+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.043+05:30 [Info] switched currmeta from 336 -> 337 force true 2023-10-21T04:16:41.065+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.065+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.069+05:30 [Info] switched currmeta from 337 -> 338 force true 2023-10-21T04:16:41.095+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.096+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.099+05:30 [Info] switched currmeta from 338 -> 339 force true 2023-10-21T04:16:41.138+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.140+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.148+05:30 [Info] switched currmeta from 339 -> 341 force true 2023-10-21T04:16:41.170+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.171+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.184+05:30 [Info] switched currmeta from 341 -> 342 force true 2023-10-21T04:16:41.208+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.217+05:30 [Info] switched currmeta from 342 -> 342 force true 2023-10-21T04:16:41.400+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.400+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.403+05:30 [Info] switched currmeta from 342 -> 343 force true 2023-10-21T04:16:41.438+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.438+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.441+05:30 [Info] switched currmeta from 343 -> 344 force true 2023-10-21T04:16:41.622+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.628+05:30 [Info] switched currmeta from 344 -> 345 force true 2023-10-21T04:16:41.654+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.654+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.659+05:30 [Info] switched currmeta from 345 -> 346 force true 2023-10-21T04:16:41.673+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.673+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.677+05:30 [Info] switched currmeta from 346 -> 347 force true 2023-10-21T04:16:41.722+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[962468716067699903:INDEX_STATE_RECOVERED 5744899830895973442:INDEX_STATE_RECOVERED 7582778950442222621:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:41.723+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 9619308266130016313, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:41.723+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 9619308266130016313, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:41.756+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12265503634969032942,realInstId: 0, indexDefn DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:41.757+05:30 [Info] Indexer::handleRecoverIndex InstId: 12265503634969032942 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:41.757+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 168.428µs 2023-10-21T04:16:41.757+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 212.358µs 2023-10-21T04:16:41.757+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12265503634969032942 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:16:41.757+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12265503634969032942 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:16:41.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:41.757+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:16:41.757+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12265503634969032942, indexDefn DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:41.757+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 34.533777ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:41.758+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:16:41.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.767+05:30 [Info] switched currmeta from 347 -> 348 force true 2023-10-21T04:16:41.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 153 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:41.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:41.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:41.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:41.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex started 2023-10-21T04:16:41.769+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12265503634969032942 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:41.769+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12265503634969032942 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:16:41.769+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:16:41.769+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:41.769+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:41.769+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:16:41.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:16:41.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:41.769+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:16:41.769+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:16:41.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:41.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex stopped 2023-10-21T04:16:41.769+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:16:41.769+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:16:41.769+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:16:41.770+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:16:41.770+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:16:41.777+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:16:41.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:16:41.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex closed 2023-10-21T04:16:41.777+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:16:41.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex ... 2023-10-21T04:16:41.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 153 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex ... 2023-10-21T04:16:41.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:41.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index destroyed 2023-10-21T04:16:41.788+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:16:41.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:16:41.789+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:16:41.789+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:16:41.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 184 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:41.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:16:41.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:16:41.794+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:41.794+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:16:41.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex started 2023-10-21T04:16:41.795+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12265503634969032942 PartitionId 0 Warmup took 37.528454ms 2023-10-21T04:16:41.795+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12265503634969032942 partitionId 0 WriterThreads 4 2023-10-21T04:16:41.795+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12265503634969032942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index Index: 12265503634969032942 Partition: 0 2023-10-21T04:16:41.795+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12265503634969032942, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:16:41.795+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12265503634969032942, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:16:41.795+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12265503634969032942 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:41.795+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:41.795+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:41.796+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12265503634969032942 2023-10-21T04:16:41.796+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12265503634969032942 Name: #primary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:41.797+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:41.798+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:41.799+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12265503634969032942 Name: #primary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:41.799+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12265503634969032942 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:41.800+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12265503634969032942 Partitions [0] 2023-10-21T04:16:41.800+05:30 [Info] StorageMgr::openSnapshot IndexInst:12265503634969032942 Partition:0 No Snapshot Found. 2023-10-21T04:16:41.800+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12265503634969032942 Adding Nil Snapshot. 2023-10-21T04:16:41.802+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.707228ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:41.802+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 70.478µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:16:41.816+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.816+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.820+05:30 [Info] switched currmeta from 348 -> 349 force true 2023-10-21T04:16:41.877+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[18415077564616412601:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:41.877+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 18415077564616412601 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00972f260 2023-10-21T04:16:41.878+05:30 [Info] handleUpdateIndexRState: Index instance 18415077564616412601 rstate moved to ACTIVE 2023-10-21T04:16:41.915+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 37.292919ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:41.915+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [249887442666643345 14340086159258068923 5197198800322775808] belonging to collectionId: c, ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c. Initiating build 2023-10-21T04:16:41.940+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.940+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.944+05:30 [Info] switched currmeta from 349 -> 351 force true 2023-10-21T04:16:41.973+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.973+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.975+05:30 [Info] switched currmeta from 351 -> 351 force true 2023-10-21T04:16:41.990+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:41.990+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:41.999+05:30 [Info] switched currmeta from 351 -> 352 force true 2023-10-21T04:16:42.004+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:16:42.009+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [14975131556821987100 17357232428819395534 1265729768899070816] &{2} 2023-10-21T04:16:42.009+05:30 [Info] Indexer::handleBuildRecoveredIndexes [14975131556821987100 17357232428819395534 1265729768899070816] 2023-10-21T04:16:42.009+05:30 [Info] bulkUpdateRState: Index instance 14975131556821987100 rstate moved to PENDING 2023-10-21T04:16:42.009+05:30 [Info] bulkUpdateRState: Index instance 17357232428819395534 rstate moved to PENDING 2023-10-21T04:16:42.011+05:30 [Info] bulkUpdateRState: Index instance 1265729768899070816 rstate moved to PENDING 2023-10-21T04:16:42.011+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [14975131556821987100 17357232428819395534 1265729768899070816] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:42.012+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:42.019+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:42.022+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:42.046+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.047+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.049+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:16:42.050+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:42.050+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:42.051+05:30 [Info] switched currmeta from 352 -> 353 force true 2023-10-21T04:16:42.051+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:42.053+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:16:42.053+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:42.053+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:42.053+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:42.053+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:16:42.053+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:42.053+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:42.053+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:16:42.053+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:16:42.054+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId 79187 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:42.054+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:42.054+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:42.055+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:16:42.055+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:16:42.055+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 1265729768899070816 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:42.055+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:2]] 2023-10-21T04:16:42.055+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 14975131556821987100 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:42.055+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:6 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:3]] 2023-10-21T04:16:42.055+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17357232428819395534 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:42.055+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:16:42.055+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: true SessionId: 79187 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:16:42.056+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:16:42.056+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:16:42.057+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:42.058+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 49.398533ms 2023-10-21T04:16:42.058+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [14975131556821987100 17357232428819395534 1265729768899070816] 2023-10-21T04:16:42.059+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 141.476399ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.058+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 49.642865ms 2023-10-21T04:16:42.062+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[1265729768899070816:true 14975131556821987100:true 17357232428819395534:true] 2023-10-21T04:16:42.062+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:16:42.065+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:16:42.076+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.925135ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.102+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 26.212713ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.115+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.444292ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.142+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48876" doAuth() ... 2023-10-21T04:16:42.143+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48876" auth successful 2023-10-21T04:16:42.146+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:48876" +1 2023-10-21T04:16:42.162+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.162+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.164+05:30 [Info] switched currmeta from 353 -> 356 force true 2023-10-21T04:16:42.178+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.179+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.186+05:30 [Info] switched currmeta from 356 -> 356 force true 2023-10-21T04:16:42.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.384+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.387+05:30 [Info] switched currmeta from 356 -> 357 force true 2023-10-21T04:16:42.477+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.477+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.481+05:30 [Info] switched currmeta from 357 -> 358 force true 2023-10-21T04:16:42.511+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.511+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.519+05:30 [Info] switched currmeta from 358 -> 359 force true 2023-10-21T04:16:42.550+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 263268055970459 52 79187 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:42.553+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 215749515705641 45 79187 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:42.553+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 35403626585217 50 79187 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:42.556+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 159113185399719 58 79187 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:42.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 261493429196201 61 79187 172.31.5.112:9000. HWT [60-61,61]. 2023-10-21T04:16:42.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 3874519919278 54 79187 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:42.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 176639880977228 61 79187 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:42.559+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 170288557775498 52 79187 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:42.561+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 264975169139761 56 79187 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:42.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 236398693961229 62 79187 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:42.568+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 234758907181738 56 79187 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:42.570+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 116104661063478 55 79187 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:42.575+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 102350060311343 53 79187 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:42.575+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 140236896721046 60 79187 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:42.575+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 206533515052028 50 79187 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:42.576+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 162158720181435 67 79187 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:42.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 200752780087280 54 79187 172.31.5.112:9000. HWT [49-54,54]. 2023-10-21T04:16:42.582+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 98820809009123 44 79187 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:16:42.582+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 207134287529256 46 79187 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:42.584+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 208836052671833 45 79187 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:42.586+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 92956357968546 50 79187 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:42.588+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 27979933591532 54 79187 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:42.590+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 154934968850942 51 79187 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:42.590+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 192215088742243 55 79187 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:42.593+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 271667630317209 52 79187 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:42.596+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 120989102157727 59 79187 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:42.596+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 168740244208559 61 79187 172.31.5.112:9000. HWT [59-61,61]. 2023-10-21T04:16:42.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5715487545729 54 79187 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:42.615+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 43594649874772 52 79187 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:42.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 73432771436458 60 79187 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:42.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 236953356346174 54 79187 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:42.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 60545336618717 46 79187 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:42.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 51776324008509 60 79187 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:42.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 67362620867705 67 79187 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:42.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 24368176313355 50 79187 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:42.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 181267443431019 53 79187 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:42.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 83647976044922 59 79187 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:42.626+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 130231725986856 49 79187 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:42.626+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 278426329668882 54 79187 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:42.627+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 100659545510465 49 79187 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:16:42.629+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 45479338375128 55 79187 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:42.630+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 153478150002150 68 79187 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:16:42.630+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 249919763471169 66 79187 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:42.635+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 116785907755771 55 79187 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:42.635+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 166675095997741 57 79187 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:42.648+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 49796766757346 61 79187 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:42.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.667+05:30 [Info] switched currmeta from 359 -> 360 force true 2023-10-21T04:16:42.681+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.681+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.685+05:30 [Info] switched currmeta from 360 -> 362 force true 2023-10-21T04:16:42.689+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 124194574848641 54 79187 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:42.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 158076890855634 54 79187 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:42.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 196777743741606 51 79187 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:42.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 138935694816848 61 79187 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:42.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 149022976193937 55 79187 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:42.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 19274425379955 55 79187 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:42.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 184487117398647 55 79187 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:42.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 199728167534433 66 79187 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:42.705+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 61798873988101 67 79187 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:42.706+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 24856128127173 60 79187 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:42.715+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 29071415952852 61 79187 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:42.728+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 191719959781498 63 79187 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:42.728+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 56626109228904 46 79187 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:42.728+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.728+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.733+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 140942236111856 63 79187 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:42.733+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 103336286678362 46 79187 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:42.742+05:30 [Info] switched currmeta from 362 -> 362 force true 2023-10-21T04:16:42.743+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 252537041863196 73 79187 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:16:42.750+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 177105814270799 58 79187 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:42.750+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 119871837460216 39 79187 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:16:42.767+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c1 InstanceIds [1265729768899070816 14975131556821987100 17357232428819395534] 2023-10-21T04:16:42.767+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 79187 2023-10-21T04:16:42.767+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79187 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 1265729768899070816 INDEX_STATE_CATCHUP 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 14975131556821987100 INDEX_STATE_CATCHUP 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 17357232428819395534 INDEX_STATE_CATCHUP 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 17357232428819395534 true 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 1265729768899070816 true 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 14975131556821987100 true 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 14975131556821987100 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 79187 BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:16:42.767+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:16:42.768+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79187 STREAM_ACTIVE 2023-10-21T04:16:42.768+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 79187 2023-10-21T04:16:42.768+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 79187 2023-10-21T04:16:42.768+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:42.769+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:42.769+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:16:42.769+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:16:42.769+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:42.769+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:42.770+05:30 [Info] send progress stats to clients 2023-10-21T04:16:42.770+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:16:42.770+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:16:42.770+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:7 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:3]] 2023-10-21T04:16:42.770+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:8 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:3]] 2023-10-21T04:16:42.770+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:3]] 2023-10-21T04:16:42.770+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 79187 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:16:42.776+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[12265503634969032942:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:16:42.776+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12265503634969032942 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc004c9a1e0 2023-10-21T04:16:42.777+05:30 [Info] handleUpdateIndexRState: Index instance 12265503634969032942 rstate moved to ACTIVE 2023-10-21T04:16:42.779+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:16:42.787+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 2023-10-21T04:16:42.787+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79187 2023-10-21T04:16:42.787+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79187 2023-10-21T04:16:42.787+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 1265729768899070816 false 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 14975131556821987100 false 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 17357232428819395534 false 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 17357232428819395534 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 1265729768899070816 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 14975131556821987100 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:16:42.788+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 79187 2023-10-21T04:16:42.788+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:42.802+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 26.415409ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.815+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.815+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.819+05:30 [Info] switched currmeta from 362 -> 363 force true 2023-10-21T04:16:42.846+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 43.245414ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.859+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.519977ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.870+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.999456ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:42.873+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [14026476026078401783 12140458060295645238 5103684059183126577] belonging to collectionId: d, ttid: ShardTokena7:37:e9:2a:6a:83:12:d0. Initiating build 2023-10-21T04:16:42.892+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.892+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.897+05:30 [Info] switched currmeta from 363 -> 366 force true 2023-10-21T04:16:42.957+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:16:42.957+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:16:42.959+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. err: operation_canceled 2023-10-21T04:16:42.959+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:16:42.959+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/cf56ae547f07d182689428d60f3e89c9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:cf56ae547f07d182689428d60f3e89c9]}]} 2023-10-21T04:16:42.960+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.960+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:42.965+05:30 [Info] switched currmeta from 366 -> 367 force true 2023-10-21T04:16:42.966+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:16:42.966+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:16:42.978+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:42.978+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.013+05:30 [Info] switched currmeta from 367 -> 369 force true 2023-10-21T04:16:43.021+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.021+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.024+05:30 [Info] switched currmeta from 369 -> 369 force true 2023-10-21T04:16:43.039+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [5744899830895973442 962468716067699903 7582778950442222621] &{2} 2023-10-21T04:16:43.040+05:30 [Info] Indexer::handleBuildRecoveredIndexes [5744899830895973442 962468716067699903 7582778950442222621] 2023-10-21T04:16:43.040+05:30 [Info] bulkUpdateRState: Index instance 5744899830895973442 rstate moved to PENDING 2023-10-21T04:16:43.040+05:30 [Info] bulkUpdateRState: Index instance 962468716067699903 rstate moved to PENDING 2023-10-21T04:16:43.040+05:30 [Info] bulkUpdateRState: Index instance 7582778950442222621 rstate moved to PENDING 2023-10-21T04:16:43.040+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [5744899830895973442 962468716067699903 7582778950442222621] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:43.040+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:43.041+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:43.041+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:43.059+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.059+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.064+05:30 [Info] switched currmeta from 369 -> 370 force true 2023-10-21T04:16:43.138+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.138+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.141+05:30 [Info] switched currmeta from 370 -> 371 force true 2023-10-21T04:16:43.161+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:16:43.161+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.161+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:43.162+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.162+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:43.162+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:43.162+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:16:43.162+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:16:43.162+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:43.162+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:43.162+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc00042c480 }] 2023-10-21T04:16:43.162+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.162+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:43.162+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.162+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:43.162+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c2% reset bucket state to NIL 2023-10-21T04:16:43.162+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:16:43.162+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% SessionId 95346 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:43.190+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.177+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.190+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.190+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:43.191+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.191+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:43.191+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c2% Added for Stream INIT_STREAM 2023-10-21T04:16:43.191+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_default:c2%:1]] 2023-10-21T04:16:43.192+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 5744899830895973442 stream INIT_STREAM keyspaceId bucket_1:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:43.192+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_default:c2%:2]] 2023-10-21T04:16:43.192+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 962468716067699903 stream INIT_STREAM keyspaceId bucket_1:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:43.192+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:7] INIT_STREAM:map[bucket_%2:_default:c1:3 bucket_1:_default:c2%:3]] 2023-10-21T04:16:43.192+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 7582778950442222621 stream INIT_STREAM keyspaceId bucket_1:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:43.193+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:16:43.194+05:30 [Info] switched currmeta from 371 -> 373 force true 2023-10-21T04:16:43.194+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 95346 CollectionId: d CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:16:43.197+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc00042c480 } bucket_1:_default:c2%:{0xc000493b00 }] 2023-10-21T04:16:43.199+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.199+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.201+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c2% Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:16:43.202+05:30 [Info] switched currmeta from 373 -> 373 force true 2023-10-21T04:16:43.205+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c2% Check pending stream merge. 2023-10-21T04:16:43.224+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.225+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.229+05:30 [Info] switched currmeta from 373 -> 374 force true 2023-10-21T04:16:43.197+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:43.231+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 191.319327ms 2023-10-21T04:16:43.234+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [5744899830895973442 962468716067699903 7582778950442222621] 2023-10-21T04:16:43.236+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 362.678501ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.234+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 194.10644ms 2023-10-21T04:16:43.238+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[962468716067699903:true 5744899830895973442:true 7582778950442222621:true] 2023-10-21T04:16:43.254+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.628123ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.293+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.293+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.297+05:30 [Info] switched currmeta from 374 -> 376 force true 2023-10-21T04:16:43.307+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.307+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.309+05:30 [Info] switched currmeta from 376 -> 376 force true 2023-10-21T04:16:43.314+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 60.035983ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.331+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.332+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.338+05:30 [Info] switched currmeta from 376 -> 377 force true 2023-10-21T04:16:43.349+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 29.801131ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.367+05:30 [Info] switched currmeta from 377 -> 378 force true 2023-10-21T04:16:43.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3131348393020297642:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:43.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3131348393020297642:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:43.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3131348393020297642:[0] 3593050551190633737:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:43.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3131348393020297642:[0] 3593050551190633737:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:43.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3131348393020297642:[0] 3593050551190633737:[3 4 5 6 7 8 1 2] 10590902705185414921:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:43.564+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.564+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.567+05:30 [Info] switched currmeta from 378 -> 379 force true 2023-10-21T04:16:43.615+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.616+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.620+05:30 [Info] switched currmeta from 379 -> 380 force true 2023-10-21T04:16:43.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 0 186517626268899 55 95346 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:43.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 1 178840478062605 57 95346 172.31.5.112:9000. HWT [54-57,57]. 2023-10-21T04:16:43.643+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 2 148075163102199 54 95346 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:43.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 3 215374859564337 35 95346 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:16:43.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 4 28605411869539 58 95346 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:43.649+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 5 39552632604610 52 95346 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:43.651+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 6 242327631330054 62 95346 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:43.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 8 50961953886801 45 95346 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:43.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 7 191903564840858 63 95346 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:43.658+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 9 225280996447309 55 95346 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:16:43.663+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 10 274410200699189 54 95346 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:43.667+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 12 4429827396609 42 95346 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:16:43.667+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 11 66881023717063 51 95346 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:43.669+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 13 183148377680315 46 95346 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:43.670+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 14 226814108379280 72 95346 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:16:43.673+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 15 127185710837163 50 95346 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:43.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 16 257031483891250 58 95346 172.31.5.112:9000. HWT [55-58,58]. 2023-10-21T04:16:43.677+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 17 55290066231400 51 95346 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:43.679+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 19 210306088158507 48 95346 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:16:43.679+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 18 242578104468179 63 95346 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:43.682+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 20 230239359684242 50 95346 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:43.686+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 22 260036488774725 51 95346 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:43.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 23 216462437971699 57 95346 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:43.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 21 209771427122473 48 95346 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:16:43.690+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 24 160961818008256 59 95346 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:43.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 28 102578907139247 58 95346 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:43.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 26 219132834127088 52 95346 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:43.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 25 280874278672777 71 95346 172.31.5.112:9000. HWT [71-71,71]. 2023-10-21T04:16:43.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 27 184056281482149 49 95346 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:43.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 29 45957426777107 57 95346 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:43.702+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 30 99433130332778 58 95346 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:43.705+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 31 217381278987664 67 95346 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:43.707+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 32 82469662608686 60 95346 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:43.709+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 33 136958029391999 65 95346 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:16:43.713+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 35 70213628416850 47 95346 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:16:43.714+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 34 28417669987133 58 95346 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:43.719+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 36 253845164956592 58 95346 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:43.719+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 37 253494845579900 54 95346 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:43.721+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 38 188107826286720 60 95346 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:43.733+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 39 222555662236724 67 95346 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:43.735+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 40 5915864305121 60 95346 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:43.740+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 43 146267938201125 55 95346 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:43.741+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 42 252931432434966 46 95346 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:43.741+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 41 259381390638966 58 95346 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:43.743+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 45 12301831552869 66 95346 172.31.5.112:9000. HWT [63-66,66]. 2023-10-21T04:16:43.744+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 44 158545118983293 67 95346 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:43.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 46 242994859770444 54 95346 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:43.747+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 47 271074273345339 53 95346 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:43.750+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 48 227378777358533 59 95346 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:16:43.752+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 49 215467668600630 34 95346 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:16:43.754+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 50 207367350302991 64 95346 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:16:43.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 51 200436537417141 54 95346 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:43.759+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 52 188718367400378 56 95346 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:43.762+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 53 204867593700407 56 95346 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:43.762+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 54 279362078165491 55 95346 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:43.763+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 55 170056388266420 70 95346 172.31.5.112:9000. HWT [70-70,70]. 2023-10-21T04:16:43.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 56 264835102027079 53 95346 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:16:43.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 57 129411666769864 55 95346 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:43.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 58 16044440367455 53 95346 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:43.770+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:16:43.771+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:16:43.773+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 59 125009072873642 53 95346 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:43.773+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 1265729768899070816 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 79187, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:43.774+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 1265729768899070816 INDEX_STATE_ACTIVE 2023-10-21T04:16:43.774+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 14975131556821987100 INDEX_STATE_ACTIVE 2023-10-21T04:16:43.774+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 17357232428819395534 INDEX_STATE_ACTIVE 2023-10-21T04:16:43.774+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:16:43.774+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:16:43.774+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:43.774+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:43.774+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 79187 2023-10-21T04:16:43.776+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:43.777+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:43.778+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:16:43.779+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79187 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:43.779+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 14975131556821987100 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:16:43.779+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17357232428819395534 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:16:43.779+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 1265729768899070816 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:16:43.779+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:16:43.779+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:16:43.779+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:43.779+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79187 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:43.779+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_default:c2%:{0xc000493b00 }] 2023-10-21T04:16:43.780+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.780+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:43.780+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.780+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:43.780+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.780+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:43.780+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:43.780+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:16:43.780+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:16:43.781+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c2% InstanceIds [1265729768899070816 14975131556821987100 17357232428819395534 5744899830895973442 962468716067699903 7582778950442222621] 2023-10-21T04:16:43.781+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% Cid d SessionId 95346 2023-10-21T04:16:43.781+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:43.781+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 79187 2023-10-21T04:16:43.781+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 60 75566139929862 56 95346 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:43.783+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:16:43.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 61 106932309485550 53 95346 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:43.785+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 62 10527297481171 57 95346 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:43.785+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95346 2023-10-21T04:16:43.786+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:16:43.786+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:16:43.786+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 5744899830895973442 INDEX_STATE_CATCHUP 2023-10-21T04:16:43.786+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 962468716067699903 INDEX_STATE_CATCHUP 2023-10-21T04:16:43.787+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 7582778950442222621 INDEX_STATE_CATCHUP 2023-10-21T04:16:43.787+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 5744899830895973442 true 2023-10-21T04:16:43.787+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 962468716067699903 true 2023-10-21T04:16:43.788+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 7582778950442222621 true 2023-10-21T04:16:43.788+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 5744899830895973442 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% Session: 95346 BuildTS: [0=55,1=57,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=55,10=54,11=51,12=42,13=46,14=72,15=50,16=58,17=51,18=63,19=48,20=50,21=48,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=57,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=60,41=58,42=46,43=55,44=67,45=66,46=54,47=53,48=59,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=53,57=55,58=53,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:16:43.788+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c2%, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:16:43.790+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:16:43.790+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:16:43.790+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:16:43.791+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95346 STREAM_ACTIVE 2023-10-21T04:16:43.791+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 63 133748675726909 56 95346 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:43.791+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [0=55,1=53,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=68,15=50,16=55,17=51,18=62,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=56,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=57,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=52,59=53,60=56,61=53,62=57,63=56] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:43.791+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId: 95346 2023-10-21T04:16:43.791+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c2% 95346 2023-10-21T04:16:43.792+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:43.792+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:43.793+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:43.793+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:43.795+05:30 [Info] send progress stats to clients 2023-10-21T04:16:43.795+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c2% set minMergeTs 2023-10-21T04:16:43.795+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c2% StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:16:43.795+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:8] INIT_STREAM:map[bucket_1:_default:c2%:3]] 2023-10-21T04:16:43.795+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:9] INIT_STREAM:map[bucket_1:_default:c2%:3]] 2023-10-21T04:16:43.795+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:10] INIT_STREAM:map[bucket_1:_default:c2%:3]] 2023-10-21T04:16:43.795+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 95346 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:16:43.795+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 8 264975169139761 56 2023-10-21T04:16:43.795+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.795+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 0 263268055970459 52 2023-10-21T04:16:43.795+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.795+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 1 176639880977228 61 2023-10-21T04:16:43.795+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.795+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 2 35403626585217 50 2023-10-21T04:16:43.795+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.796+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 3 215749515705641 45 2023-10-21T04:16:43.796+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.796+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 11 234758907181738 56 2023-10-21T04:16:43.796+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.796+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 4 159113185399719 58 2023-10-21T04:16:43.796+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.796+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 9 236398693961229 62 2023-10-21T04:16:43.796+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.796+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 10 206533515052028 50 2023-10-21T04:16:43.796+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.796+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 6 261493429196201 61 2023-10-21T04:16:43.796+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.796+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 5 3874519919278 54 2023-10-21T04:16:43.797+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.797+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 7 170288557775498 52 2023-10-21T04:16:43.798+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.798+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 19 208836052671833 45 2023-10-21T04:16:43.798+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.798+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 51 196777743741606 51 2023-10-21T04:16:43.798+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.798+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 24 271667630317209 52 2023-10-21T04:16:43.798+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.798+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 56 29071415952852 61 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 32 67362620867705 67 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 54 24856128127173 60 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 22 192215088742243 55 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 63 252537041863196 73 2023-10-21T04:16:43.799+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79187 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 31 73432771436458 60 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 20 92956357968546 50 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 52 158076890855634 54 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 57 191719959781498 63 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 25 168740244208559 61 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 13 102350060311343 53 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 45 49796766757346 61 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 27 5715487545729 54 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 59 140942236111856 63 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 46 124194574848641 54 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 14 140236896721046 60 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 48 19274425379955 55 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.799+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 16 200752780087280 54 2023-10-21T04:16:43.799+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 17 98820809009123 44 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 49 184487117398647 55 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 18 207134287529256 46 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 50 199728167534433 66 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 36 43594649874772 52 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 37 83647976044922 59 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 38 181267443431019 53 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 55 61798873988101 67 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 23 154934968850942 51 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 60 56626109228904 46 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 28 236953356346174 54 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 12 116104661063478 55 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 44 166675095997741 57 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 26 120989102157727 59 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 58 103336286678362 46 2023-10-21T04:16:43.800+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.800+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 35 24368176313355 50 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 40 153478150002150 68 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 47 149022976193937 55 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 15 162158720181435 67 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 53 138935694816848 61 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 21 27979933591532 54 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 62 177105814270799 58 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 30 60545336618717 46 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 33 278426329668882 54 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 41 116785907755771 55 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 42 45479338375128 55 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 29 130231725986856 49 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 61 119871837460216 39 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 39 100659545510465 49 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 34 51776324008509 60 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 43 249919763471169 66 2023-10-21T04:16:43.801+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:16:43.801+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79187 2023-10-21T04:16:43.801+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:16:43.803+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:16:43.803+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.778709ms len(expediates) 5 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.811+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.735703ms len(expediates) 4 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.843+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.843+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.854+05:30 [Info] switched currmeta from 380 -> 382 force true 2023-10-21T04:16:43.855+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 43.943816ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.858+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 2023-10-21T04:16:43.858+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95346 2023-10-21T04:16:43.862+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95346 2023-10-21T04:16:43.862+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c2% set minMergeTs 2023-10-21T04:16:43.862+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:16:43.863+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 962468716067699903 false 2023-10-21T04:16:43.864+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 7582778950442222621 false 2023-10-21T04:16:43.864+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 5744899830895973442 false 2023-10-21T04:16:43.864+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c2% 5744899830895973442 2023-10-21T04:16:43.864+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c2% 962468716067699903 2023-10-21T04:16:43.863+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.257015ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.870+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c2% 7582778950442222621 2023-10-21T04:16:43.872+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c2%. 2023-10-21T04:16:43.872+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 95346 2023-10-21T04:16:43.876+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.876+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.882+05:30 [Info] switched currmeta from 382 -> 384 force true 2023-10-21T04:16:43.909+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:43.911+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 42.177663ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.916+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.918+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.930+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.764611ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:43.933+05:30 [Info] switched currmeta from 384 -> 385 force true 2023-10-21T04:16:43.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.944+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.952+05:30 [Info] switched currmeta from 385 -> 386 force true 2023-10-21T04:16:43.961+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.962+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.964+05:30 [Info] switched currmeta from 386 -> 386 force true 2023-10-21T04:16:43.969+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:43.969+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:43.972+05:30 [Info] switched currmeta from 386 -> 386 force true 2023-10-21T04:16:43.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9612572573023088, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:16:44.132+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00943cae0 2023-10-21T04:16:44.132+05:30 [Info] handleUpdateIndexRState: Index instance 14975131556821987100 rstate moved to ACTIVE 2023-10-21T04:16:44.138+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.58405ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.138+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1265729768899070816 Defn: DefnId: 5197198800322775808 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00922a5a0 2023-10-21T04:16:44.138+05:30 [Info] handleUpdateIndexRState: Index instance 1265729768899070816 rstate moved to ACTIVE 2023-10-21T04:16:44.139+05:30 [Info] Indexer::handleMergePartition Source 15841249126653343762 Target 17357232428819395534 2023-10-21T04:16:44.139+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00c111e60 2023-10-21T04:16:44.139+05:30 [Info] MergePartition: sent async request to update index instance 17357232428819395534 rstate moved to ACTIVE 2023-10-21T04:16:44.145+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.145+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.147+05:30 [Info] switched currmeta from 386 -> 387 force true 2023-10-21T04:16:44.158+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.767477ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.159+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 4992762551574016313. Retrying... 2023-10-21T04:16:44.159+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[1265729768899070816:true 14975131556821987100:true 17357232428819395534:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:44.165+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.165+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.168+05:30 [Info] switched currmeta from 387 -> 388 force true 2023-10-21T04:16:44.182+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 24.028352ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.183+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 15841249126653343762, realInst: 17357232428819395534 2023-10-21T04:16:44.184+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 3674078063353481907, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:44.184+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 3674078063353481907, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:16:44.189+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.189+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.194+05:30 [Info] switched currmeta from 388 -> 389 force true 2023-10-21T04:16:44.220+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4992762551574016313,realInstId: 0, indexDefn DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:16:44.220+05:30 [Info] Indexer::handleRecoverIndex InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:16:44.220+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 147.82µs 2023-10-21T04:16:44.220+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 188.735µs 2023-10-21T04:16:44.220+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4992762551574016313 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:16:44.220+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4992762551574016313, indexDefn DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:16:44.220+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 36.48041ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.221+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4992762551574016313 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:16:44.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:16:44.221+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 64 2023-10-21T04:16:44.221+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 720010 count: 21 metaSz: 2545 rp [1] sn: 1080007 count: 20 metaSz: 2546 2023-10-21T04:16:44.222+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: doInit: data UsedSpace 118172 recovery UsedSpace 182548 2023-10-21T04:16:44.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:16:44.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:16:44.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:16:44.232+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:16:44.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex started 2023-10-21T04:16:44.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:16:44.234+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:16:44.234+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:16:44.235+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.235+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.240+05:30 [Info] switched currmeta from 389 -> 390 force true 2023-10-21T04:16:44.259+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Warmup took 38.591578ms 2023-10-21T04:16:44.259+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4992762551574016313 partitionId 0 WriterThreads 4 2023-10-21T04:16:44.259+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4992762551574016313 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index Index: 4992762551574016313 Partition: 0 2023-10-21T04:16:44.259+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4992762551574016313, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:16:44.259+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4992762551574016313, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:16:44.259+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:44.260+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4992762551574016313 2023-10-21T04:16:44.260+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:44.260+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:16:44.261+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:44.261+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:44.262+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:44.262+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4992762551574016313 Partitions [0] 2023-10-21T04:16:44.263+05:30 [Info] StorageMgr::openSnapshot IndexInst:4992762551574016313 Partition:0 Attempting to open snapshot (SnapshotInfo: count:20 committed:false) 2023-10-21T04:16:44.282+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.282+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.291+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 30.941593ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.290+05:30 [Info] switched currmeta from 390 -> 392 force true 2023-10-21T04:16:44.301+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 82.243µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.318+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.318+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.321+05:30 [Info] switched currmeta from 392 -> 393 force true 2023-10-21T04:16:44.584+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.584+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.588+05:30 [Info] switched currmeta from 393 -> 394 force true 2023-10-21T04:16:44.660+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.660+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.663+05:30 [Info] switched currmeta from 394 -> 395 force true 2023-10-21T04:16:44.791+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c2% StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c2% 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 5744899830895973442 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% SessionId: 95346, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 55 55} { 1 a2a78b2a980d 3 57 54 57} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 52 52 52} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 54 54 54} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 46 46 46} { 14 ce49465c2890 3 72 72 72} { 15 73acbb1d6dab 3 50 50 50} { 16 e9c4ce4b1e32 3 58 55 58} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 57 57 57} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 71 71 71} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 49 49 49} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 65 65 65} { 34 19d8811ffb3d 3 58 58 58} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 67 67 67} { 40 56164fd8de1 3 60 60 60} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 66 63 66} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 59 57 59} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 70 70 70} { 56 f0ddba2e2d47 3 53 52 53} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 53 53 53} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 56 56 56} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 5744899830895973442 INDEX_STATE_ACTIVE 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 962468716067699903 INDEX_STATE_ACTIVE 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 7582778950442222621 INDEX_STATE_ACTIVE 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% State Changed to INACTIVE 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:16:44.791+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:16:44.791+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 1 New 0 2023-10-21T04:16:44.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 14975131556821987100:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 14975131556821987100:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 14975131556821987100:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 14975131556821987100:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 14975131556821987100:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 14975131556821987100:[0] 17357232428819395534:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 14975131556821987100:[0] 17357232428819395534:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:44.791+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId: 95346 2023-10-21T04:16:44.793+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:44.793+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:44.793+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 0 2023-10-21T04:16:44.794+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 95346 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:44.794+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 5744899830895973442 from stream INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:16:44.794+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 962468716067699903 from stream INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:16:44.794+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 7582778950442222621 from stream INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:16:44.794+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:16:44.794+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:16:44.794+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:44.794+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 95346 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:44.794+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:16:44.795+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c2% total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:44.795+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c2% latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:44.796+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:16:44.796+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:48876: use of closed network connection 2023-10-21T04:16:44.796+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:48876" exit: read tcp4 127.0.0.1:9121->127.0.0.1:48876: use of closed network connection 2023-10-21T04:16:44.796+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:48876" closed ! 2023-10-21T04:16:44.796+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:16:44.796+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:44.796+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:44.797+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:44.797+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:44.797+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:16:44.797+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:44.797+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId 95346 2023-10-21T04:16:44.799+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c2%] 2023-10-21T04:16:44.815+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c2%]. Err genServer.closed 2023-10-21T04:16:44.815+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c2%] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:16:44.815+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c2%] Treating genServer.closed As Success 2023-10-21T04:16:44.815+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95346 2023-10-21T04:16:44.815+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95346 2023-10-21T04:16:44.815+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:16:44.833+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 35.948801ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.860+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.860+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.862+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 28.895919ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.867+05:30 [Info] switched currmeta from 395 -> 396 force true 2023-10-21T04:16:44.872+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.588297ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:44.896+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.897+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.921+05:30 [Info] switched currmeta from 396 -> 398 force true 2023-10-21T04:16:44.955+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:44.955+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:44.968+05:30 [Info] switched currmeta from 398 -> 398 force true 2023-10-21T04:16:44.992+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8260988490331769, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:16:45.242+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4992762551574016313:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:16:45.242+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [3674078063353481907] belonging to collectionId: 9, ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c. Initiating build 2023-10-21T04:16:45.253+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [4992762551574016313] &{2} 2023-10-21T04:16:45.254+05:30 [Info] Indexer::handleBuildRecoveredIndexes [4992762551574016313] 2023-10-21T04:16:45.255+05:30 [Info] bulkUpdateRState: Index instance 4992762551574016313 rstate moved to PENDING 2023-10-21T04:16:45.256+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [4992762551574016313] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:16:45.257+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:45.259+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:45.259+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:45.263+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:16:45.264+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:45.265+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:45.268+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:45.270+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:16:45.270+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:16:45.271+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:45.272+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:45.274+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:16:45.274+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:45.273+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.276+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.275+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:45.277+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_system:_query reset bucket state to NIL 2023-10-21T04:16:45.277+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:16:45.277+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId 30399 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:45.277+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:45.277+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:45.277+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:16:45.278+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:9 bucket_1:10] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:16:45.278+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4992762551574016313 stream INIT_STREAM keyspaceId bucket_%2:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:16:45.278+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:16:45.278+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 30399 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 44 44 44} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} NumVBuckets: 64 2023-10-21T04:16:45.278+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:16:45.278+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:16:45.279+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:45.279+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 24.438344ms 2023-10-21T04:16:45.279+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 24.480444ms 2023-10-21T04:16:45.279+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [4992762551574016313] 2023-10-21T04:16:45.279+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 35.878961ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:45.281+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4992762551574016313:true] 2023-10-21T04:16:45.288+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_system:_query Check pending stream merge. 2023-10-21T04:16:45.290+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.636716ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:45.302+05:30 [Info] switched currmeta from 398 -> 400 force true 2023-10-21T04:16:45.302+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00529b080 2023-10-21T04:16:45.306+05:30 [Info] handleUpdateIndexRState: Index instance 962468716067699903 rstate moved to ACTIVE 2023-10-21T04:16:45.306+05:30 [Info] Indexer::handleMergePartition Source 11368506836388887403 Target 5744899830895973442 2023-10-21T04:16:45.306+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00eaa56e0 2023-10-21T04:16:45.307+05:30 [Info] MergePartition: sent async request to update index instance 5744899830895973442 rstate moved to ACTIVE 2023-10-21T04:16:45.323+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.88537ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:45.324+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7582778950442222621 Defn: DefnId: 5103684059183126577 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00a16a6c0 2023-10-21T04:16:45.324+05:30 [Info] handleUpdateIndexRState: Index instance 7582778950442222621 rstate moved to ACTIVE 2023-10-21T04:16:45.327+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.328+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.359+05:30 [Info] switched currmeta from 400 -> 401 force true 2023-10-21T04:16:45.363+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 39.603723ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:45.365+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 11368506836388887403, realInst: 5744899830895973442 2023-10-21T04:16:45.369+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.47717ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:45.369+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[962468716067699903:true 5744899830895973442:true 7582778950442222621:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:45.371+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardTokena7:37:e9:2a:6a:83:12:d0, elapsed(sec): 29.68536245 2023-10-21T04:16:45.382+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.382+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.405+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:45.407+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardTokena7:37:e9:2a:6a:83:12:d0 2023-10-21T04:16:45.416+05:30 [Info] switched currmeta from 401 -> 403 force true 2023-10-21T04:16:45.535+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.535+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.567+05:30 [Info] switched currmeta from 403 -> 403 force true 2023-10-21T04:16:45.638+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.638+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.644+05:30 [Info] switched currmeta from 403 -> 405 force true 2023-10-21T04:16:45.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.661+05:30 [Info] switched currmeta from 405 -> 406 force true 2023-10-21T04:16:45.680+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.680+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.697+05:30 [Info] switched currmeta from 406 -> 407 force true 2023-10-21T04:16:45.730+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.730+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.745+05:30 [Info] switched currmeta from 407 -> 407 force true 2023-10-21T04:16:45.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.903+05:30 [Info] switched currmeta from 407 -> 408 force true 2023-10-21T04:16:45.957+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:45.957+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:45.985+05:30 [Info] switched currmeta from 408 -> 410 force true 2023-10-21T04:16:46.005+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:46.005+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:46.008+05:30 [Info] switched currmeta from 410 -> 410 force true 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[962468716067699903:[0] 5744899830895973442:[5 6 7 8 1 2 3 4] 7582778950442222621:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:16:46.670+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:46.670+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:46.673+05:30 [Info] switched currmeta from 410 -> 411 force true 2023-10-21T04:16:47.054+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:47.054+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:47.057+05:30 [Info] switched currmeta from 411 -> 412 force true 2023-10-21T04:16:47.096+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:47.096+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:47.100+05:30 [Info] switched currmeta from 412 -> 414 force true 2023-10-21T04:16:47.108+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:47.108+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:47.133+05:30 [Info] switched currmeta from 414 -> 414 force true 2023-10-21T04:16:47.283+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:16:47.296+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:49200" doAuth() ... 2023-10-21T04:16:47.297+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:49200" auth successful 2023-10-21T04:16:47.298+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:49200" +1 2023-10-21T04:16:47.541+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 0 263268055970459 52 30399 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:47.551+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 1 176639880977228 61 30399 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:47.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 2 35403626585217 50 30399 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:47.559+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 4 159113185399719 58 30399 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:47.560+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 3 215749515705641 45 30399 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:47.563+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 5 3874519919278 54 30399 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:47.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 6 261493429196201 61 30399 172.31.5.112:9000. HWT [60-61,61]. 2023-10-21T04:16:47.567+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 7 170288557775498 52 30399 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:47.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 9 236398693961229 62 30399 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:16:47.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 8 264975169139761 56 30399 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:47.576+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 10 206533515052028 50 30399 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:47.583+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 11 234758907181738 56 30399 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:16:47.586+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 13 102350060311343 53 30399 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:47.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 14 140236896721046 60 30399 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:47.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 12 116104661063478 55 30399 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:47.592+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 16 200752780087280 54 30399 172.31.5.112:9000. HWT [49-54,54]. 2023-10-21T04:16:47.592+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 15 162158720181435 67 30399 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:47.595+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 18 207134287529256 46 30399 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:47.595+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 17 98820809009123 44 30399 172.31.5.112:9000. HWT [44-44,44]. 2023-10-21T04:16:47.598+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_system:_query setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:16:47.598+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 20 92956357968546 50 30399 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:47.598+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 19 208836052671833 45 30399 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:16:47.599+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:16:47.599+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4992762551574016313 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:21 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.15µs SnapOpenDur 117.944µs 2023-10-21T04:16:47.600+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:16:47.602+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 22 192215088742243 55 30399 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:47.602+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 21 27979933591532 54 30399 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:47.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 23 154934968850942 51 30399 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:47.607+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 25 168740244208559 61 30399 172.31.5.112:9000. HWT [59-61,61]. 2023-10-21T04:16:47.607+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 24 271667630317209 52 30399 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:47.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 26 120989102157727 59 30399 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:47.613+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 27 5715487545729 54 30399 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:47.613+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 28 236953356346174 54 30399 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:47.617+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 29 130231725986856 49 30399 172.31.5.112:9000. HWT [49-49,49]. 2023-10-21T04:16:47.619+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 31 73432771436458 60 30399 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:47.619+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 30 60545336618717 46 30399 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:47.618+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 2 Created recovery point (took 18.836691ms) 2023-10-21T04:16:47.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 32 67362620867705 67 30399 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:47.623+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:16:47.624+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 33 278426329668882 54 30399 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:47.627+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 34 51776324008509 60 30399 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:47.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 35 24368176313355 50 30399 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:16:47.631+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 36 43594649874772 52 30399 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:16:47.632+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 37 83647976044922 59 30399 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:16:47.635+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 38 181267443431019 53 30399 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:16:47.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 39 100659545510465 49 30399 172.31.5.112:9000. HWT [48-49,49]. 2023-10-21T04:16:47.645+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 42 45479338375128 55 30399 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:47.645+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 41 116785907755771 55 30399 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:47.648+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 45 49796766757346 61 30399 172.31.5.112:9000. HWT [58-61,61]. 2023-10-21T04:16:47.648+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 46 124194574848641 54 30399 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:47.669+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 40 153478150002150 68 30399 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:16:47.669+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 50 199728167534433 66 30399 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:47.669+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 49 184487117398647 55 30399 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:47.670+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 43 249919763471169 66 30399 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:16:47.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 53 138935694816848 61 30399 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:47.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 54 24856128127173 60 30399 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:16:47.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 52 158076890855634 54 30399 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:16:47.675+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 44 166675095997741 57 30399 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:16:47.675+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 48 19274425379955 55 30399 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:47.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 55 61798873988101 67 30399 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:16:47.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 56 29071415952852 61 30399 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:16:47.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 47 149022976193937 55 30399 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:16:47.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 51 196777743741606 51 30399 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:16:47.684+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 57 191719959781498 63 30399 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:47.687+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:47.687+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:47.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 59 140942236111856 63 30399 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:16:47.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 58 103336286678362 46 30399 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:47.691+05:30 [Info] switched currmeta from 414 -> 415 force true 2023-10-21T04:16:47.693+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 60 56626109228904 46 30399 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:16:47.693+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 61 119871837460216 39 30399 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:16:47.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 62 177105814270799 58 30399 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:16:47.699+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_system:_query InstanceIds [4992762551574016313] 2023-10-21T04:16:47.699+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query Cid 9 SessionId 30399 2023-10-21T04:16:47.699+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 30399 2023-10-21T04:16:47.699+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:16:47.700+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:16:47.701+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 4992762551574016313 INDEX_STATE_CATCHUP 2023-10-21T04:16:47.701+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 4992762551574016313 true 2023-10-21T04:16:47.701+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 4992762551574016313 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query Session: 30399 BuildTS: [0=52,1=61,2=50,3=45,4=58,5=54,6=61,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=60,15=67,16=54,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=61,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=68,41=55,42=55,43=66,44=57,45=61,46=54,47=55,48=55,49=55,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:16:47.701+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_system:_query, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 45 44 45} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} , fetchKVSeq false 2023-10-21T04:16:47.701+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:16:47.701+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 30399 STREAM_ACTIVE 2023-10-21T04:16:47.701+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=4,1=61,2=4,3=45,4=4,5=4,6=61,7=4,8=4,9=4,10=4,11=4,12=4,13=4,14=60,15=4,16=54,17=45,18=4,19=18,20=4,21=4,22=4,23=22,24=16,25=61,26=4,27=4,28=4,29=4,30=4,31=4,32=4,33=25,34=4,35=4,36=34,37=4,38=26,39=4,40=68,41=4,42=4,43=66,44=57,45=61,46=4,47=4,48=22,49=55,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=4,59=4,60=4,61=15,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:47.701+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 63 252537041863196 73 30399 172.31.5.112:9000. HWT [72-73,73]. 2023-10-21T04:16:47.701+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 30399 2023-10-21T04:16:47.701+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_system:_query 30399 2023-10-21T04:16:47.702+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:47.702+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:47.702+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:47.703+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:47.704+05:30 [Info] send progress stats to clients 2023-10-21T04:16:47.707+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:16:47.707+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:16:47.708+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:10] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:16:47.708+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 30399 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:16:47.712+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:16:47.714+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.212494ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:47.727+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:47.727+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:47.730+05:30 [Info] switched currmeta from 415 -> 416 force true 2023-10-21T04:16:47.731+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 2023-10-21T04:16:47.731+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 30399 2023-10-21T04:16:47.731+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 30399 2023-10-21T04:16:47.731+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:16:47.731+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:16:47.731+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 4992762551574016313 false 2023-10-21T04:16:47.731+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_system:_query 4992762551574016313 2023-10-21T04:16:47.731+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:16:47.731+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 30399 2023-10-21T04:16:47.791+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:16:48.061+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is enabled 2023-10-21T04:16:48.087+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.087+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.093+05:30 [Info] switched currmeta from 416 -> 417 force true 2023-10-21T04:16:48.105+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.105+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.109+05:30 [Info] switched currmeta from 417 -> 418 force true 2023-10-21T04:16:48.149+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.151+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.158+05:30 [Info] switched currmeta from 418 -> 419 force true 2023-10-21T04:16:48.341+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.341+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.344+05:30 [Info] switched currmeta from 419 -> 420 force true 2023-10-21T04:16:48.379+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:48.403+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenac:66:40:4e:c2:71:1a:f2 MasterId: SourceId: DestId: RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:48.403+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9112972261750078043 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.404+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3100906924534408756 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.404+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14343533858105260407 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.404+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 962468716067699903 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.404+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7494249569853685720 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.405+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11790381330881939912 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.405+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.405+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7582778950442222621 PartitionId 0 Started Billing WUs 2023-10-21T04:16:48.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:16:48.538+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:48.598+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc4:7d:5f:e7:df:39:a4:47 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardTokena7:37:e9:2a:6a:83:12:d0 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7706346479242740906 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 5852031406248543250 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11796582816259958249 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 17179610934845473267 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3798074699393052011 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 656937514048976938 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1767085885017778314 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10051274925833832274 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3131348393020297642 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 11200171968618803270 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8177459947280813077 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 6589365266016780433 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:48.609+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardTokenc4:7d:5f:e7:df:39:a4:47 2023-10-21T04:16:48.702+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:16:48.702+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_system:_query 2023-10-21T04:16:48.702+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 4992762551574016313 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId: 30399, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 52 52 52} { 1 a0a72d5bdb4c 3 61 58 61} { 2 20330c961c81 3 50 50 50} { 3 c439193e7d29 3 45 45 45} { 4 90b66ce513a7 3 58 58 58} { 5 3861b622aae 3 54 54 54} { 6 edd3aef555a9 3 61 60 61} { 7 9ae064feda8a 3 52 52 52} { 8 f0fe56d4b431 3 56 56 56} { 9 d700dc2dce0d 3 62 62 62} { 10 bbd754cc83fc 3 50 50 50} { 11 d58311453aaa 3 56 56 56} { 12 6998b9145736 3 55 55 55} { 13 5d163afcd32f 3 53 53 53} { 14 7f8b7269d896 3 60 60 60} { 15 937b84e9e4bb 3 67 67 67} { 16 b69566250ff0 3 54 49 54} { 17 59e08337a7e3 3 45 44 45} { 18 bc6335a10528 3 46 46 46} { 19 bdef6ebc5d59 3 45 45 45} { 20 548b16b242a2 3 50 50 50} { 21 19729600a7ec 3 54 54 54} { 22 aed19015bf63 3 55 55 55} { 23 8ce99bc819fe 3 51 51 51} { 24 f7148c9a1e99 3 52 52 52} { 25 9977e650bbaf 3 61 59 61} { 26 6e09f874af9f 3 59 59 59} { 27 532bd9ad981 3 54 54 54} { 28 d78200a2233e 3 54 54 54} { 29 7671efc41428 3 49 49 49} { 30 3710cf482edd 3 46 46 46} { 31 42c966419faa 3 60 60 60} { 32 3d4415168479 3 67 67 67} { 33 fd3a2e79dd12 3 54 54 54} { 34 2f171d667a3d 3 60 60 60} { 35 1629a87e340b 3 50 50 50} { 36 27a62b9eb154 3 52 52 52} { 37 4c13cfe5f97a 3 59 59 59} { 38 a4dc9d93366b 3 53 53 53} { 39 5b8ca0741241 3 49 48 49} { 40 8b966a9c95e6 3 68 68 68} { 41 6a37568beafb 3 55 55 55} { 42 295cfbcef7d8 3 55 55 55} { 43 e34cfab95b41 3 66 66 66} { 44 979711e3cd2d 3 57 57 57} { 45 2d4a3697f1e2 3 61 58 61} { 46 70f44d845281 3 54 54 54} { 47 87891d8f3591 3 55 55 55} { 48 1187ad472c73 3 55 55 55} { 49 a7ca41192277 3 55 55 55} { 50 b5a6d67a3f61 3 66 66 66} { 51 b2f7e383f2a6 3 51 51 51} { 52 8fc524ed38d2 3 54 54 54} { 53 7e5c7cbc7650 3 61 61 61} { 54 169b44b000c5 3 60 60 60} { 55 3834abee7c05 3 67 67 67} { 56 1a70b76bc5d4 3 61 61 61} { 57 ae5e4819607a 3 63 63 63} { 58 5dfbdaa8ed5a 3 46 46 46} { 59 802fabead3f0 3 63 63 63} { 60 33804b202768 3 46 46 46} { 61 6d05d64a56f8 3 39 39 39} { 62 a113a925b34f 3 58 58 58} { 63 e5ae5cac421c 3 73 72 73} 2023-10-21T04:16:48.703+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 4992762551574016313 INDEX_STATE_ACTIVE 2023-10-21T04:16:48.703+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query State Changed to INACTIVE 2023-10-21T04:16:48.703+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:16:48.703+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:16:48.703+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:48.703+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 30399 2023-10-21T04:16:48.703+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:48.704+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:16:48.704+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 4 ReplicaId: 1 2023-10-21T04:16:48.704+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 30399 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:48.705+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4992762551574016313 from stream INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:16:48.705+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:16:48.705+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:16:48.705+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:16:48.705+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 30399 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:16:48.705+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:16:48.705+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:1 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:48.705+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:45 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:16:48.705+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:16:48.705+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:49200: use of closed network connection 2023-10-21T04:16:48.705+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:49200" exit: read tcp4 127.0.0.1:9121->127.0.0.1:49200: use of closed network connection 2023-10-21T04:16:48.705+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:49200" closed ! 2023-10-21T04:16:48.705+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:16:48.705+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:49200" already gone 2023-10-21T04:16:48.705+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:48.706+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:48.706+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:16:48.706+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:16:48.706+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:16:48.706+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:16:48.706+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId 30399 2023-10-21T04:16:48.706+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:16:48.715+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.645946ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:48.731+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_system:_query]. Err genServer.closed 2023-10-21T04:16:48.731+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:16:48.731+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Treating genServer.closed As Success 2023-10-21T04:16:48.731+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 30399 2023-10-21T04:16:48.731+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 30399 2023-10-21T04:16:48.731+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:16:48.738+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.738+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.754+05:30 [Info] switched currmeta from 420 -> 421 force true 2023-10-21T04:16:48.929+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.929+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.933+05:30 [Info] switched currmeta from 421 -> 422 force true 2023-10-21T04:16:48.950+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.950+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.952+05:30 [Info] switched currmeta from 422 -> 423 force true 2023-10-21T04:16:48.972+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:48.972+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:48.981+05:30 [Info] switched currmeta from 423 -> 424 force true 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 2542196297959826589:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.007+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 2542196297959826589:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:16:49.017+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:49.051+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:49.100+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.100+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.106+05:30 [Info] switched currmeta from 424 -> 424 force true 2023-10-21T04:16:49.335+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0056273e0 2023-10-21T04:16:49.335+05:30 [Info] handleUpdateIndexRState: Index instance 4992762551574016313 rstate moved to ACTIVE 2023-10-21T04:16:49.343+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.484024ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:49.343+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[4992762551574016313:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:16:49.343+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c, elapsed(sec): 32.16451393 2023-10-21T04:16:49.353+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken8f:ea:6f:63:2c:70:f5:8c 2023-10-21T04:16:49.357+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:49.391+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken63:23:c0:69:75:f3:f7:e8 MasterId: SourceId: DestId: RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:49.391+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 9720186692804226822 PartitionId 0 Started Billing WUs 2023-10-21T04:16:49.391+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 11050396066566371125 PartitionId 0 Started Billing WUs 2023-10-21T04:16:49.391+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8606677778651310689 PartitionId 0 Started Billing WUs 2023-10-21T04:16:49.391+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13739166377675070800 PartitionId 0 Started Billing WUs 2023-10-21T04:16:49.391+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 1265729768899070816 PartitionId 0 Started Billing WUs 2023-10-21T04:16:49.391+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 18415077564616412601 PartitionId 0 Started Billing WUs 2023-10-21T04:16:49.427+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.427+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.436+05:30 [Info] switched currmeta from 424 -> 425 force true 2023-10-21T04:16:49.472+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:49.508+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.508+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.515+05:30 [Info] switched currmeta from 425 -> 426 force true 2023-10-21T04:16:49.565+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.565+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.568+05:30 [Info] switched currmeta from 426 -> 427 force true 2023-10-21T04:16:49.576+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.576+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.592+05:30 [Info] switched currmeta from 427 -> 428 force true 2023-10-21T04:16:49.615+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.615+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.620+05:30 [Info] switched currmeta from 428 -> 428 force true 2023-10-21T04:16:49.641+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.641+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.667+05:30 [Info] switched currmeta from 428 -> 428 force true 2023-10-21T04:16:49.709+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.709+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.722+05:30 [Info] switched currmeta from 428 -> 429 force true 2023-10-21T04:16:49.749+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken32:9a:c2:77:93:42:88:d5 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Copy SiblingTokenId: ShardToken8f:ea:6f:63:2c:70:f5:8c Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2432386300762706992 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2542196297959826589 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 17773599902537064133 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 10610515785657656441 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 16422752179446142451 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5282181291074303797 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 864958057973929476 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13864207785805646348 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6603493508072150979 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3341581108461425291 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 2109208245573027560 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 1408113278413491434 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:49.773+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.773+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.814+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken32:9a:c2:77:93:42:88:d5 2023-10-21T04:16:49.825+05:30 [Info] switched currmeta from 429 -> 430 force true 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:16:49.979+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.979+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:49.987+05:30 [Info] switched currmeta from 430 -> 432 force true 2023-10-21T04:16:49.998+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:49.998+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.003+05:30 [Info] switched currmeta from 432 -> 433 force true 2023-10-21T04:16:50.012+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.013+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.022+05:30 [Info] switched currmeta from 433 -> 433 force true 2023-10-21T04:16:50.034+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.034+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.038+05:30 [Info] switched currmeta from 433 -> 434 force true 2023-10-21T04:16:50.098+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.098+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.104+05:30 [Info] switched currmeta from 434 -> 434 force true 2023-10-21T04:16:50.150+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.150+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.165+05:30 [Info] switched currmeta from 434 -> 435 force true 2023-10-21T04:16:50.201+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.201+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.219+05:30 [Info] switched currmeta from 435 -> 436 force true 2023-10-21T04:16:50.264+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.264+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.267+05:30 [Info] switched currmeta from 436 -> 436 force true 2023-10-21T04:16:50.296+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.296+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.310+05:30 [Info] switched currmeta from 436 -> 437 force true 2023-10-21T04:16:50.373+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.373+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.381+05:30 [Info] switched currmeta from 437 -> 438 force true 2023-10-21T04:16:50.452+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.452+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.454+05:30 [Info] switched currmeta from 438 -> 438 force true 2023-10-21T04:16:50.491+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.491+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.496+05:30 [Info] switched currmeta from 438 -> 439 force true 2023-10-21T04:16:50.557+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.557+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.563+05:30 [Info] switched currmeta from 439 -> 440 force true 2023-10-21T04:16:50.617+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.629+05:30 [Info] switched currmeta from 440 -> 441 force true 2023-10-21T04:16:50.647+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.647+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.660+05:30 [Info] switched currmeta from 441 -> 442 force true 2023-10-21T04:16:50.682+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.682+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.697+05:30 [Info] switched currmeta from 442 -> 442 force true 2023-10-21T04:16:50.710+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.710+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.715+05:30 [Info] switched currmeta from 442 -> 442 force true 2023-10-21T04:16:50.749+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.749+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.759+05:30 [Info] switched currmeta from 442 -> 443 force true 2023-10-21T04:16:50.769+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.769+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.772+05:30 [Info] switched currmeta from 443 -> 444 force true 2023-10-21T04:16:50.833+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.833+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.842+05:30 [Info] switched currmeta from 444 -> 444 force true 2023-10-21T04:16:50.883+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.883+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.888+05:30 [Info] switched currmeta from 444 -> 445 force true 2023-10-21T04:16:50.896+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.896+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.902+05:30 [Info] switched currmeta from 445 -> 446 force true 2023-10-21T04:16:50.924+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.925+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.932+05:30 [Info] switched currmeta from 446 -> 447 force true 2023-10-21T04:16:50.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:50.945+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:50.966+05:30 [Info] switched currmeta from 447 -> 448 force true 2023-10-21T04:16:51.002+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.003+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.013+05:30 [Info] switched currmeta from 448 -> 448 force true 2023-10-21T04:16:51.037+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.037+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.047+05:30 [Info] switched currmeta from 448 -> 449 force true 2023-10-21T04:16:51.055+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.055+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.060+05:30 [Info] switched currmeta from 449 -> 449 force true 2023-10-21T04:16:51.129+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.137+05:30 [Info] switched currmeta from 449 -> 450 force true 2023-10-21T04:16:51.157+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.158+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.164+05:30 [Info] switched currmeta from 450 -> 450 force true 2023-10-21T04:16:51.187+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.187+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.191+05:30 [Info] switched currmeta from 450 -> 451 force true 2023-10-21T04:16:51.224+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.224+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.227+05:30 [Info] switched currmeta from 451 -> 452 force true 2023-10-21T04:16:51.245+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.249+05:30 [Info] switched currmeta from 452 -> 452 force true 2023-10-21T04:16:51.265+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.265+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.270+05:30 [Info] switched currmeta from 452 -> 453 force true 2023-10-21T04:16:51.331+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.331+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.334+05:30 [Info] switched currmeta from 453 -> 454 force true 2023-10-21T04:16:51.410+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.411+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.414+05:30 [Info] switched currmeta from 454 -> 454 force true 2023-10-21T04:16:51.432+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.432+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.436+05:30 [Info] switched currmeta from 454 -> 455 force true 2023-10-21T04:16:51.463+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.463+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.467+05:30 [Info] switched currmeta from 455 -> 456 force true 2023-10-21T04:16:51.489+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.489+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.492+05:30 [Info] switched currmeta from 456 -> 456 force true 2023-10-21T04:16:51.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.501+05:30 [Info] switched currmeta from 456 -> 457 force true 2023-10-21T04:16:51.565+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.565+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.567+05:30 [Info] switched currmeta from 457 -> 458 force true 2023-10-21T04:16:51.669+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.670+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.694+05:30 [Info] switched currmeta from 458 -> 458 force true 2023-10-21T04:16:51.708+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.708+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.779+05:30 [Info] switched currmeta from 458 -> 459 force true 2023-10-21T04:16:51.789+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:51.790+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:51.802+05:30 [Info] switched currmeta from 459 -> 460 force true 2023-10-21T04:16:51.996+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9848959074666447, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:16:52.046+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.047+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.056+05:30 [Info] switched currmeta from 460 -> 462 force true 2023-10-21T04:16:52.067+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.067+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.081+05:30 [Info] switched currmeta from 462 -> 463 force true 2023-10-21T04:16:52.098+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.100+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.120+05:30 [Info] switched currmeta from 463 -> 464 force true 2023-10-21T04:16:52.129+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.132+05:30 [Info] switched currmeta from 464 -> 464 force true 2023-10-21T04:16:52.175+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.175+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.179+05:30 [Info] switched currmeta from 464 -> 464 force true 2023-10-21T04:16:52.218+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.218+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.236+05:30 [Info] switched currmeta from 464 -> 466 force true 2023-10-21T04:16:52.261+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.261+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.266+05:30 [Info] switched currmeta from 466 -> 466 force true 2023-10-21T04:16:52.282+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.282+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.286+05:30 [Info] switched currmeta from 466 -> 466 force true 2023-10-21T04:16:52.293+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.294+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.299+05:30 [Info] switched currmeta from 466 -> 467 force true 2023-10-21T04:16:52.468+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.468+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.476+05:30 [Info] switched currmeta from 467 -> 468 force true 2023-10-21T04:16:52.510+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.510+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.517+05:30 [Info] switched currmeta from 468 -> 468 force true 2023-10-21T04:16:52.543+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.543+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.554+05:30 [Info] switched currmeta from 468 -> 470 force true 2023-10-21T04:16:52.563+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.563+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.566+05:30 [Info] switched currmeta from 470 -> 470 force true 2023-10-21T04:16:52.643+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.643+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.651+05:30 [Info] switched currmeta from 470 -> 472 force true 2023-10-21T04:16:52.667+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.667+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.671+05:30 [Info] switched currmeta from 472 -> 472 force true 2023-10-21T04:16:52.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.704+05:30 [Info] switched currmeta from 472 -> 473 force true 2023-10-21T04:16:52.711+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.711+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.723+05:30 [Info] switched currmeta from 473 -> 473 force true 2023-10-21T04:16:52.752+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.752+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.764+05:30 [Info] switched currmeta from 473 -> 474 force true 2023-10-21T04:16:52.803+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.803+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.810+05:30 [Info] switched currmeta from 474 -> 476 force true 2023-10-21T04:16:52.826+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.827+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.833+05:30 [Info] switched currmeta from 476 -> 478 force true 2023-10-21T04:16:52.855+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.855+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.862+05:30 [Info] switched currmeta from 478 -> 480 force true 2023-10-21T04:16:52.884+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.884+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.887+05:30 [Info] switched currmeta from 480 -> 480 force true 2023-10-21T04:16:52.891+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.891+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.893+05:30 [Info] switched currmeta from 480 -> 480 force true 2023-10-21T04:16:52.906+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.907+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.922+05:30 [Info] switched currmeta from 480 -> 483 force true 2023-10-21T04:16:52.928+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.928+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.941+05:30 [Info] switched currmeta from 483 -> 484 force true 2023-10-21T04:16:52.947+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.947+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.950+05:30 [Info] switched currmeta from 484 -> 484 force true 2023-10-21T04:16:52.963+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.963+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.966+05:30 [Info] switched currmeta from 484 -> 484 force true 2023-10-21T04:16:52.992+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9777884671287006, throttleDelayMs (new, old, change): (6, 3, 3) 2023-10-21T04:16:52.994+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:52.994+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:52.998+05:30 [Info] switched currmeta from 484 -> 485 force true 2023-10-21T04:16:53.033+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.039+05:30 [Info] switched currmeta from 485 -> 486 force true 2023-10-21T04:16:53.063+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.063+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.092+05:30 [Info] switched currmeta from 486 -> 488 force true 2023-10-21T04:16:53.129+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.143+05:30 [Info] switched currmeta from 488 -> 490 force true 2023-10-21T04:16:53.150+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.150+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.154+05:30 [Info] switched currmeta from 490 -> 490 force true 2023-10-21T04:16:53.173+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.173+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.177+05:30 [Info] switched currmeta from 490 -> 491 force true 2023-10-21T04:16:53.204+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.204+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.209+05:30 [Info] switched currmeta from 491 -> 492 force true 2023-10-21T04:16:53.248+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.249+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.258+05:30 [Info] switched currmeta from 492 -> 494 force true 2023-10-21T04:16:53.265+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.265+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.268+05:30 [Info] switched currmeta from 494 -> 494 force true 2023-10-21T04:16:53.299+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.299+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.301+05:30 [Info] switched currmeta from 494 -> 494 force true 2023-10-21T04:16:53.333+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.333+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.339+05:30 [Info] switched currmeta from 494 -> 495 force true 2023-10-21T04:16:53.359+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.359+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.364+05:30 [Info] switched currmeta from 495 -> 496 force true 2023-10-21T04:16:53.427+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.428+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.436+05:30 [Info] switched currmeta from 496 -> 498 force true 2023-10-21T04:16:53.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.455+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.482+05:30 [Info] switched currmeta from 498 -> 499 force true 2023-10-21T04:16:53.498+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.498+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.514+05:30 [Info] switched currmeta from 499 -> 500 force true 2023-10-21T04:16:53.519+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.519+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.522+05:30 [Info] switched currmeta from 500 -> 500 force true 2023-10-21T04:16:53.541+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.541+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.549+05:30 [Info] switched currmeta from 500 -> 500 force true 2023-10-21T04:16:53.565+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:53.565+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:53.571+05:30 [Info] switched currmeta from 500 -> 501 force true 2023-10-21T04:16:53.824+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:53.825+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [10890719122371003479 7659339677706884510] 2023-10-21T04:16:53.825+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : all deamons started 2023-10-21T04:16:53.825+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : all deamons started 2023-10-21T04:16:53.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:16:53.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:16:53.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : RestoreShardDone completed 2023-10-21T04:16:53.825+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : all deamons started 2023-10-21T04:16:53.825+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : all deamons started 2023-10-21T04:16:53.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:16:53.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:16:53.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : RestoreShardDone completed 2023-10-21T04:16:53.825+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [10890719122371003479 7659339677706884510], elapsed: 561.475µs 2023-10-21T04:16:53.825+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[7659339677706884510:true 10890719122371003479:true] 2023-10-21T04:16:53.825+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9720186692804226822, partnId: 0 2023-10-21T04:16:53.825+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 18415077564616412601, partnId: 0 2023-10-21T04:16:53.825+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1265729768899070816, partnId: 0 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.825+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2296526085582506469, partnId: 0 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 5 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 6 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 7 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 8 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 1 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 2 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 3 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10477796393418268202, partnId: 4 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13739166377675070800, partnId: 0 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 5 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 6 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 7 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 8 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 1 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 2 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 3 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12762948908485355776, partnId: 4 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 3 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 4 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 5 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 6 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 7 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 8 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 1 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 2 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 3 2023-10-21T04:16:53.828+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 4 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 5 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 6 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 7 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 8 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 1 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 2 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11050396066566371125, partnId: 0 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4611895227247053723, partnId: 0 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 1 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 2 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 3 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 4 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 5 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 6 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 7 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14579235200590321485, partnId: 8 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13624500265195250810, partnId: 0 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16733302726553098964, partnId: 0 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15425360657090425347, partnId: 0 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14975131556821987100, partnId: 0 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 7 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 8 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 1 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 2 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 3 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 4 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 5 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17357232428819395534, partnId: 6 2023-10-21T04:16:53.829+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4992762551574016313, partnId: 0 2023-10-21T04:16:53.830+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8606677778651310689, partnId: 0 2023-10-21T04:16:53.830+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.52µs 2023-10-21T04:16:53.831+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [10890719122371003479 7659339677706884510] 2023-10-21T04:16:53.831+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [10890719122371003479 7659339677706884510], errMap: map[7659339677706884510: 10890719122371003479:], elapsed: 10.292µs 2023-10-21T04:16:53.831+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:16:53.831+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:16:53.831+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_DONE] 2023-10-21T04:16:53.831+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 31.363µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:53.846+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:53.846+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [2864853750198994630 12177293282243154215] 2023-10-21T04:16:53.846+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : all deamons started 2023-10-21T04:16:53.846+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : all deamons started 2023-10-21T04:16:53.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:16:53.847+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.847+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.847+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.847+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:16:53.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : RestoreShardDone completed 2023-10-21T04:16:53.871+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : all deamons started 2023-10-21T04:16:53.871+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : all deamons started 2023-10-21T04:16:53.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:16:53.871+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.871+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:16:53.871+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.872+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:16:53.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:16:53.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : RestoreShardDone completed 2023-10-21T04:16:53.873+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [2864853750198994630 12177293282243154215], elapsed: 26.278927ms 2023-10-21T04:16:53.873+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[2864853750198994630:true 12177293282243154215:true] 2023-10-21T04:16:53.874+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4173530108555528677, partnId: 0 2023-10-21T04:16:53.875+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12265503634969032942, partnId: 0 2023-10-21T04:16:53.875+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14213410630307008054, partnId: 0 2023-10-21T04:16:53.875+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11790381330881939912, partnId: 0 2023-10-21T04:16:53.875+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 3100906924534408756, partnId: 0 2023-10-21T04:16:53.875+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 1 2023-10-21T04:16:53.875+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 2 2023-10-21T04:16:53.876+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 3 2023-10-21T04:16:53.877+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 4 2023-10-21T04:16:53.877+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 5 2023-10-21T04:16:53.877+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 6 2023-10-21T04:16:53.877+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 7 2023-10-21T04:16:53.878+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4440544141961093904, partnId: 8 2023-10-21T04:16:53.878+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 1 2023-10-21T04:16:53.878+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 2 2023-10-21T04:16:53.878+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 3 2023-10-21T04:16:53.878+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 4 2023-10-21T04:16:53.878+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 5 2023-10-21T04:16:53.879+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 6 2023-10-21T04:16:53.880+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 7 2023-10-21T04:16:53.880+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5744899830895973442, partnId: 8 2023-10-21T04:16:53.880+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 5 2023-10-21T04:16:53.880+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 6 2023-10-21T04:16:53.880+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 7 2023-10-21T04:16:53.880+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 8 2023-10-21T04:16:53.881+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 1 2023-10-21T04:16:53.881+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 2 2023-10-21T04:16:53.881+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 3 2023-10-21T04:16:53.882+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 4 2023-10-21T04:16:53.883+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14271070195512901425, partnId: 0 2023-10-21T04:16:53.883+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14343533858105260407, partnId: 0 2023-10-21T04:16:53.883+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7494249569853685720, partnId: 0 2023-10-21T04:16:53.883+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9112972261750078043, partnId: 0 2023-10-21T04:16:53.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 3 2023-10-21T04:16:53.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 4 2023-10-21T04:16:53.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 5 2023-10-21T04:16:53.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 6 2023-10-21T04:16:53.884+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 7 2023-10-21T04:16:53.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 8 2023-10-21T04:16:53.885+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 1 2023-10-21T04:16:53.886+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 759049595436785640, partnId: 2 2023-10-21T04:16:53.890+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 8 2023-10-21T04:16:53.890+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 1 2023-10-21T04:16:53.890+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 2 2023-10-21T04:16:53.890+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 3 2023-10-21T04:16:53.892+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 4 2023-10-21T04:16:53.892+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 5 2023-10-21T04:16:53.892+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 6 2023-10-21T04:16:53.892+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11375521620401506748, partnId: 7 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 4 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 5 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 6 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 7 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 8 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 1 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 2 2023-10-21T04:16:53.893+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 3 2023-10-21T04:16:53.894+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15667952562388529486, partnId: 0 2023-10-21T04:16:53.894+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7582778950442222621, partnId: 0 2023-10-21T04:16:53.894+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5004576769623010783, partnId: 0 2023-10-21T04:16:53.894+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 962468716067699903, partnId: 0 2023-10-21T04:16:53.894+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.733µs 2023-10-21T04:16:53.894+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [2864853750198994630 12177293282243154215] 2023-10-21T04:16:53.895+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [2864853750198994630 12177293282243154215], errMap: map[2864853750198994630: 12177293282243154215:], elapsed: 12.959µs 2023-10-21T04:16:53.895+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:16:53.895+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:16:53.895+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_DONE] 2023-10-21T04:16:53.895+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 29.239µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:53.949+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8f:ea:6f:63:2c:70:f5:8c MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken32:9a:c2:77:93:42:88:d5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15425360657090425347 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 9720186692804226822 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 3288617707771216315 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 6291351234483181222 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11050396066566371125 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 13624500265195250810 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14975131556821987100 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14888890778837396466 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 976404723154064977 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4087317880204280526 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 15841249126653343762 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8606677778651310689 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:53.949+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken8f:ea:6f:63:2c:70:f5:8c 2023-10-21T04:16:53.950+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:37:e9:2a:6a:83:12:d0 MasterId: 458c2b8d760618b2a2f6a998f4c93e74 SourceId: 217f03d04907d0ef82070fe4f41363c1 (127.0.0.1:9001) DestId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) RebalId: cf56ae547f07d182689428d60f3e89c9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc4:7d:5f:e7:df:39:a4:47 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7691103359834243171 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 14213410630307008054 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9112972261750078043 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11368506836388887403 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3381906773469851 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 8538178595515381903 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 11579858587830428859 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 4412153410740056345 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [4 4 4 4 4 4 4 4] InstId: 3100906924534408756 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 14343533858105260407 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 4173530108555528677 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 962468716067699903 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [4] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [4] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [4] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [4] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:16:54.001+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9399065581374471, throttleDelayMs (new, old, change): (5, 6, -1) 2023-10-21T04:16:54.114+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken 2023-10-21T04:16:54.114+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token Deleted. Mark Done. 2023-10-21T04:16:54.114+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:16:54.114+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:16:54.114+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:16:54.114+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:16:54.180+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:16:54.180+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/cf56ae547f07d182689428d60f3e89c9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:cf56ae547f07d182689428d60f3e89c9]}]} 2023-10-21T04:16:54.192+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:16:54.192+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:16:54.192+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.514µs 2023-10-21T04:16:54.192+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:16:54.192+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:16:54.192+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:16:54.195+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:16:54.195+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:16:54.195+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:16:54.195+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:16:54.195+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 34.77µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:54.196+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:16:54.196+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:16:54.196+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:16:54.196+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:16:54.210+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:16:54.210+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:16:54.256+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/cf56ae547f07d182689428d60f3e89c9, rev: [] 2023-10-21T04:16:54.256+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/cf56ae547f07d182689428d60f3e89c9, rev: [] 2023-10-21T04:16:54.256+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:16:54.256+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:16:54.256+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:16:54.256+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:16:54.256+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 31.34µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:54.258+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:16:54.258+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:16:54.258+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/cf56ae547f07d182689428d60f3e89c9, rev: [] call. err: 2023-10-21T04:16:54.258+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:16:54.258+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:16:54.261+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:16:54.261+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:16:54.264+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:16:54.264+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:16:54.266+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:16:54.266+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:16:54.537+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:16:54.755+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:54.862+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:48876 2023-10-21T04:16:54.949+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:54.949+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:54.949+05:30 [Info] UnwatchMetadata(): indexer 217f03d04907d0ef82070fe4f41363c1 2023-10-21T04:16:54.950+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:16:54.950+05:30 [Info] UnwatchMetadata(): finish for indexer 217f03d04907d0ef82070fe4f41363c1 2023-10-21T04:16:54.951+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:37536->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:16:54.973+05:30 [Info] switched currmeta from 501 -> 502 force true 2023-10-21T04:16:54.973+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:16:54.973+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] ... stopped 2023-10-21T04:16:54.997+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:54.997+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:55.011+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:55.018+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:55.045+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 161.071685ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:16:55.048+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:48876. Reason = EOF. Cleaning Pipe. 2023-10-21T04:16:55.048+05:30 [Info] messageListener.start(): message channel closed. Remove peer a9:b5:8e:52:c7:9c:ce:91 as follower. 2023-10-21T04:16:55.383+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 1 unhealthy node 0 add node 0 2023-10-21T04:16:55.383+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:55.383+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:55.415+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:55.526+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:16:55.527+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:16:55.541+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Stopping releaseConnsRoutine 2023-10-21T04:16:55.547+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:16:55.553+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:46298. Error = read tcp4 127.0.0.1:9118->127.0.0.1:46298: read: connection reset by peer. Kill Pipe. 2023-10-21T04:16:55.553+05:30 [Info] messageListener.start(): message channel closed. Remove peer c0:2b:f8:6d:9d:c6:e4:1a as follower. 2023-10-21T04:16:55.554+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:57960 closed 2023-10-21T04:16:55.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:55.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:55.992+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996408787211919, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:16:56.237+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:56.486+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 226] updated cluster rebalancing status to false 2023-10-21T04:16:56.574+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:16:56.574+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:56.574+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:56.578+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:56.606+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:16:56.607+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:56.621+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:56.621+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:56.624+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:56.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:56.629+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:56.631+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:56.927+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:16:56.958+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:56.958+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:56.969+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:56.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9762717609591457 2023-10-21T04:16:57.011+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:16:57.020+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:16:57.020+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:16:57.023+05:30 [Info] switched currmeta from 502 -> 502 force true 2023-10-21T04:16:57.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7681327808504597, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:16:58.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6347905699072983, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:17:03.227+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is enabled 2023-10-21T04:17:05.527+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60872" doAuth() ... 2023-10-21T04:17:05.530+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60872" doReceive() ... 2023-10-21T04:17:05.574+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60880" doAuth() ... 2023-10-21T04:17:05.576+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60880" doReceive() ... 2023-10-21T04:17:05.624+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60890" doAuth() ... 2023-10-21T04:17:05.627+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60890" doReceive() ... 2023-10-21T04:17:05.695+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60896" doAuth() ... 2023-10-21T04:17:05.697+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60896" doReceive() ... 2023-10-21T04:17:05.712+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60898" doAuth() ... 2023-10-21T04:17:05.714+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60898" doReceive() ... 2023-10-21T04:17:05.751+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60904" doAuth() ... 2023-10-21T04:17:05.753+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60904" doReceive() ... 2023-10-21T04:17:05.788+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60908" doAuth() ... 2023-10-21T04:17:05.789+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60908" doReceive() ... 2023-10-21T04:17:05.824+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60916" doAuth() ... 2023-10-21T04:17:05.827+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60916" doReceive() ... 2023-10-21T04:17:05.859+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60918" doAuth() ... 2023-10-21T04:17:05.859+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60918" doReceive() ... 2023-10-21T04:17:05.989+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60934" doAuth() ... 2023-10-21T04:17:05.992+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9567945573712435, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:17:05.993+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60934" doReceive() ... 2023-10-21T04:17:06.055+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60944" doAuth() ... 2023-10-21T04:17:06.055+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60944" doReceive() ... 2023-10-21T04:17:06.131+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60948" doAuth() ... 2023-10-21T04:17:06.132+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60948" doReceive() ... 2023-10-21T04:17:06.223+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60962" doAuth() ... 2023-10-21T04:17:06.224+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60962" doReceive() ... 2023-10-21T04:17:06.256+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60968" doAuth() ... 2023-10-21T04:17:06.257+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60968" doReceive() ... 2023-10-21T04:17:06.284+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60976" doAuth() ... 2023-10-21T04:17:06.287+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60976" doReceive() ... 2023-10-21T04:17:06.418+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60984" doAuth() ... 2023-10-21T04:17:06.424+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60984" doReceive() ... 2023-10-21T04:17:06.427+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60988" doAuth() ... 2023-10-21T04:17:06.430+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60988" doReceive() ... 2023-10-21T04:17:06.459+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60994" doAuth() ... 2023-10-21T04:17:06.474+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:60994" doReceive() ... 2023-10-21T04:17:06.537+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32812" doAuth() ... 2023-10-21T04:17:06.538+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32812" doReceive() ... 2023-10-21T04:17:06.557+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32818" doAuth() ... 2023-10-21T04:17:06.578+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32818" doReceive() ... 2023-10-21T04:17:06.585+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32824" doAuth() ... 2023-10-21T04:17:06.587+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32824" doReceive() ... 2023-10-21T04:17:06.617+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32828" doAuth() ... 2023-10-21T04:17:06.619+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32828" doReceive() ... 2023-10-21T04:17:06.622+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32830" doAuth() ... 2023-10-21T04:17:06.626+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32830" doReceive() ... 2023-10-21T04:17:06.742+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32848" doAuth() ... 2023-10-21T04:17:06.746+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32848" doReceive() ... 2023-10-21T04:17:06.856+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32858" doAuth() ... 2023-10-21T04:17:06.859+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32858" doReceive() ... 2023-10-21T04:17:06.885+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32862" doAuth() ... 2023-10-21T04:17:06.887+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32862" doReceive() ... 2023-10-21T04:17:06.895+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32864" doAuth() ... 2023-10-21T04:17:06.895+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32864" doReceive() ... 2023-10-21T04:17:06.909+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32868" doAuth() ... 2023-10-21T04:17:06.918+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32868" doReceive() ... 2023-10-21T04:17:06.981+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32874" doAuth() ... 2023-10-21T04:17:06.993+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32874" doReceive() ... 2023-10-21T04:17:06.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9674921811514023, throttleDelayMs (new, old, change): (3, 1, 2) 2023-10-21T04:17:07.029+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32876" doAuth() ... 2023-10-21T04:17:07.077+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32876" doReceive() ... 2023-10-21T04:17:07.151+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32888" doAuth() ... 2023-10-21T04:17:07.153+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32888" doReceive() ... 2023-10-21T04:17:07.184+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32898" doAuth() ... 2023-10-21T04:17:07.186+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32898" doReceive() ... 2023-10-21T04:17:07.219+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32904" doAuth() ... 2023-10-21T04:17:07.219+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32904" doReceive() ... 2023-10-21T04:17:07.257+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32914" doAuth() ... 2023-10-21T04:17:07.257+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32914" doReceive() ... 2023-10-21T04:17:07.315+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32922" doAuth() ... 2023-10-21T04:17:07.316+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32922" doReceive() ... 2023-10-21T04:17:07.319+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32924" doAuth() ... 2023-10-21T04:17:07.319+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32924" doReceive() ... 2023-10-21T04:17:07.338+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32926" doAuth() ... 2023-10-21T04:17:07.346+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32926" doReceive() ... 2023-10-21T04:17:07.440+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32932" doAuth() ... 2023-10-21T04:17:07.442+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32932" doReceive() ... 2023-10-21T04:17:07.444+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32934" doAuth() ... 2023-10-21T04:17:07.446+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32934" doReceive() ... 2023-10-21T04:17:07.481+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32942" doAuth() ... 2023-10-21T04:17:07.483+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32942" doReceive() ... 2023-10-21T04:17:07.485+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32944" doAuth() ... 2023-10-21T04:17:07.486+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32944" doReceive() ... 2023-10-21T04:17:07.493+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32946" doAuth() ... 2023-10-21T04:17:07.494+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32946" doReceive() ... 2023-10-21T04:17:07.533+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32954" doAuth() ... 2023-10-21T04:17:07.535+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32954" doReceive() ... 2023-10-21T04:17:07.538+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32956" doAuth() ... 2023-10-21T04:17:07.539+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32956" doReceive() ... 2023-10-21T04:17:07.581+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32962" doAuth() ... 2023-10-21T04:17:07.583+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32962" doReceive() ... 2023-10-21T04:17:07.613+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32966" doAuth() ... 2023-10-21T04:17:07.614+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32966" doReceive() ... 2023-10-21T04:17:07.657+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32970" doAuth() ... 2023-10-21T04:17:07.659+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32970" doReceive() ... 2023-10-21T04:17:07.697+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32974" doAuth() ... 2023-10-21T04:17:07.699+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32974" doReceive() ... 2023-10-21T04:17:07.732+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32982" doAuth() ... 2023-10-21T04:17:07.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:32982" doReceive() ... 2023-10-21T04:17:07.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.972066925295832, throttleDelayMs (new, old, change): (5, 3, 2) 2023-10-21T04:17:08.668+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33010" doAuth() ... 2023-10-21T04:17:08.668+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33010" doReceive() ... 2023-10-21T04:17:08.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9135128487397226, throttleDelayMs (new, old, change): (1, 5, -4) 2023-10-21T04:17:09.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8974128285148325, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:17:12.004+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:17:12.006+05:30 [Info] janitor: running cleanup. 2023-10-21T04:17:12.006+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[4138047918192025493 4535197604682045916 4860549750398470299 5465835793930010953 12195715103909237684 14889401051926000176]} 2023-10-21T04:17:12.007+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:17:12.010+05:30 [Info] memstats {"Alloc":88180392, "TotalAlloc":90844168, "Sys":108740888, "Lookups":0, "Mallocs":107169,"Frees":24157, "HeapAlloc":88180392, "HeapSys":94732288, "HeapIdle":4481024, "HeapInuse":90251264,"HeapReleased":4481024, "HeapObjects":83012,"MSpanInuse":249920, "MSpanSys": 261120, "StackInuse": 1736704,"GCSys":9776520, "LastGC":1697841971623503856,"PauseTotalNs":15772155, "PauseNs":[15068444, 374028, 65057, 264626], "NumGC":4} 2023-10-21T04:17:12.308+05:30 [Info] connected with 2 indexers 2023-10-21T04:17:12.309+05:30 [Info] client stats current counts: current: 76, not current: 0 2023-10-21T04:17:12.441+05:30 [Info] num concurrent scans {0} 2023-10-21T04:17:12.441+05:30 [Info] average scan response {0 ms} 2023-10-21T04:17:12.650+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:17:12.705+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:17:12.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9655802373209493, throttleDelayMs (new, old, change): (2, 0, 2) 2023-10-21T04:17:13.017+05:30 [Info] Indexer::ReadMemstats Time Taken 11.684621ms 2023-10-21T04:17:13.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9831574488599766, throttleDelayMs (new, old, change): (5, 2, 3) 2023-10-21T04:17:14.848+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33250" doAuth() ... 2023-10-21T04:17:14.859+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33250" doReceive() ... 2023-10-21T04:17:14.867+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33254" doAuth() ... 2023-10-21T04:17:14.867+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33254" doReceive() ... 2023-10-21T04:17:14.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9869525012630118, throttleDelayMs (new, old, change): (9, 5, 4) 2023-10-21T04:17:15.217+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33312" doAuth() ... 2023-10-21T04:17:15.221+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33312" doReceive() ... 2023-10-21T04:17:15.283+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33320" doAuth() ... 2023-10-21T04:17:15.284+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33320" doReceive() ... 2023-10-21T04:17:15.318+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33324" doAuth() ... 2023-10-21T04:17:15.320+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33324" doReceive() ... 2023-10-21T04:17:15.339+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33326" doAuth() ... 2023-10-21T04:17:15.341+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33326" doReceive() ... 2023-10-21T04:17:15.344+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33328" doAuth() ... 2023-10-21T04:17:15.346+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33328" doReceive() ... 2023-10-21T04:17:15.415+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33332" doAuth() ... 2023-10-21T04:17:15.417+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33332" doReceive() ... 2023-10-21T04:17:15.437+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33334" doAuth() ... 2023-10-21T04:17:15.440+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33334" doReceive() ... 2023-10-21T04:17:15.512+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33340" doAuth() ... 2023-10-21T04:17:15.515+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33340" doReceive() ... 2023-10-21T04:17:15.561+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33346" doAuth() ... 2023-10-21T04:17:15.563+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33346" doReceive() ... 2023-10-21T04:17:15.587+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33352" doAuth() ... 2023-10-21T04:17:15.590+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33352" doReceive() ... 2023-10-21T04:17:15.628+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33354" doAuth() ... 2023-10-21T04:17:15.631+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33354" doReceive() ... 2023-10-21T04:17:15.746+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33368" doAuth() ... 2023-10-21T04:17:15.746+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33368" doReceive() ... 2023-10-21T04:17:15.753+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33372" doAuth() ... 2023-10-21T04:17:15.754+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33372" doReceive() ... 2023-10-21T04:17:15.756+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33374" doAuth() ... 2023-10-21T04:17:15.757+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33374" doReceive() ... 2023-10-21T04:17:15.915+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33388" doAuth() ... 2023-10-21T04:17:15.917+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33388" doReceive() ... 2023-10-21T04:17:15.920+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33390" doAuth() ... 2023-10-21T04:17:15.921+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33390" doReceive() ... 2023-10-21T04:17:15.941+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33392" doAuth() ... 2023-10-21T04:17:15.941+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33392" doReceive() ... 2023-10-21T04:17:15.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8643702917114123, throttleDelayMs (new, old, change): (4, 9, -5) 2023-10-21T04:17:16.015+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33400" doAuth() ... 2023-10-21T04:17:16.015+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33400" doReceive() ... 2023-10-21T04:17:16.054+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33408" doAuth() ... 2023-10-21T04:17:16.054+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33408" doReceive() ... 2023-10-21T04:17:16.057+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33410" doAuth() ... 2023-10-21T04:17:16.059+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33410" doReceive() ... 2023-10-21T04:17:16.073+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33412" doAuth() ... 2023-10-21T04:17:16.074+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33412" doReceive() ... 2023-10-21T04:17:16.148+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33426" doAuth() ... 2023-10-21T04:17:16.149+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33426" doReceive() ... 2023-10-21T04:17:16.155+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33430" doAuth() ... 2023-10-21T04:17:16.155+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33430" doReceive() ... 2023-10-21T04:17:16.180+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33436" doAuth() ... 2023-10-21T04:17:16.181+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33436" doReceive() ... 2023-10-21T04:17:16.219+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33442" doAuth() ... 2023-10-21T04:17:16.219+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33442" doReceive() ... 2023-10-21T04:17:16.304+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33446" doAuth() ... 2023-10-21T04:17:16.305+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33446" doReceive() ... 2023-10-21T04:17:16.307+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33448" doAuth() ... 2023-10-21T04:17:16.307+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33448" doReceive() ... 2023-10-21T04:17:16.310+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33450" doAuth() ... 2023-10-21T04:17:16.310+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33450" doReceive() ... 2023-10-21T04:17:16.341+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33458" doAuth() ... 2023-10-21T04:17:16.342+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33458" doReceive() ... 2023-10-21T04:17:16.378+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33464" doAuth() ... 2023-10-21T04:17:16.379+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33464" doReceive() ... 2023-10-21T04:17:16.395+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33468" doAuth() ... 2023-10-21T04:17:16.397+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33468" doReceive() ... 2023-10-21T04:17:16.399+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33470" doAuth() ... 2023-10-21T04:17:16.400+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33470" doReceive() ... 2023-10-21T04:17:16.404+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33472" doAuth() ... 2023-10-21T04:17:16.404+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33472" doReceive() ... 2023-10-21T04:17:16.435+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33480" doAuth() ... 2023-10-21T04:17:16.438+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33480" doReceive() ... 2023-10-21T04:17:16.441+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33482" doAuth() ... 2023-10-21T04:17:16.442+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33482" doReceive() ... 2023-10-21T04:17:16.478+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33484" doAuth() ... 2023-10-21T04:17:16.478+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33484" doReceive() ... 2023-10-21T04:17:16.497+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33490" doAuth() ... 2023-10-21T04:17:16.497+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33490" doReceive() ... 2023-10-21T04:17:16.573+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33500" doAuth() ... 2023-10-21T04:17:16.573+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33500" doReceive() ... 2023-10-21T04:17:16.579+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33502" doAuth() ... 2023-10-21T04:17:16.580+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33502" doReceive() ... 2023-10-21T04:17:16.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33506" doAuth() ... 2023-10-21T04:17:16.599+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33506" doReceive() ... 2023-10-21T04:17:16.642+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33510" doAuth() ... 2023-10-21T04:17:16.644+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33510" doReceive() ... 2023-10-21T04:17:16.660+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33518" doAuth() ... 2023-10-21T04:17:16.662+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33518" doReceive() ... 2023-10-21T04:17:16.667+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33522" doAuth() ... 2023-10-21T04:17:16.667+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33522" doReceive() ... 2023-10-21T04:17:16.706+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33528" doAuth() ... 2023-10-21T04:17:16.707+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33528" doReceive() ... 2023-10-21T04:17:16.769+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33542" doAuth() ... 2023-10-21T04:17:16.770+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33542" doReceive() ... 2023-10-21T04:17:16.793+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33544" doAuth() ... 2023-10-21T04:17:16.793+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33544" doReceive() ... 2023-10-21T04:17:16.806+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33546" doAuth() ... 2023-10-21T04:17:16.808+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33546" doReceive() ... 2023-10-21T04:17:16.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.682891233613553, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:17:20.516+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33666" doAuth() ... 2023-10-21T04:17:20.518+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33666" doReceive() ... 2023-10-21T04:17:20.521+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33668" doAuth() ... 2023-10-21T04:17:20.522+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33668" doReceive() ... 2023-10-21T04:17:20.526+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33670" doAuth() ... 2023-10-21T04:17:20.527+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33670" doReceive() ... 2023-10-21T04:17:20.530+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33672" doAuth() ... 2023-10-21T04:17:20.531+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33672" doReceive() ... 2023-10-21T04:17:20.567+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33680" doAuth() ... 2023-10-21T04:17:20.569+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33680" doReceive() ... 2023-10-21T04:17:20.605+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33686" doAuth() ... 2023-10-21T04:17:20.606+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33686" doReceive() ... 2023-10-21T04:17:20.608+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33688" doAuth() ... 2023-10-21T04:17:20.609+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33688" doReceive() ... 2023-10-21T04:17:20.621+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33692" doAuth() ... 2023-10-21T04:17:20.622+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33692" doReceive() ... 2023-10-21T04:17:20.634+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33696" doAuth() ... 2023-10-21T04:17:20.635+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33696" doReceive() ... 2023-10-21T04:17:20.669+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33702" doAuth() ... 2023-10-21T04:17:20.671+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33702" doReceive() ... 2023-10-21T04:17:20.692+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33706" doAuth() ... 2023-10-21T04:17:20.694+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33706" doReceive() ... 2023-10-21T04:17:20.697+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33708" doAuth() ... 2023-10-21T04:17:20.699+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33708" doReceive() ... 2023-10-21T04:17:20.707+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33710" doAuth() ... 2023-10-21T04:17:20.707+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33710" doReceive() ... 2023-10-21T04:17:20.710+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33712" doAuth() ... 2023-10-21T04:17:20.711+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33712" doReceive() ... 2023-10-21T04:17:20.714+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33714" doAuth() ... 2023-10-21T04:17:20.715+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33714" doReceive() ... 2023-10-21T04:17:20.733+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33718" doAuth() ... 2023-10-21T04:17:20.735+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33718" doReceive() ... 2023-10-21T04:17:20.745+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33720" doAuth() ... 2023-10-21T04:17:20.747+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33720" doReceive() ... 2023-10-21T04:17:20.769+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33730" doAuth() ... 2023-10-21T04:17:20.771+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33730" doReceive() ... 2023-10-21T04:17:20.796+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33734" doAuth() ... 2023-10-21T04:17:20.798+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33734" doReceive() ... 2023-10-21T04:17:20.838+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33740" doAuth() ... 2023-10-21T04:17:20.841+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33740" doReceive() ... 2023-10-21T04:17:20.856+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33744" doAuth() ... 2023-10-21T04:17:20.858+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33744" doReceive() ... 2023-10-21T04:17:20.865+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33748" doAuth() ... 2023-10-21T04:17:20.868+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33748" doReceive() ... 2023-10-21T04:17:20.870+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33750" doAuth() ... 2023-10-21T04:17:20.873+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33750" doReceive() ... 2023-10-21T04:17:20.875+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33752" doAuth() ... 2023-10-21T04:17:20.877+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33752" doReceive() ... 2023-10-21T04:17:20.895+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33758" doAuth() ... 2023-10-21T04:17:20.897+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33758" doReceive() ... 2023-10-21T04:17:20.899+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33760" doAuth() ... 2023-10-21T04:17:20.901+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33760" doReceive() ... 2023-10-21T04:17:20.904+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33762" doAuth() ... 2023-10-21T04:17:20.906+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33762" doReceive() ... 2023-10-21T04:17:20.918+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33766" doAuth() ... 2023-10-21T04:17:20.921+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33766" doReceive() ... 2023-10-21T04:17:20.923+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33768" doAuth() ... 2023-10-21T04:17:20.925+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33768" doReceive() ... 2023-10-21T04:17:20.943+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33772" doAuth() ... 2023-10-21T04:17:20.945+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33772" doReceive() ... 2023-10-21T04:17:20.956+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33774" doAuth() ... 2023-10-21T04:17:20.959+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33774" doReceive() ... 2023-10-21T04:17:20.961+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33776" doAuth() ... 2023-10-21T04:17:20.964+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33776" doReceive() ... 2023-10-21T04:17:20.993+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33784" doAuth() ... 2023-10-21T04:17:20.996+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33784" doReceive() ... 2023-10-21T04:17:21.115+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33792" doAuth() ... 2023-10-21T04:17:21.117+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33792" doReceive() ... 2023-10-21T04:17:21.124+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33794" doAuth() ... 2023-10-21T04:17:21.126+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33794" doReceive() ... 2023-10-21T04:17:21.134+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33796" doAuth() ... 2023-10-21T04:17:21.136+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33796" doReceive() ... 2023-10-21T04:17:21.170+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33800" doAuth() ... 2023-10-21T04:17:21.172+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33800" doReceive() ... 2023-10-21T04:17:21.196+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33804" doAuth() ... 2023-10-21T04:17:21.198+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33804" doReceive() ... 2023-10-21T04:17:21.212+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33808" doAuth() ... 2023-10-21T04:17:21.214+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33808" doReceive() ... 2023-10-21T04:17:21.224+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33810" doAuth() ... 2023-10-21T04:17:21.226+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33810" doReceive() ... 2023-10-21T04:17:21.228+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33812" doAuth() ... 2023-10-21T04:17:21.231+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33812" doReceive() ... 2023-10-21T04:17:21.256+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33816" doAuth() ... 2023-10-21T04:17:21.258+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33816" doReceive() ... 2023-10-21T04:17:21.272+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33818" doAuth() ... 2023-10-21T04:17:21.274+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33818" doReceive() ... 2023-10-21T04:17:21.276+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33820" doAuth() ... 2023-10-21T04:17:21.278+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33820" doReceive() ... 2023-10-21T04:17:21.302+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33826" doAuth() ... 2023-10-21T04:17:21.305+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33826" doReceive() ... 2023-10-21T04:17:21.382+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33842" doAuth() ... 2023-10-21T04:17:21.385+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33842" doReceive() ... 2023-10-21T04:17:21.413+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33846" doAuth() ... 2023-10-21T04:17:21.415+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33846" doReceive() ... 2023-10-21T04:17:21.427+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33850" doAuth() ... 2023-10-21T04:17:21.430+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33850" doReceive() ... 2023-10-21T04:17:21.441+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33854" doAuth() ... 2023-10-21T04:17:21.443+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33854" doReceive() ... 2023-10-21T04:17:21.461+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33856" doAuth() ... 2023-10-21T04:17:21.463+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33856" doReceive() ... 2023-10-21T04:17:21.477+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33858" doAuth() ... 2023-10-21T04:17:21.477+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33858" doReceive() ... 2023-10-21T04:17:21.547+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33872" doAuth() ... 2023-10-21T04:17:21.550+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33872" doReceive() ... 2023-10-21T04:17:21.604+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33886" doAuth() ... 2023-10-21T04:17:21.605+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33886" doReceive() ... 2023-10-21T04:17:21.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33892" doAuth() ... 2023-10-21T04:17:21.634+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33892" doReceive() ... 2023-10-21T04:17:21.636+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33894" doAuth() ... 2023-10-21T04:17:21.638+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33894" doReceive() ... 2023-10-21T04:17:21.662+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33898" doAuth() ... 2023-10-21T04:17:21.664+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33898" doReceive() ... 2023-10-21T04:17:21.698+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33904" doAuth() ... 2023-10-21T04:17:21.701+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33904" doReceive() ... 2023-10-21T04:17:24.267+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:17:24.267+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:17:24.267+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:17:24.267+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:17:24.267+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:17:24.268+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:17:24.268+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:17:25.521+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33962" doAuth() ... 2023-10-21T04:17:25.523+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33962" doReceive() ... 2023-10-21T04:17:25.558+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33968" doAuth() ... 2023-10-21T04:17:25.563+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33968" doReceive() ... 2023-10-21T04:17:25.565+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33970" doAuth() ... 2023-10-21T04:17:25.568+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33970" doReceive() ... 2023-10-21T04:17:25.580+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33972" doAuth() ... 2023-10-21T04:17:25.581+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33972" doReceive() ... 2023-10-21T04:17:25.596+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33976" doAuth() ... 2023-10-21T04:17:25.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33976" doReceive() ... 2023-10-21T04:17:25.628+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33982" doAuth() ... 2023-10-21T04:17:25.631+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33982" doReceive() ... 2023-10-21T04:17:25.748+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33994" doAuth() ... 2023-10-21T04:17:25.748+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33994" doReceive() ... 2023-10-21T04:17:25.758+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33996" doAuth() ... 2023-10-21T04:17:25.760+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:33996" doReceive() ... 2023-10-21T04:17:25.768+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34000" doAuth() ... 2023-10-21T04:17:25.771+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34000" doReceive() ... 2023-10-21T04:17:25.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34002" doAuth() ... 2023-10-21T04:17:25.776+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34002" doReceive() ... 2023-10-21T04:17:25.786+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34008" doAuth() ... 2023-10-21T04:17:25.789+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34008" doReceive() ... 2023-10-21T04:17:25.797+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34012" doAuth() ... 2023-10-21T04:17:25.799+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34012" doReceive() ... 2023-10-21T04:17:25.802+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34014" doAuth() ... 2023-10-21T04:17:25.804+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34014" doReceive() ... 2023-10-21T04:17:25.851+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34022" doAuth() ... 2023-10-21T04:17:25.854+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34022" doReceive() ... 2023-10-21T04:17:25.878+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34030" doAuth() ... 2023-10-21T04:17:25.882+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34030" doReceive() ... 2023-10-21T04:17:25.926+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34036" doAuth() ... 2023-10-21T04:17:25.929+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34036" doReceive() ... 2023-10-21T04:17:25.932+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34038" doAuth() ... 2023-10-21T04:17:25.935+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34038" doReceive() ... 2023-10-21T04:17:25.944+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34040" doAuth() ... 2023-10-21T04:17:25.947+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34040" doReceive() ... 2023-10-21T04:17:25.958+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34042" doAuth() ... 2023-10-21T04:17:25.961+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34042" doReceive() ... 2023-10-21T04:17:25.978+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34046" doAuth() ... 2023-10-21T04:17:25.980+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34046" doReceive() ... 2023-10-21T04:17:26.057+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34054" doAuth() ... 2023-10-21T04:17:26.060+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34054" doReceive() ... 2023-10-21T04:17:26.177+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34072" doAuth() ... 2023-10-21T04:17:26.179+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34072" doReceive() ... 2023-10-21T04:17:26.182+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34074" doAuth() ... 2023-10-21T04:17:26.185+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34074" doReceive() ... 2023-10-21T04:17:26.216+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34078" doAuth() ... 2023-10-21T04:17:26.218+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34078" doReceive() ... 2023-10-21T04:17:26.261+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34086" doAuth() ... 2023-10-21T04:17:26.263+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34086" doReceive() ... 2023-10-21T04:17:26.280+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34092" doAuth() ... 2023-10-21T04:17:26.282+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34092" doReceive() ... 2023-10-21T04:17:26.315+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34100" doAuth() ... 2023-10-21T04:17:26.317+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34100" doReceive() ... 2023-10-21T04:17:26.328+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34102" doAuth() ... 2023-10-21T04:17:26.330+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34102" doReceive() ... 2023-10-21T04:17:26.339+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34104" doAuth() ... 2023-10-21T04:17:26.341+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34104" doReceive() ... 2023-10-21T04:17:26.343+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34106" doAuth() ... 2023-10-21T04:17:26.346+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34106" doReceive() ... 2023-10-21T04:17:26.348+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34108" doAuth() ... 2023-10-21T04:17:26.350+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34108" doReceive() ... 2023-10-21T04:17:26.375+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34112" doAuth() ... 2023-10-21T04:17:26.377+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34112" doReceive() ... 2023-10-21T04:17:26.379+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34114" doAuth() ... 2023-10-21T04:17:26.381+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34114" doReceive() ... 2023-10-21T04:17:26.385+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34116" doAuth() ... 2023-10-21T04:17:26.387+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34116" doReceive() ... 2023-10-21T04:17:26.412+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34128" doAuth() ... 2023-10-21T04:17:26.415+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34128" doReceive() ... 2023-10-21T04:17:26.418+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34130" doAuth() ... 2023-10-21T04:17:26.423+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34130" doReceive() ... 2023-10-21T04:17:26.426+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34132" doAuth() ... 2023-10-21T04:17:26.429+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34132" doReceive() ... 2023-10-21T04:17:26.442+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34140" doAuth() ... 2023-10-21T04:17:26.444+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34140" doReceive() ... 2023-10-21T04:17:26.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34146" doAuth() ... 2023-10-21T04:17:26.466+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34146" doReceive() ... 2023-10-21T04:17:26.499+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34150" doAuth() ... 2023-10-21T04:17:26.501+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34150" doReceive() ... 2023-10-21T04:17:26.533+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34156" doAuth() ... 2023-10-21T04:17:26.534+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34156" doReceive() ... 2023-10-21T04:17:26.537+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34158" doAuth() ... 2023-10-21T04:17:26.540+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34158" doReceive() ... 2023-10-21T04:17:26.543+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34160" doAuth() ... 2023-10-21T04:17:26.545+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34160" doReceive() ... 2023-10-21T04:17:26.596+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34172" doAuth() ... 2023-10-21T04:17:26.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34172" doReceive() ... 2023-10-21T04:17:26.619+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34176" doAuth() ... 2023-10-21T04:17:26.621+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34176" doReceive() ... 2023-10-21T04:17:26.656+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34180" doAuth() ... 2023-10-21T04:17:26.658+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34180" doReceive() ... 2023-10-21T04:17:26.979+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:17:26.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 3 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:17:26.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.649µs SnapOpenDur 102.601µs 2023-10-21T04:17:26.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 4 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:17:26.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:140 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.343µs SnapOpenDur 44.398µs 2023-10-21T04:17:26.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 5 Creating New Snapshot SnapshotInfo: count:116 committed:true 2023-10-21T04:17:26.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:116 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.076µs SnapOpenDur 40.912µs 2023-10-21T04:17:26.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:17:26.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.28µs SnapOpenDur 42.161µs 2023-10-21T04:17:26.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:17:26.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.195µs SnapOpenDur 40.886µs 2023-10-21T04:17:26.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 8 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:17:26.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:124 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.159µs SnapOpenDur 40.543µs 2023-10-21T04:17:26.979+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 1 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:17:26.979+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.908µs SnapOpenDur 40.663µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 2 Creating New Snapshot SnapshotInfo: count:110 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:110 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.032µs SnapOpenDur 42.509µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2296526085582506469 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2296526085582506469 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.293µs SnapOpenDur 65.393µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4992762551574016313 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:21 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.229µs SnapOpenDur 53.352µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1265729768899070816 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1265729768899070816 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.047µs SnapOpenDur 82.068µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11050396066566371125 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11050396066566371125 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.187µs SnapOpenDur 56.306µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.139µs SnapOpenDur 51.435µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.878µs SnapOpenDur 67.732µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 4 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.359µs SnapOpenDur 55.119µs 2023-10-21T04:17:26.980+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 5 Creating New Snapshot SnapshotInfo: count:117 committed:true 2023-10-21T04:17:26.980+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:117 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.852µs SnapOpenDur 34.845µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 6 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:137 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.841µs SnapOpenDur 33.918µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 7 Creating New Snapshot SnapshotInfo: count:102 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:102 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.792µs SnapOpenDur 36.152µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 8 Creating New Snapshot SnapshotInfo: count:148 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:148 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.028µs SnapOpenDur 38.726µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 1 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.914µs SnapOpenDur 33.634µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14975131556821987100 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14975131556821987100 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.464µs SnapOpenDur 68.88µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 5 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.809µs SnapOpenDur 91.71µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 6 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:106 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.137µs SnapOpenDur 41.417µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 7 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.967µs SnapOpenDur 41.023µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 8 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.19µs SnapOpenDur 46.832µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 1 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:120 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.439µs SnapOpenDur 43.384µs 2023-10-21T04:17:26.981+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 2 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:17:26.981+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2µs SnapOpenDur 45.279µs 2023-10-21T04:17:26.982+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:17:26.982+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.279µs SnapOpenDur 46.796µs 2023-10-21T04:17:26.982+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 4 Creating New Snapshot SnapshotInfo: count:130 committed:true 2023-10-21T04:17:26.982+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:130 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.088µs SnapOpenDur 45.517µs 2023-10-21T04:17:26.982+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9720186692804226822 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:17:26.982+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9720186692804226822 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.823µs SnapOpenDur 66.031µs 2023-10-21T04:17:26.982+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13739166377675070800 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:17:26.982+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13739166377675070800 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.358µs SnapOpenDur 50.788µs 2023-10-21T04:17:26.982+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.982+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:17:26.983+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14975131556821987100, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:26.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:17:27.120+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 SnapshotId 2 Created recovery point (took 136.277886ms) 2023-10-21T04:17:27.121+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:17:27.121+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:17:27.139+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 SnapshotId 2 Created recovery point (took 155.680617ms) 2023-10-21T04:17:27.140+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.148+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 3 Created recovery point (took 164.427992ms) 2023-10-21T04:17:27.149+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:20 committed:false. num RPs 3. 2023-10-21T04:17:27.171+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 SnapshotId 2 Created recovery point (took 186.946711ms) 2023-10-21T04:17:27.172+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.340+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 SnapshotId 2 Created recovery point (took 355.754306ms) 2023-10-21T04:17:27.341+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.366+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 SnapshotId 2 Created recovery point (took 383.989094ms) 2023-10-21T04:17:27.367+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:17:27.368+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:17:27.390+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 SnapshotId 2 Created recovery point (took 407.352935ms) 2023-10-21T04:17:27.391+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:17:27.392+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:17:27.420+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 SnapshotId 2 Created recovery point (took 437.814408ms) 2023-10-21T04:17:27.422+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 4. 2023-10-21T04:17:27.422+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:17:27.423+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 4. 2023-10-21T04:17:27.423+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:17:27.434+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 SnapshotId 2 Created recovery point (took 451.000153ms) 2023-10-21T04:17:27.435+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:17:27.435+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:17:27.449+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 SnapshotId 2 Created recovery point (took 466.386884ms) 2023-10-21T04:17:27.451+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:17:27.452+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:17:27.461+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 SnapshotId 2 Created recovery point (took 478.29112ms) 2023-10-21T04:17:27.463+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:17:27.463+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:17:27.470+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 SnapshotId 2 Created recovery point (took 487.248971ms) 2023-10-21T04:17:27.472+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:17:27.472+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:17:27.480+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 SnapshotId 2 Created recovery point (took 496.417922ms) 2023-10-21T04:17:27.481+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:17:27.482+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:17:27.489+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 SnapshotId 2 Created recovery point (took 506.233376ms) 2023-10-21T04:17:27.491+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:17:27.492+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:17:27.505+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 SnapshotId 2 Created recovery point (took 521.28375ms) 2023-10-21T04:17:27.513+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:17:27.514+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:17:27.538+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 SnapshotId 2 Created recovery point (took 554.973466ms) 2023-10-21T04:17:27.539+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:17:27.540+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:17:27.557+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14975131556821987100, PartitionId 0 SnapshotId 2 Created recovery point (took 573.09564ms) 2023-10-21T04:17:27.560+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14975131556821987100, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.560+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14975131556821987100, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.573+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 SnapshotId 2 Created recovery point (took 589.731049ms) 2023-10-21T04:17:27.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.578+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.594+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 SnapshotId 2 Created recovery point (took 610.273302ms) 2023-10-21T04:17:27.596+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:17:27.597+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:17:27.610+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 SnapshotId 2 Created recovery point (took 626.209646ms) 2023-10-21T04:17:27.611+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:17:27.612+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:17:27.624+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 SnapshotId 2 Created recovery point (took 639.788474ms) 2023-10-21T04:17:27.625+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:17:27.625+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:17:27.642+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 SnapshotId 2 Created recovery point (took 658.111107ms) 2023-10-21T04:17:27.643+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:17:27.644+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:17:27.651+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 SnapshotId 2 Created recovery point (took 667.130683ms) 2023-10-21T04:17:27.652+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:17:27.653+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:17:27.659+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 SnapshotId 2 Created recovery point (took 675.426742ms) 2023-10-21T04:17:27.661+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:17:27.661+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:17:27.668+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 SnapshotId 2 Created recovery point (took 683.748716ms) 2023-10-21T04:17:27.673+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:17:27.673+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:17:27.681+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 SnapshotId 2 Created recovery point (took 696.783848ms) 2023-10-21T04:17:27.687+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:17:27.687+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:17:27.688+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 SnapshotId 2 Created recovery point (took 704.12115ms) 2023-10-21T04:17:27.689+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.690+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:17:27.704+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 SnapshotId 2 Created recovery point (took 719.402677ms) 2023-10-21T04:17:27.707+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:17:27.707+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:17:27.721+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 SnapshotId 2 Created recovery point (took 737.196496ms) 2023-10-21T04:17:27.723+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:17:27.723+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:17:27.742+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 SnapshotId 2 Created recovery point (took 757.91787ms) 2023-10-21T04:17:27.744+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:17:27.745+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:17:27.756+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 SnapshotId 2 Created recovery point (took 771.41478ms) 2023-10-21T04:17:27.765+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:17:27.766+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:17:30.611+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34284" doAuth() ... 2023-10-21T04:17:30.612+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34284" doReceive() ... 2023-10-21T04:17:30.656+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34294" doAuth() ... 2023-10-21T04:17:30.658+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34294" doReceive() ... 2023-10-21T04:17:30.683+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34300" doAuth() ... 2023-10-21T04:17:30.685+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34300" doReceive() ... 2023-10-21T04:17:30.714+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34306" doAuth() ... 2023-10-21T04:17:30.721+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34306" doReceive() ... 2023-10-21T04:17:30.738+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34308" doAuth() ... 2023-10-21T04:17:30.741+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34308" doReceive() ... 2023-10-21T04:17:30.754+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34312" doAuth() ... 2023-10-21T04:17:30.756+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34312" doReceive() ... 2023-10-21T04:17:30.785+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34318" doAuth() ... 2023-10-21T04:17:30.787+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34318" doReceive() ... 2023-10-21T04:17:30.831+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34322" doAuth() ... 2023-10-21T04:17:30.833+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34322" doReceive() ... 2023-10-21T04:17:30.871+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34328" doAuth() ... 2023-10-21T04:17:30.873+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34328" doReceive() ... 2023-10-21T04:17:30.876+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34330" doAuth() ... 2023-10-21T04:17:30.879+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34330" doReceive() ... 2023-10-21T04:17:30.908+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34334" doAuth() ... 2023-10-21T04:17:30.911+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34334" doReceive() ... 2023-10-21T04:17:30.924+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34338" doAuth() ... 2023-10-21T04:17:30.926+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34338" doReceive() ... 2023-10-21T04:17:30.965+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34344" doAuth() ... 2023-10-21T04:17:30.967+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34344" doReceive() ... 2023-10-21T04:17:30.970+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34346" doAuth() ... 2023-10-21T04:17:30.972+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34346" doReceive() ... 2023-10-21T04:17:30.980+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34348" doAuth() ... 2023-10-21T04:17:30.982+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34348" doReceive() ... 2023-10-21T04:17:30.984+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34350" doAuth() ... 2023-10-21T04:17:30.986+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34350" doReceive() ... 2023-10-21T04:17:31.112+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34366" doAuth() ... 2023-10-21T04:17:31.115+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34366" doReceive() ... 2023-10-21T04:17:31.123+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34368" doAuth() ... 2023-10-21T04:17:31.126+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34368" doReceive() ... 2023-10-21T04:17:31.137+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34372" doAuth() ... 2023-10-21T04:17:31.139+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34372" doReceive() ... 2023-10-21T04:17:31.159+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34376" doAuth() ... 2023-10-21T04:17:31.161+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34376" doReceive() ... 2023-10-21T04:17:31.181+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34378" doAuth() ... 2023-10-21T04:17:31.183+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34378" doReceive() ... 2023-10-21T04:17:31.227+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34384" doAuth() ... 2023-10-21T04:17:31.230+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34384" doReceive() ... 2023-10-21T04:17:31.244+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34388" doAuth() ... 2023-10-21T04:17:31.246+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34388" doReceive() ... 2023-10-21T04:17:31.272+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34392" doAuth() ... 2023-10-21T04:17:31.274+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34392" doReceive() ... 2023-10-21T04:17:31.277+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34394" doAuth() ... 2023-10-21T04:17:31.279+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34394" doReceive() ... 2023-10-21T04:17:31.320+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34404" doAuth() ... 2023-10-21T04:17:31.321+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34404" doReceive() ... 2023-10-21T04:17:31.335+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34408" doAuth() ... 2023-10-21T04:17:31.338+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34408" doReceive() ... 2023-10-21T04:17:31.392+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34418" doAuth() ... 2023-10-21T04:17:31.394+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34418" doReceive() ... 2023-10-21T04:17:31.462+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34426" doAuth() ... 2023-10-21T04:17:31.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34426" doReceive() ... 2023-10-21T04:17:31.467+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34428" doAuth() ... 2023-10-21T04:17:31.470+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34428" doReceive() ... 2023-10-21T04:17:31.517+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34430" doAuth() ... 2023-10-21T04:17:31.517+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34430" doReceive() ... 2023-10-21T04:17:31.539+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34432" doAuth() ... 2023-10-21T04:17:31.543+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34432" doReceive() ... 2023-10-21T04:17:31.572+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34442" doAuth() ... 2023-10-21T04:17:31.577+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34442" doReceive() ... 2023-10-21T04:17:31.634+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34448" doAuth() ... 2023-10-21T04:17:31.636+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34448" doReceive() ... 2023-10-21T04:17:31.649+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34452" doAuth() ... 2023-10-21T04:17:31.651+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34452" doReceive() ... 2023-10-21T04:17:35.354+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34512" doAuth() ... 2023-10-21T04:17:35.355+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34512" doReceive() ... 2023-10-21T04:17:35.408+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34514" doAuth() ... 2023-10-21T04:17:35.409+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34514" doReceive() ... 2023-10-21T04:17:35.412+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34516" doAuth() ... 2023-10-21T04:17:35.413+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34516" doReceive() ... 2023-10-21T04:17:35.415+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34518" doAuth() ... 2023-10-21T04:17:35.417+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34518" doReceive() ... 2023-10-21T04:17:35.426+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34520" doAuth() ... 2023-10-21T04:17:35.427+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34520" doReceive() ... 2023-10-21T04:17:35.429+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34522" doAuth() ... 2023-10-21T04:17:35.430+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34522" doReceive() ... 2023-10-21T04:17:35.437+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34524" doAuth() ... 2023-10-21T04:17:35.438+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34524" doReceive() ... 2023-10-21T04:17:35.445+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34526" doAuth() ... 2023-10-21T04:17:35.446+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34526" doReceive() ... 2023-10-21T04:17:35.465+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34530" doAuth() ... 2023-10-21T04:17:35.465+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34530" doReceive() ... 2023-10-21T04:17:35.469+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34532" doAuth() ... 2023-10-21T04:17:35.470+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34532" doReceive() ... 2023-10-21T04:17:35.473+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34534" doAuth() ... 2023-10-21T04:17:35.474+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34534" doReceive() ... 2023-10-21T04:17:35.496+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34540" doAuth() ... 2023-10-21T04:17:35.497+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34540" doReceive() ... 2023-10-21T04:17:35.511+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34542" doAuth() ... 2023-10-21T04:17:35.513+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34542" doReceive() ... 2023-10-21T04:17:35.522+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34548" doAuth() ... 2023-10-21T04:17:35.523+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34548" doReceive() ... 2023-10-21T04:17:35.553+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34554" doAuth() ... 2023-10-21T04:17:35.553+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34554" doReceive() ... 2023-10-21T04:17:35.562+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34556" doAuth() ... 2023-10-21T04:17:35.563+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34556" doReceive() ... 2023-10-21T04:17:35.565+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34558" doAuth() ... 2023-10-21T04:17:35.566+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34558" doReceive() ... 2023-10-21T04:17:35.584+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34562" doAuth() ... 2023-10-21T04:17:35.585+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34562" doReceive() ... 2023-10-21T04:17:35.586+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34564" doAuth() ... 2023-10-21T04:17:35.587+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34564" doReceive() ... 2023-10-21T04:17:35.601+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34566" doAuth() ... 2023-10-21T04:17:35.601+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34566" doReceive() ... 2023-10-21T04:17:35.604+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34568" doAuth() ... 2023-10-21T04:17:35.604+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34568" doReceive() ... 2023-10-21T04:17:35.613+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34570" doAuth() ... 2023-10-21T04:17:35.613+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34570" doReceive() ... 2023-10-21T04:17:35.615+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34574" doAuth() ... 2023-10-21T04:17:35.616+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34574" doReceive() ... 2023-10-21T04:17:35.625+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34576" doAuth() ... 2023-10-21T04:17:35.626+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34576" doReceive() ... 2023-10-21T04:17:35.628+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34578" doAuth() ... 2023-10-21T04:17:35.629+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34578" doReceive() ... 2023-10-21T04:17:35.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34580" doAuth() ... 2023-10-21T04:17:35.633+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34580" doReceive() ... 2023-10-21T04:17:35.653+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34584" doAuth() ... 2023-10-21T04:17:35.654+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34584" doReceive() ... 2023-10-21T04:17:35.661+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34588" doAuth() ... 2023-10-21T04:17:35.662+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34588" doReceive() ... 2023-10-21T04:17:35.664+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34590" doAuth() ... 2023-10-21T04:17:35.666+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34590" doReceive() ... 2023-10-21T04:17:35.708+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34604" doAuth() ... 2023-10-21T04:17:35.710+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34604" doReceive() ... 2023-10-21T04:17:35.739+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34606" doAuth() ... 2023-10-21T04:17:35.739+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34606" doReceive() ... 2023-10-21T04:17:35.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34608" doAuth() ... 2023-10-21T04:17:35.774+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34608" doReceive() ... 2023-10-21T04:17:35.795+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34612" doAuth() ... 2023-10-21T04:17:35.796+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34612" doReceive() ... 2023-10-21T04:17:35.802+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34616" doAuth() ... 2023-10-21T04:17:35.803+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34616" doReceive() ... 2023-10-21T04:17:35.805+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34618" doAuth() ... 2023-10-21T04:17:35.806+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34618" doReceive() ... 2023-10-21T04:17:35.818+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34622" doAuth() ... 2023-10-21T04:17:35.820+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34622" doReceive() ... 2023-10-21T04:17:35.836+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34624" doAuth() ... 2023-10-21T04:17:35.837+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34624" doReceive() ... 2023-10-21T04:17:35.869+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34626" doAuth() ... 2023-10-21T04:17:35.870+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34626" doReceive() ... 2023-10-21T04:17:35.895+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34630" doAuth() ... 2023-10-21T04:17:35.895+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34630" doReceive() ... 2023-10-21T04:17:35.920+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34638" doAuth() ... 2023-10-21T04:17:35.921+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34638" doReceive() ... 2023-10-21T04:17:35.926+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34642" doAuth() ... 2023-10-21T04:17:35.927+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34642" doReceive() ... 2023-10-21T04:17:35.937+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34644" doAuth() ... 2023-10-21T04:17:35.938+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34644" doReceive() ... 2023-10-21T04:17:35.947+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34650" doAuth() ... 2023-10-21T04:17:35.949+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34650" doReceive() ... 2023-10-21T04:17:35.951+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34652" doAuth() ... 2023-10-21T04:17:35.952+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34652" doReceive() ... 2023-10-21T04:17:35.958+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34654" doAuth() ... 2023-10-21T04:17:35.961+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34654" doReceive() ... 2023-10-21T04:17:35.973+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34658" doAuth() ... 2023-10-21T04:17:35.974+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34658" doReceive() ... 2023-10-21T04:17:35.991+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34662" doAuth() ... 2023-10-21T04:17:35.993+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34662" doReceive() ... 2023-10-21T04:17:35.995+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34664" doAuth() ... 2023-10-21T04:17:35.998+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34664" doReceive() ... 2023-10-21T04:17:36.000+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34666" doAuth() ... 2023-10-21T04:17:36.001+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34666" doReceive() ... 2023-10-21T04:17:36.035+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34670" doAuth() ... 2023-10-21T04:17:36.038+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34670" doReceive() ... 2023-10-21T04:17:36.085+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34672" doAuth() ... 2023-10-21T04:17:36.089+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34672" doReceive() ... 2023-10-21T04:17:36.092+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34674" doAuth() ... 2023-10-21T04:17:36.094+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34674" doReceive() ... 2023-10-21T04:17:36.116+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34680" doAuth() ... 2023-10-21T04:17:36.116+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34680" doReceive() ... 2023-10-21T04:17:36.119+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34682" doAuth() ... 2023-10-21T04:17:36.120+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34682" doReceive() ... 2023-10-21T04:17:36.141+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34688" doAuth() ... 2023-10-21T04:17:36.144+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34688" doReceive() ... 2023-10-21T04:17:36.188+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34696" doAuth() ... 2023-10-21T04:17:36.191+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34696" doReceive() ... 2023-10-21T04:17:36.210+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34700" doAuth() ... 2023-10-21T04:17:36.212+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34700" doReceive() ... 2023-10-21T04:17:36.231+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34704" doAuth() ... 2023-10-21T04:17:36.234+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34704" doReceive() ... 2023-10-21T04:17:36.236+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34706" doAuth() ... 2023-10-21T04:17:36.238+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34706" doReceive() ... 2023-10-21T04:17:36.252+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34708" doAuth() ... 2023-10-21T04:17:36.254+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34708" doReceive() ... 2023-10-21T04:17:36.268+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34710" doAuth() ... 2023-10-21T04:17:36.271+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34710" doReceive() ... 2023-10-21T04:17:36.315+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34718" doAuth() ... 2023-10-21T04:17:36.317+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34718" doReceive() ... 2023-10-21T04:17:36.343+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34724" doAuth() ... 2023-10-21T04:17:36.345+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34724" doReceive() ... 2023-10-21T04:17:36.348+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34726" doAuth() ... 2023-10-21T04:17:36.350+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34726" doReceive() ... 2023-10-21T04:17:36.352+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34728" doAuth() ... 2023-10-21T04:17:36.355+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34728" doReceive() ... 2023-10-21T04:17:36.382+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34730" doAuth() ... 2023-10-21T04:17:36.384+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34730" doReceive() ... 2023-10-21T04:17:36.397+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34732" doAuth() ... 2023-10-21T04:17:36.399+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34732" doReceive() ... 2023-10-21T04:17:36.401+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34734" doAuth() ... 2023-10-21T04:17:36.403+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34734" doReceive() ... 2023-10-21T04:17:36.405+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34736" doAuth() ... 2023-10-21T04:17:36.407+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34736" doReceive() ... 2023-10-21T04:17:36.437+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34740" doAuth() ... 2023-10-21T04:17:36.439+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34740" doReceive() ... 2023-10-21T04:17:36.456+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34746" doAuth() ... 2023-10-21T04:17:36.458+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34746" doReceive() ... 2023-10-21T04:17:36.466+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34748" doAuth() ... 2023-10-21T04:17:36.469+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34748" doReceive() ... 2023-10-21T04:17:36.485+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34756" doAuth() ... 2023-10-21T04:17:36.487+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34756" doReceive() ... 2023-10-21T04:17:36.495+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34758" doAuth() ... 2023-10-21T04:17:36.498+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34758" doReceive() ... 2023-10-21T04:17:38.481+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 14213410630307008054 &{0} 2023-10-21T04:17:38.481+05:30 [Info] Indexer::handleDropIndex - IndexInstId 14213410630307008054 2023-10-21T04:17:38.481+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:17:38.482+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:17:38.482+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14213410630307008054 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:17:38.483+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [14213410630307008054] 2023-10-21T04:17:38.483+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [14213410630307008054] 2023-10-21T04:17:38.483+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:17:38.483+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [14213410630307008054] 2023-10-21T04:17:38.485+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14213410630307008054, PartitionId 0, IndexDefnId 2048006213723315482. 2023-10-21T04:17:38.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:17:38.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:17:38.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:17:38.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex] 2023-10-21T04:17:38.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex 2023-10-21T04:17:38.485+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex] 2023-10-21T04:17:38.485+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex] 2023-10-21T04:17:38.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex 2023-10-21T04:17:38.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex stopped 2023-10-21T04:17:38.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:17:38.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:17:38.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:17:38.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:17:38.484+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:17:38.487+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14213410630307008054 Defn: DefnId: 2048006213723315482 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:17:38.487+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 6.808431ms 2023-10-21T04:17:38.488+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [14213410630307008054] 2023-10-21T04:17:38.487+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 14213410630307008054 2023-10-21T04:17:38.493+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:17:38.493+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:17:38.493+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.025598ms 2023-10-21T04:17:38.493+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 12.902964ms 2023-10-21T04:17:38.507+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.507+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.511+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:17:38.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex] 2023-10-21T04:17:38.513+05:30 [Info] switched currmeta from 502 -> 503 force true 2023-10-21T04:17:38.514+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 40.162087ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:17:38.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex closed 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:17:38.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex] 2023-10-21T04:17:38.516+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex 2023-10-21T04:17:38.516+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex] 2023-10-21T04:17:38.516+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex] 2023-10-21T04:17:38.516+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex 2023-10-21T04:17:38.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex stopped 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:17:38.516+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:17:38.518+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:17:38.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex] 2023-10-21T04:17:38.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex closed 2023-10-21T04:17:38.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:17:38.520+05:30 [Info] Indexer::cleanupIndexData IndexInst 14213410630307008054 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:17:38.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex ... 2023-10-21T04:17:38.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 138 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex ... 2023-10-21T04:17:38.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:17:38.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/docIndex sucessfully destroyed 2023-10-21T04:17:38.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex ... 2023-10-21T04:17:38.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 154 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex ... 2023-10-21T04:17:38.532+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.532+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:17:38.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14213410630307008054_0.index destroyed 2023-10-21T04:17:38.538+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14213410630307008054, PartitionId 0, IndexDefnId 2048006213723315482. 2023-10-21T04:17:38.538+05:30 [Info] Indexer::cleanupIndexData IndexInst 14213410630307008054 Partition 0 Destroy Done 2023-10-21T04:17:38.543+05:30 [Info] switched currmeta from 503 -> 504 force true 2023-10-21T04:17:38.556+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.556+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.563+05:30 [Info] switched currmeta from 504 -> 505 force true 2023-10-21T04:17:38.575+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.576+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.581+05:30 [Info] switched currmeta from 505 -> 506 force true 2023-10-21T04:17:38.592+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.592+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.598+05:30 [Info] switched currmeta from 506 -> 506 force true 2023-10-21T04:17:38.636+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 14975131556821987100 &{0} 2023-10-21T04:17:38.636+05:30 [Info] Indexer::handleDropIndex - IndexInstId 14975131556821987100 2023-10-21T04:17:38.637+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:17:38.637+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:17:38.637+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14975131556821987100 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:17:38.639+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [14975131556821987100] 2023-10-21T04:17:38.639+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [14975131556821987100] 2023-10-21T04:17:38.639+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:17:38.639+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [14975131556821987100] 2023-10-21T04:17:38.641+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14975131556821987100, PartitionId 0, IndexDefnId 249887442666643345. 2023-10-21T04:17:38.641+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:17:38.641+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:17:38.641+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:17:38.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex] 2023-10-21T04:17:38.641+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex 2023-10-21T04:17:38.641+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex] 2023-10-21T04:17:38.641+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex] 2023-10-21T04:17:38.641+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex 2023-10-21T04:17:38.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex stopped 2023-10-21T04:17:38.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:17:38.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:17:38.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:17:38.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:17:38.641+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:17:38.642+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14975131556821987100 Defn: DefnId: 249887442666643345 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:17:38.642+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 5.686119ms 2023-10-21T04:17:38.642+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 14975131556821987100 2023-10-21T04:17:38.642+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [14975131556821987100] 2023-10-21T04:17:38.646+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:17:38.646+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:17:38.646+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 3.755828ms 2023-10-21T04:17:38.646+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 9.512445ms 2023-10-21T04:17:38.648+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:17:38.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex] 2023-10-21T04:17:38.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex closed 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:17:38.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex] 2023-10-21T04:17:38.649+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex] 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex] 2023-10-21T04:17:38.649+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex 2023-10-21T04:17:38.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex stopped 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:17:38.649+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:17:38.661+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 34.49983ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:17:38.666+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.666+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.681+05:30 [Info] switched currmeta from 506 -> 509 force true 2023-10-21T04:17:38.681+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:17:38.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex] 2023-10-21T04:17:38.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex closed 2023-10-21T04:17:38.683+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:17:38.683+05:30 [Info] Indexer::cleanupIndexData IndexInst 14975131556821987100 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:17:38.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex ... 2023-10-21T04:17:38.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 138 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex ... 2023-10-21T04:17:38.694+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.694+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:17:38.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/docIndex sucessfully destroyed 2023-10-21T04:17:38.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex ... 2023-10-21T04:17:38.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 154 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex ... 2023-10-21T04:17:38.697+05:30 [Info] switched currmeta from 509 -> 509 force true 2023-10-21T04:17:38.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:17:38.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_14975131556821987100_0.index destroyed 2023-10-21T04:17:38.705+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14975131556821987100, PartitionId 0, IndexDefnId 249887442666643345. 2023-10-21T04:17:38.705+05:30 [Info] Indexer::cleanupIndexData IndexInst 14975131556821987100 Partition 0 Destroy Done 2023-10-21T04:17:38.714+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.714+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.720+05:30 [Info] switched currmeta from 509 -> 510 force true 2023-10-21T04:17:38.727+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:38.727+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:38.732+05:30 [Info] switched currmeta from 510 -> 510 force true 2023-10-21T04:17:39.493+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:17:39.498+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:39.498+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:39.500+05:30 [Info] switched currmeta from 510 -> 510 force true 2023-10-21T04:17:42.012+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:17:54.269+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:17:54.269+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:17:54.269+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:17:54.269+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:17:54.269+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:17:54.271+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:17:54.271+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:17:59.521+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 8504686614261377811 2023-10-21T04:17:59.521+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 382.869µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:17:59.567+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36760 2023-10-21T04:17:59.571+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36760 2023-10-21T04:17:59.580+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36760 2023-10-21T04:17:59.695+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 8504686614261377811, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:17:59.695+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 8504686614261377811, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:17:59.705+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 6261854707981462523, indexDefn DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:17:59.706+05:30 [Info] Indexer::handleCreateIndex InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:17:59.709+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 6261854707981462523 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:17:59.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:17:59.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:17:59.718+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:17:59.718+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:17:59.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:17:59.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:17:59.729+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:17:59.730+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:17:59.741+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 166 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex 2023-10-21T04:17:59.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:17:59.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:17:59.748+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex 2023-10-21T04:17:59.750+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex 2023-10-21T04:17:59.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex started 2023-10-21T04:17:59.753+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:17:59.753+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:17:59.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 185 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex 2023-10-21T04:17:59.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:17:59.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:17:59.755+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex 2023-10-21T04:17:59.755+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex 2023-10-21T04:17:59.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex started 2023-10-21T04:17:59.756+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:17:59.756+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:17:59.757+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 6261854707981462523 partitionId 0 WriterThreads 4 2023-10-21T04:17:59.758+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 6261854707981462523 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index Index: 6261854707981462523 Partition: 0 2023-10-21T04:17:59.761+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 6261854707981462523 2023-10-21T04:17:59.761+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:17:59.762+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:59.762+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:59.762+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:17:59.764+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:17:59.765+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:17:59.765+05:30 [Info] switched currmeta from 510 -> 513 force true 2023-10-21T04:17:59.765+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 59.680092ms 2023-10-21T04:17:59.766+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 6261854707981462523, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], indexDefn DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:17:59.766+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 60.060379ms 2023-10-21T04:17:59.777+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:59.777+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:59.783+05:30 [Info] switched currmeta from 513 -> 514 force true 2023-10-21T04:17:59.795+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:59.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:59.808+05:30 [Info] switched currmeta from 514 -> 516 force true 2023-10-21T04:17:59.818+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [6261854707981462523] &{0} 2023-10-21T04:17:59.818+05:30 [Info] Indexer::handleBuildIndex [6261854707981462523] 2023-10-21T04:17:59.822+05:30 [Info] bulkUpdateRState: Index instance 6261854707981462523 rstate moved to ACTIVE 2023-10-21T04:17:59.822+05:30 [Info] Indexer::handleBuildIndex Added Index: [6261854707981462523] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:17:59.823+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:17:59.823+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:17:59.823+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:17:59.824+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:17:59.824+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:17:59.824+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:17:59.824+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65895 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:17:59.824+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:17:59.824+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:17:59.824+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:17:59.824+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65895 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:17:59.824+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:17:59.824+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:17:59.824+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:17:59.824+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:17:59.824+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:17:59.824+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:17:59.824+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:17:59.824+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:17:59.825+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:10] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:17:59.825+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 6261854707981462523 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:17:59.825+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:17:59.825+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [ InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 65896 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:17:59.825+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:17:59.825+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:17:59.826+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:17:59.826+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 7.988194ms 2023-10-21T04:17:59.826+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 8.030232ms 2023-10-21T04:17:59.826+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [6261854707981462523] 2023-10-21T04:17:59.830+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:17:59.836+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:17:59.837+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 8504686614261377811 2023-10-21T04:17:59.837+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 142.454251ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:17:59.849+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.381896ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:17:59.873+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:59.874+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:59.888+05:30 [Info] switched currmeta from 516 -> 519 force true 2023-10-21T04:17:59.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:17:59.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:17:59.897+05:30 [Info] switched currmeta from 519 -> 519 force true 2023-10-21T04:17:59.902+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:52684" doAuth() ... 2023-10-21T04:17:59.904+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:52684" auth successful 2023-10-21T04:17:59.905+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:52684" +1 2023-10-21T04:17:59.959+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:17:59.959+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:17:59.965+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51928 2023-10-21T04:17:59.966+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:17:59.992+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 11.221397ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:17:59.993+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:17:59.993+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:17:59.993+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:17:59.998+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:18:00.028+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:18:00.028+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:18:00.028+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:18:00.028+05:30 [Info] DDLServiceMgr: processing create index token /indexing/ddl/commandToken/create/8504686614261377811/0 2023-10-21T04:18:00.035+05:30 [Info] DDLServiceMgr: Update Replica Count. Index Defn 8504686614261377811 replica Count {true 1 0 0} 2023-10-21T04:18:00.036+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 166.456µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:00.068+05:30 [Info] DDLServiceMgr: Remove create index token 8504686614261377811. 2023-10-21T04:18:00.069+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:18:00.069+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:18:00.069+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:18:00.071+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:51928->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:18:00.071+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:48306->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:18:00.071+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51928. Reason = EOF. Cleaning Pipe. 2023-10-21T04:18:00.071+05:30 [Info] messageListener.start(): message channel closed. Remove peer a5:d8:8b:56:18:94:0:d as follower. 2023-10-21T04:18:00.142+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51946 2023-10-21T04:18:00.202+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 44.436243ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:00.241+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51946. Reason = EOF. Cleaning Pipe. 2023-10-21T04:18:00.241+05:30 [Info] messageListener.start(): message channel closed. Remove peer 45:e8:e7:fc:c4:83:65:ca as follower. 2023-10-21T04:18:00.299+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 186517626268899 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.301+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 178840478062605 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.308+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 28605411869539 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.308+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 215374859564337 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.308+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 148075163102199 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.314+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 39552632604610 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.316+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 242327631330054 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.317+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 191903564840858 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.320+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 50961953886801 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.324+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 274410200699189 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.324+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 225280996447309 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.324+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 66881023717063 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.331+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 226814108379280 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.332+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 127185710837163 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.336+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 242578104468179 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.342+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 210306088158507 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 183148377680315 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 230239359684242 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 4429827396609 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 257031483891250 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 55290066231400 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 209771427122473 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 260036488774725 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 216462437971699 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.358+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 160961818008256 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.360+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 102578907139247 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.360+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 280874278672777 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.360+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 219132834127088 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 184056281482149 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 99433130332778 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 45957426777107 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.370+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 217381278987664 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.375+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 136958029391999 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 82469662608686 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.388+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 28417669987133 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.393+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 253845164956592 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.395+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 253494845579900 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.397+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 70213628416850 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.397+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 188107826286720 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.401+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 5915864305121 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.403+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 259381390638966 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.405+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 252931432434966 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.408+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 158545118983293 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.411+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 12301831552869 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.413+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 242994859770444 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.419+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 227378777358533 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.423+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 215467668600630 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.425+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 207367350302991 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.429+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 188718367400378 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.430+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 279362078165491 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.432+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 264835102027079 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.443+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 16044440367455 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.443+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 125009072873642 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.445+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 222555662236724 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.446+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 170056388266420 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.446+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 200436537417141 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.446+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 75566139929862 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.446+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 129411666769864 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.447+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 146267938201125 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.447+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 204867593700407 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.447+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 271074273345339 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.448+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 10527297481171 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.448+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 106932309485550 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.452+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c1 InstanceIds [6261854707981462523] 2023-10-21T04:18:00.452+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 65896 2023-10-21T04:18:00.452+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:00.452+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:00.453+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_1:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:18:00.453+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 54 Crc64: 11593925038715171663 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 22 0 55} { 1 a2a78b2a980d 2 33 0 57} { 2 86ac6f7f2bf7 2 33 0 54} { 3 c3e1ddff7531 2 20 0 35} { 4 1a043769c363 0 0 0 0} { 5 23f9109dc7c2 0 0 0 0} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 0 0 0 0} { 8 2e5981289251 0 0 0 0} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} HWT bucket: bucket_1, scopeId: , collectionId: , vbuckets: 63 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 22 0 55} { 1 a2a78b2a980d 2 57 0 57} { 2 86ac6f7f2bf7 2 33 0 54} { 3 c3e1ddff7531 2 35 0 35} { 4 1a043769c363 0 0 0 0} { 5 23f9109dc7c2 0 0 0 0} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 0 0 0 0} { 8 2e5981289251 0 0 0 0} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} { 61 61411e5267ee 0 0 0 0} { 62 99313d3a9d3 0 0 0 0} 2023-10-21T04:18:00.454+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:18:00.456+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:18:00.459+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 133748675726909 0 65896 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:00.459+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:00.460+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:00.460+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:00.494+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:18:00.704+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:18:00.906+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:00.911+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 6261854707981462523 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:00.911+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 6261854707981462523 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.63µs SnapOpenDur 59.223µs 2023-10-21T04:18:00.911+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 6261854707981462523 INDEX_STATE_CATCHUP 2023-10-21T04:18:00.911+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 6261854707981462523 true 2023-10-21T04:18:00.911+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 6261854707981462523 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 65896 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] 2023-10-21T04:18:00.911+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:00.913+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 65896 2023-10-21T04:18:00.913+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 65896 2023-10-21T04:18:00.913+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:00.914+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:00.914+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:00.914+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:00.915+05:30 [Info] send progress stats to clients 2023-10-21T04:18:00.915+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:18:00.915+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:00.915+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:11] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:18:00.915+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 65896 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:00.917+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:00.928+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.449623ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:00.941+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:00.941+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:00.946+05:30 [Info] switched currmeta from 519 -> 520 force true 2023-10-21T04:18:00.949+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 2023-10-21T04:18:00.949+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65896 2023-10-21T04:18:00.950+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65896 2023-10-21T04:18:00.951+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:18:00.951+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:00.951+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 6261854707981462523 false 2023-10-21T04:18:00.951+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 6261854707981462523 2023-10-21T04:18:00.951+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:18:00.951+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 65896 2023-10-21T04:18:00.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:00.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:00.970+05:30 [Info] switched currmeta from 520 -> 521 force true 2023-10-21T04:18:00.987+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 SnapshotId 2 Created recovery point (took 75.257106ms) 2023-10-21T04:18:01.459+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:01.459+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:01.459+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 6261854707981462523 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 65896, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 55 0 55} { 1 a2a78b2a980d 2 57 0 57} { 2 86ac6f7f2bf7 2 54 0 54} { 3 c3e1ddff7531 2 35 0 35} { 4 1a043769c363 2 58 0 58} { 5 23f9109dc7c2 2 52 0 52} { 6 dc654c627b06 2 62 0 62} { 7 ae8907d03f9a 2 63 0 63} { 8 2e5981289251 2 45 0 45} { 9 cce451c4944d 2 55 0 55} { 10 f9931a891135 2 54 0 54} { 11 3cd3f3a92ec7 2 51 0 51} { 12 4076649d401 2 42 0 42} { 13 a6928dfe29bb 2 46 0 46} { 14 ce49465c2890 2 72 0 72} { 15 73acbb1d6dab 2 50 0 50} { 16 e9c4ce4b1e32 2 58 0 58} { 17 324938c1e068 2 51 0 51} { 18 dc9f9dbf42d3 2 63 0 63} { 19 bf45b3adc52b 2 48 0 48} { 20 d166c73f4e92 2 50 0 50} { 21 bec93765cd29 2 48 0 48} { 22 ec80768ac445 2 51 0 51} { 23 c4df16b8f6f3 2 57 0 57} { 24 9264d7fc1ac0 2 59 0 59} { 25 ff74239b5989 2 71 0 71} { 26 c74cd6bc9cf0 2 52 0 52} { 27 a765f146c3a5 2 49 0 49} { 28 5d4b83524caf 2 58 0 58} { 29 29cc4c190c13 2 57 0 57} { 30 5a6f1469926a 2 58 0 58} { 31 c5b505ea3590 2 67 0 67} { 32 4b0176f1cd2e 2 60 0 60} { 33 7c9006b0e87f 2 65 0 65} { 34 19d8811ffb3d 2 58 0 58} { 35 3fdbe2615352 2 47 0 47} { 36 e6deeee29bb0 2 58 0 58} { 37 e68d5e39227c 2 54 0 54} { 38 ab15442a5080 2 60 0 60} { 39 ca69c730bc34 2 67 0 67} { 40 56164fd8de1 2 60 0 60} { 41 ebe7efa67f76 2 58 0 58} { 42 e60a302ecd16 2 46 0 46} { 43 8507a87f9e25 2 55 0 55} { 44 9032297f747d 2 67 0 67} { 45 b303e4cbf65 2 66 0 66} { 46 dd00a64c824c 2 54 0 54} { 47 f68a65c53f3b 2 53 0 53} { 48 ceccbf4100c5 2 59 0 59} { 49 c3f779d8f736 2 34 0 34} { 50 bc997940b90f 2 64 0 64} { 51 b64bc49cddb5 2 54 0 54} { 52 aba36b4061ba 2 56 0 56} { 53 ba5374260437 2 56 0 56} { 54 fe140d6ec9f3 2 55 0 55} { 55 9aaa569d49b4 2 70 0 70} { 56 f0ddba2e2d47 2 53 0 53} { 57 75b3006d77c8 2 55 0 55} { 58 e97a32b315f 2 53 0 53} { 59 71b1f161e0aa 2 53 0 53} { 60 44ba1ced7506 2 56 0 56} { 61 61411e5267ee 2 53 0 53} { 62 99313d3a9d3 2 57 0 57} { 63 79a4ca4e4e3d 2 56 0 56} 2023-10-21T04:18:01.460+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 6261854707981462523 INDEX_STATE_ACTIVE 2023-10-21T04:18:01.461+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:18:01.461+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:18:01.462+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:01.462+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 1 New 0 2023-10-21T04:18:01.463+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 65896 2023-10-21T04:18:01.464+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:01.465+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:01.465+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:01.466+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65896 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:01.467+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 6261854707981462523 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:18:01.467+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:18:01.467+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:01.467+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:01.467+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65896 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:01.468+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:01.468+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:18 1:14 2:15 3:7 4:19 5:8 6:18 7:18 8:9 9:11 10:17 11:12 12:14 13:15 14:21 15:12 16:10 17:12 18:18 19:8 20:15 21:8 22:14 23:13 24:11 25:23 26:11 27:16 28:17 29:18 30:14 31:19 32:17 33:17 34:17 35:15 36:14 37:15 38:16 39:25 40:22 41:17 42:11 43:13 44:33 45:16 46:17 47:14 48:17 49:14 50:22 51:23 52:17 53:13 54:23 55:22 56:10 57:13 58:14 59:13 60:16 61:14 62:19 63:16] 2023-10-21T04:18:01.468+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:36 1:34 2:36 3:21 4:38 5:38 6:46 7:46 8:34 9:31 10:42 11:34 12:34 13:31 14:48 15:36 16:35 17:34 18:45 19:29 20:35 21:32 22:32 23:34 24:36 25:50 26:29 27:35 28:36 29:38 30:44 31:44 32:41 33:43 34:41 35:37 36:37 37:41 38:38 39:46 40:43 41:38 42:34 43:34 44:55 45:41 46:42 47:37 48:42 49:28 50:47 51:43 52:40 53:38 54:45 55:46 56:39 57:38 58:33 59:41 60:41 61:36 62:41 63:40] 2023-10-21T04:18:01.468+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:01.468+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:52684: use of closed network connection 2023-10-21T04:18:01.468+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:52684" exit: read tcp4 127.0.0.1:9121->127.0.0.1:52684: use of closed network connection 2023-10-21T04:18:01.468+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:52684" closed ! 2023-10-21T04:18:01.468+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:01.468+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:01.468+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:01.468+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:01.469+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:01.469+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:01.469+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:01.469+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 65896 2023-10-21T04:18:01.471+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:18:01.483+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.586216ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:01.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:01.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:01.503+05:30 [Info] switched currmeta from 521 -> 523 force true 2023-10-21T04:18:01.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:01.518+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:01.531+05:30 [Info] switched currmeta from 523 -> 523 force true 2023-10-21T04:18:01.543+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:18:01.544+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:01.544+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:18:01.544+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65896 2023-10-21T04:18:01.544+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65896 2023-10-21T04:18:01.544+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:02.004+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6975716383766796239:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:02.004+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6975716383766796239:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6261854707981462523:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:02.880+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7582778950442222621 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7582778950442222621 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.765µs SnapOpenDur 101.331µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3100906924534408756 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3100906924534408756 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.872µs SnapOpenDur 55.069µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15667952562388529486 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15667952562388529486 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.798µs SnapOpenDur 45.94µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 5 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:131 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.758µs SnapOpenDur 51.296µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 6 Creating New Snapshot SnapshotInfo: count:115 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:115 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.923µs SnapOpenDur 42.936µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 7 Creating New Snapshot SnapshotInfo: count:109 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:109 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.766µs SnapOpenDur 42.661µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 8 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:128 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.913µs SnapOpenDur 40.195µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 1 Creating New Snapshot SnapshotInfo: count:135 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:135 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2µs SnapOpenDur 62.682µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 2 Creating New Snapshot SnapshotInfo: count:107 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:107 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.075µs SnapOpenDur 38.482µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 3 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:138 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.849µs SnapOpenDur 35.948µs 2023-10-21T04:18:02.881+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 4 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:18:02.881+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:137 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.248µs SnapOpenDur 45.482µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5004576769623010783 PartitionId 0 Creating New Snapshot SnapshotInfo: count:20 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5004576769623010783 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:20 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.125µs SnapOpenDur 56.528µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 6261854707981462523 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 6261854707981462523 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.79µs SnapOpenDur 51.691µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.159µs SnapOpenDur 56.684µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 2 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:120 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.192µs SnapOpenDur 42.876µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 3 Creating New Snapshot SnapshotInfo: count:139 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:139 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2µs SnapOpenDur 40.599µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 4 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:112 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.076µs SnapOpenDur 38.028µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 5 Creating New Snapshot SnapshotInfo: count:133 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:133 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.952µs SnapOpenDur 38.137µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 6 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.198µs SnapOpenDur 42.809µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 7 Creating New Snapshot SnapshotInfo: count:119 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:119 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.07µs SnapOpenDur 39.79µs 2023-10-21T04:18:02.882+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 8 Creating New Snapshot SnapshotInfo: count:122 committed:true 2023-10-21T04:18:02.882+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:122 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.083µs SnapOpenDur 44.543µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 5 Creating New Snapshot SnapshotInfo: count:111 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:111 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.205µs SnapOpenDur 62.989µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 6 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:140 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.183µs SnapOpenDur 39.147µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:125 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.211µs SnapOpenDur 40.985µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 8 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.061µs SnapOpenDur 40.717µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.023µs SnapOpenDur 37.041µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 2 Creating New Snapshot SnapshotInfo: count:113 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:113 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.861µs SnapOpenDur 39.806µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:132 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.902µs SnapOpenDur 44.369µs 2023-10-21T04:18:02.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 4 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:18:02.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:124 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.971µs SnapOpenDur 41.952µs 2023-10-21T04:18:02.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 962468716067699903 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:02.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 962468716067699903 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.689µs SnapOpenDur 75.684µs 2023-10-21T04:18:02.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7494249569853685720 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:02.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7494249569853685720 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.495µs SnapOpenDur 59.283µs 2023-10-21T04:18:02.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.885+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.886+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0 SnapshotId 2 Created recovery point (took 11.602651ms) 2023-10-21T04:18:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:02.913+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0 SnapshotId 2 Created recovery point (took 28.766009ms) 2023-10-21T04:18:02.914+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:02.957+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0 SnapshotId 2 Created recovery point (took 72.124306ms) 2023-10-21T04:18:02.958+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:03.027+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 220] aggregate storage info: map[bucket_%2:map[kv:5716160 index:61540902] bucket_1:map[kv:6441152 index:50644830]] 2023-10-21T04:18:03.057+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0 SnapshotId 2 Created recovery point (took 171.700302ms) 2023-10-21T04:18:03.058+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:18:03.116+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 SnapshotId 3 Created recovery point (took 231.078896ms) 2023-10-21T04:18:03.391+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 SnapshotId 2 Created recovery point (took 506.445983ms) 2023-10-21T04:18:03.392+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:18:03.392+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:18:03.417+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 SnapshotId 2 Created recovery point (took 532.281045ms) 2023-10-21T04:18:03.418+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:03.418+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:03.447+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 SnapshotId 2 Created recovery point (took 562.276379ms) 2023-10-21T04:18:03.448+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:18:03.449+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:18:03.466+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 SnapshotId 2 Created recovery point (took 581.114507ms) 2023-10-21T04:18:03.467+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:18:03.468+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:18:03.478+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 SnapshotId 2 Created recovery point (took 593.273789ms) 2023-10-21T04:18:03.479+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:18:03.480+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:18:03.492+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 SnapshotId 2 Created recovery point (took 607.01788ms) 2023-10-21T04:18:03.493+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 4. 2023-10-21T04:18:03.493+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:18:03.494+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 4. 2023-10-21T04:18:03.495+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:18:03.503+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 SnapshotId 2 Created recovery point (took 617.516402ms) 2023-10-21T04:18:03.504+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:18:03.504+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:18:03.521+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 SnapshotId 2 Created recovery point (took 636.3355ms) 2023-10-21T04:18:03.522+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:18:03.523+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:18:03.561+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 SnapshotId 2 Created recovery point (took 675.766076ms) 2023-10-21T04:18:03.562+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:18:03.562+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:18:03.580+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 SnapshotId 2 Created recovery point (took 695.062272ms) 2023-10-21T04:18:03.581+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:18:03.582+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:18:03.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 SnapshotId 2 Created recovery point (took 720.711486ms) 2023-10-21T04:18:03.608+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:18:03.608+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:18:03.626+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 SnapshotId 2 Created recovery point (took 740.232857ms) 2023-10-21T04:18:03.627+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:18:03.627+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:18:03.642+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 SnapshotId 2 Created recovery point (took 757.011466ms) 2023-10-21T04:18:03.643+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:18:03.644+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:18:03.683+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 SnapshotId 2 Created recovery point (took 797.055768ms) 2023-10-21T04:18:03.685+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:18:03.685+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:18:03.698+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 SnapshotId 2 Created recovery point (took 812.8449ms) 2023-10-21T04:18:03.700+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:18:03.700+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:18:03.713+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 SnapshotId 2 Created recovery point (took 827.553128ms) 2023-10-21T04:18:03.714+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:18:03.715+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:18:03.734+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 SnapshotId 2 Created recovery point (took 847.991778ms) 2023-10-21T04:18:03.735+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:18:03.736+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:18:03.754+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 SnapshotId 2 Created recovery point (took 867.845387ms) 2023-10-21T04:18:03.755+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:18:03.755+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:18:03.773+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 SnapshotId 2 Created recovery point (took 887.170641ms) 2023-10-21T04:18:03.774+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:03.774+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:03.793+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 SnapshotId 2 Created recovery point (took 906.793472ms) 2023-10-21T04:18:03.794+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:18:03.794+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:18:03.818+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 SnapshotId 2 Created recovery point (took 932.246726ms) 2023-10-21T04:18:03.819+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:18:03.819+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:18:03.832+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 SnapshotId 2 Created recovery point (took 945.980098ms) 2023-10-21T04:18:03.833+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:18:03.834+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:18:03.851+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 SnapshotId 2 Created recovery point (took 964.669334ms) 2023-10-21T04:18:03.852+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:18:03.852+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:18:03.861+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 SnapshotId 2 Created recovery point (took 975.300448ms) 2023-10-21T04:18:03.862+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:18:03.863+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:18:03.870+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 SnapshotId 2 Created recovery point (took 983.700829ms) 2023-10-21T04:18:03.871+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:18:03.871+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:18:03.882+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 SnapshotId 2 Created recovery point (took 996.327456ms) 2023-10-21T04:18:03.883+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:18:03.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:18:04.671+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:18:04.677+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:04.677+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:04.680+05:30 [Info] switched currmeta from 523 -> 523 force true 2023-10-21T04:18:10.970+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 7392796230534034390 2023-10-21T04:18:10.970+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 184.386µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:11.011+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36760 2023-10-21T04:18:11.015+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36760 2023-10-21T04:18:11.021+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36760 2023-10-21T04:18:11.141+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 7392796230534034390, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:18:11.141+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 7392796230534034390, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:18:11.180+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 265587992619548631, indexDefn DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:18:11.180+05:30 [Info] Indexer::handleCreateIndex InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:18:11.181+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 265587992619548631 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:18:11.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:18:11.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:18:11.185+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:18:11.185+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:18:11.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:18:11.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:18:11.193+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:18:11.193+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:18:11.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 166 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex 2023-10-21T04:18:11.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:18:11.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:18:11.199+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex 2023-10-21T04:18:11.199+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex 2023-10-21T04:18:11.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex started 2023-10-21T04:18:11.199+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:18:11.199+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:18:11.205+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:11.205+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:11.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 185 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex 2023-10-21T04:18:11.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:18:11.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:18:11.210+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex 2023-10-21T04:18:11.212+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex 2023-10-21T04:18:11.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex started 2023-10-21T04:18:11.212+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 265587992619548631 partitionId 0 WriterThreads 4 2023-10-21T04:18:11.212+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 265587992619548631 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index Index: 265587992619548631 Partition: 0 2023-10-21T04:18:11.213+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:18:11.213+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:18:11.213+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 265587992619548631 2023-10-21T04:18:11.213+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:11.213+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:18:11.213+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:11.214+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:11.214+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 33.869003ms 2023-10-21T04:18:11.214+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 33.91181ms 2023-10-21T04:18:11.214+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 265587992619548631, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], indexDefn DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:18:11.233+05:30 [Info] switched currmeta from 523 -> 527 force true 2023-10-21T04:18:11.243+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:11.243+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:11.251+05:30 [Info] switched currmeta from 527 -> 527 force true 2023-10-21T04:18:11.261+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:11.261+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:11.264+05:30 [Info] switched currmeta from 527 -> 529 force true 2023-10-21T04:18:11.269+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [265587992619548631] &{0} 2023-10-21T04:18:11.269+05:30 [Info] Indexer::handleBuildIndex [265587992619548631] 2023-10-21T04:18:11.274+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:11.274+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:11.274+05:30 [Info] bulkUpdateRState: Index instance 265587992619548631 rstate moved to ACTIVE 2023-10-21T04:18:11.275+05:30 [Info] Indexer::handleBuildIndex Added Index: [265587992619548631] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:18:11.275+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:11.276+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:11.276+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:11.277+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:11.277+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:11.278+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:11.278+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79187 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:11.278+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:11.279+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:11.279+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:11.279+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79187 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:11.279+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:18:11.280+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:11.280+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:11.280+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:18:11.280+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:18:11.281+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:11.281+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:11.281+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:18:11.282+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:10 bucket_1:11] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:18:11.282+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 265587992619548631 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:18:11.282+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:11.282+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [ InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 79188 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:18:11.283+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:18:11.284+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:18:11.285+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:11.292+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 23.11524ms 2023-10-21T04:18:11.287+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:18:11.293+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [265587992619548631] 2023-10-21T04:18:11.293+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:18:11.293+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 23.379722ms 2023-10-21T04:18:11.306+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 7392796230534034390 2023-10-21T04:18:11.306+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 166.032224ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:11.318+05:30 [Info] switched currmeta from 529 -> 530 force true 2023-10-21T04:18:11.343+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 36.465645ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:11.346+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:52952" doAuth() ... 2023-10-21T04:18:11.346+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:52952" auth successful 2023-10-21T04:18:11.346+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:52952" +1 2023-10-21T04:18:11.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:11.378+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:11.383+05:30 [Info] switched currmeta from 530 -> 532 force true 2023-10-21T04:18:11.448+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:11.448+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:11.457+05:30 [Info] switched currmeta from 532 -> 532 force true 2023-10-21T04:18:11.777+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 263268055970459 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.779+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 176639880977228 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.780+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 35403626585217 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.782+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 215749515705641 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 159113185399719 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 261493429196201 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 170288557775498 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.817+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 206533515052028 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.823+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 236398693961229 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.825+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 3874519919278 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.831+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 264975169139761 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.832+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 140236896721046 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.834+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 200752780087280 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.834+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 207134287529256 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.838+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 192215088742243 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.839+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 116104661063478 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.839+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 92956357968546 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.839+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 120989102157727 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.839+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 271667630317209 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.843+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 236953356346174 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.859+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 51776324008509 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.877+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 60545336618717 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.878+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 98820809009123 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.878+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 208836052671833 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.878+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 154934968850942 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.878+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5715487545729 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.878+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 27979933591532 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 102350060311343 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 234758907181738 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 162158720181435 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 130231725986856 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 168740244208559 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 278426329668882 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 83647976044922 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 24368176313355 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 73432771436458 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 67362620867705 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 43594649874772 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 166675095997741 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 153478150002150 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 181267443431019 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 100659545510465 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 116785907755771 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 45479338375128 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 249919763471169 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 49796766757346 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 19274425379955 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 124194574848641 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 149022976193937 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 184487117398647 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 199728167534433 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 158076890855634 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 196777743741606 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 138935694816848 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 61798873988101 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 24856128127173 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 191719959781498 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 29071415952852 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 140942236111856 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 103336286678362 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 56626109228904 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.896+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 119871837460216 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.901+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 177105814270799 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.903+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c1 InstanceIds [265587992619548631] 2023-10-21T04:18:11.903+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 252537041863196 0 79188 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:11.903+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 79188 2023-10-21T04:18:11.903+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:11.903+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:11.903+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:18:11.903+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:18:11.903+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:11.903+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:11.904+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:11.971+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:18:11.972+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:18:11.993+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:18:12.011+05:30 [Info] memstats {"Alloc":189045280, "TotalAlloc":1532445216, "Sys":327116104, "Lookups":0, "Mallocs":15436033,"Frees":14547738, "HeapAlloc":189045280, "HeapSys":295632896, "HeapIdle":88932352, "HeapInuse":206700544,"HeapReleased":3317760, "HeapObjects":888295,"MSpanInuse":1578400, "MSpanSys": 2529600, "StackInuse": 10551296,"GCSys":15714560, "LastGC":1697842031055587731,"PauseTotalNs":45995530, "PauseNs":[2006664, 1548228, 7771738, 281263, 1902100, 309706, 483946, 253842, 197406, 14710244, 174928, 583310], "NumGC":16} 2023-10-21T04:18:12.014+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:18:12.048+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:18:12.048+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:18:12.048+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:18:12.054+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52216 2023-10-21T04:18:12.054+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:18:12.080+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 13.345638ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:12.080+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:18:12.080+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:18:12.080+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:18:12.080+05:30 [Info] DDLServiceMgr: processing create index token /indexing/ddl/commandToken/create/7392796230534034390/0 2023-10-21T04:18:12.089+05:30 [Info] DDLServiceMgr: Update Replica Count. Index Defn 7392796230534034390 replica Count {true 1 0 0} 2023-10-21T04:18:12.094+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 31.243µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:12.104+05:30 [Info] DDLServiceMgr: Remove create index token 7392796230534034390. 2023-10-21T04:18:12.104+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:18:12.104+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:18:12.104+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:18:12.105+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:52216->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:18:12.105+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:48588->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:18:12.105+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52216. Reason = EOF. Cleaning Pipe. 2023-10-21T04:18:12.105+05:30 [Info] messageListener.start(): message channel closed. Remove peer 68:34:8f:a2:96:aa:b6:b3 as follower. 2023-10-21T04:18:12.180+05:30 [Info] janitor: running cleanup. 2023-10-21T04:18:12.180+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[5465835793930010953 12195715103909237684 2048006213723315482 249887442666643345 14889401051926000176 4138047918192025493 4535197604682045916 4860549750398470299]} 2023-10-21T04:18:12.208+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52224 2023-10-21T04:18:12.309+05:30 [Info] connected with 2 indexers 2023-10-21T04:18:12.309+05:30 [Info] client stats current counts: current: 76, not current: 0 2023-10-21T04:18:12.311+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 83.055613ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:12.436+05:30 [Info] num concurrent scans {0} 2023-10-21T04:18:12.436+05:30 [Info] average scan response {0 ms} 2023-10-21T04:18:12.494+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52224. Reason = EOF. Cleaning Pipe. 2023-10-21T04:18:12.494+05:30 [Info] messageListener.start(): message channel closed. Remove peer 9c:14:6a:35:95:ad:17:b5 as follower. 2023-10-21T04:18:12.582+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:18:12.663+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:12.680+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 265587992619548631 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:12.680+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 265587992619548631 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.298µs SnapOpenDur 108.197µs 2023-10-21T04:18:12.680+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 265587992619548631 INDEX_STATE_CATCHUP 2023-10-21T04:18:12.680+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 265587992619548631 true 2023-10-21T04:18:12.680+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 265587992619548631 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 79188 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] 2023-10-21T04:18:12.680+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 265587992619548631, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:12.682+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 79188 2023-10-21T04:18:12.683+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 79188 2023-10-21T04:18:12.685+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:12.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:12.687+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:12.687+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:12.687+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:12.688+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:12.688+05:30 [Info] send progress stats to clients 2023-10-21T04:18:12.688+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:18:12.688+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:12.688+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:11] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:18:12.688+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 79188 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:12.695+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:12.695+05:30 [Info] switched currmeta from 532 -> 533 force true 2023-10-21T04:18:12.699+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:18:12.701+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.64437ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:12.705+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 2023-10-21T04:18:12.705+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79188 2023-10-21T04:18:12.706+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79188 2023-10-21T04:18:12.706+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:18:12.707+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:12.707+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 265587992619548631 false 2023-10-21T04:18:12.708+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 265587992619548631 2023-10-21T04:18:12.708+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:18:12.707+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:12.708+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 265587992619548631, PartitionId 0 SnapshotId 2 Created recovery point (took 27.891644ms) 2023-10-21T04:18:12.709+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 79188 2023-10-21T04:18:12.709+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:12.715+05:30 [Info] switched currmeta from 533 -> 534 force true 2023-10-21T04:18:12.748+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:18:12.813+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:18:12.904+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:12.904+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:12.907+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:12.908+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:12.904+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 265587992619548631 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 79188, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 52 0 52} { 1 a0a72d5bdb4c 2 61 0 61} { 2 20330c961c81 2 50 0 50} { 3 c439193e7d29 2 45 0 45} { 4 90b66ce513a7 2 58 0 58} { 5 3861b622aae 2 54 0 54} { 6 edd3aef555a9 2 61 0 61} { 7 9ae064feda8a 2 52 0 52} { 8 f0fe56d4b431 2 56 0 56} { 9 d700dc2dce0d 2 62 0 62} { 10 bbd754cc83fc 2 50 0 50} { 11 d58311453aaa 2 56 0 56} { 12 6998b9145736 2 55 0 55} { 13 5d163afcd32f 2 53 0 53} { 14 7f8b7269d896 2 60 0 60} { 15 937b84e9e4bb 2 67 0 67} { 16 b69566250ff0 2 54 0 54} { 17 59e08337a7e3 2 45 0 45} { 18 bc6335a10528 2 46 0 46} { 19 bdef6ebc5d59 2 45 0 45} { 20 548b16b242a2 2 50 0 50} { 21 19729600a7ec 2 54 0 54} { 22 aed19015bf63 2 55 0 55} { 23 8ce99bc819fe 2 51 0 51} { 24 f7148c9a1e99 2 52 0 52} { 25 9977e650bbaf 2 61 0 61} { 26 6e09f874af9f 2 59 0 59} { 27 532bd9ad981 2 54 0 54} { 28 d78200a2233e 2 54 0 54} { 29 7671efc41428 2 49 0 49} { 30 3710cf482edd 2 46 0 46} { 31 42c966419faa 2 60 0 60} { 32 3d4415168479 2 67 0 67} { 33 fd3a2e79dd12 2 54 0 54} { 34 2f171d667a3d 2 60 0 60} { 35 1629a87e340b 2 50 0 50} { 36 27a62b9eb154 2 52 0 52} { 37 4c13cfe5f97a 2 59 0 59} { 38 a4dc9d93366b 2 53 0 53} { 39 5b8ca0741241 2 49 0 49} { 40 8b966a9c95e6 2 68 0 68} { 41 6a37568beafb 2 55 0 55} { 42 295cfbcef7d8 2 55 0 55} { 43 e34cfab95b41 2 66 0 66} { 44 979711e3cd2d 2 57 0 57} { 45 2d4a3697f1e2 2 61 0 61} { 46 70f44d845281 2 54 0 54} { 47 87891d8f3591 2 55 0 55} { 48 1187ad472c73 2 55 0 55} { 49 a7ca41192277 2 55 0 55} { 50 b5a6d67a3f61 2 66 0 66} { 51 b2f7e383f2a6 2 51 0 51} { 52 8fc524ed38d2 2 54 0 54} { 53 7e5c7cbc7650 2 61 0 61} { 54 169b44b000c5 2 60 0 60} { 55 3834abee7c05 2 67 0 67} { 56 1a70b76bc5d4 2 61 0 61} { 57 ae5e4819607a 2 63 0 63} { 58 5dfbdaa8ed5a 2 46 0 46} { 59 802fabead3f0 2 63 0 63} { 60 33804b202768 2 46 0 46} { 61 6d05d64a56f8 2 39 0 39} { 62 a113a925b34f 2 58 0 58} { 63 e5ae5cac421c 2 73 0 73} 2023-10-21T04:18:12.918+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 265587992619548631 INDEX_STATE_ACTIVE 2023-10-21T04:18:12.918+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:18:12.918+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:12.918+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:12.918+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 1 New 0 2023-10-21T04:18:12.918+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 79188 2023-10-21T04:18:12.918+05:30 [Info] switched currmeta from 534 -> 535 force true 2023-10-21T04:18:12.919+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:12.919+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:12.919+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:18:12.919+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79188 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:12.919+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 265587992619548631 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:18:12.919+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:12.919+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:12.919+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:12.920+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79188 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:12.920+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:12.920+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:17 1:18 2:15 3:14 4:21 5:14 6:21 7:13 8:16 9:14 10:14 11:13 12:12 13:14 14:10 15:20 16:18 17:14 18:12 19:13 20:12 21:16 22:13 23:17 24:16 25:10 26:15 27:17 28:20 29:16 30:11 31:17 32:20 33:12 34:18 35:20 36:11 37:12 38:12 39:9 40:17 41:15 42:19 43:14 44:15 45:15 46:16 47:17 48:15 49:14 50:18 51:15 52:17 53:24 54:16 55:18 56:14 57:16 58:11 59:19 60:17 61:14 62:19 63:28] 2023-10-21T04:18:12.920+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:37 1:38 2:35 3:33 4:41 5:34 6:47 7:33 8:42 9:41 10:34 11:37 12:35 13:37 14:38 15:47 16:39 17:27 18:34 19:32 20:33 21:36 22:37 23:40 24:33 25:33 26:38 27:47 28:39 29:36 30:32 31:40 32:47 33:38 34:45 35:39 36:33 37:33 38:39 39:29 40:45 41:41 42:46 43:41 44:34 45:34 46:42 47:38 48:38 49:42 50:44 51:36 52:37 53:50 54:41 55:47 56:36 57:41 58:32 59:46 60:37 61:30 62:43 63:50] 2023-10-21T04:18:12.920+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:12.920+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:52952: use of closed network connection 2023-10-21T04:18:12.921+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:52952" exit: read tcp4 127.0.0.1:9121->127.0.0.1:52952: use of closed network connection 2023-10-21T04:18:12.920+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:52952" closed ! 2023-10-21T04:18:12.921+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:12.922+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:52952" already gone 2023-10-21T04:18:12.922+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:12.922+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:12.922+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:12.922+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:12.922+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:12.922+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:12.923+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 79188 2023-10-21T04:18:12.923+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:18:12.943+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.766649ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:12.982+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:18:12.982+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:12.982+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:18:12.982+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79188 2023-10-21T04:18:12.982+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79188 2023-10-21T04:18:12.982+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:12.988+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:12.988+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:12.994+05:30 [Info] switched currmeta from 535 -> 536 force true 2023-10-21T04:18:13.858+05:30 [Info] Indexer::ReadMemstats Time Taken 115.28µs 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.005+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3519182887891186211:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.291+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.292+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[265587992619548631:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:14.717+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:18:14.723+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:14.723+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:14.726+05:30 [Info] switched currmeta from 536 -> 536 force true 2023-10-21T04:18:21.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:21.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:21.756+05:30 [Info] switched currmeta from 536 -> 537 force true 2023-10-21T04:18:21.769+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [14343533858105260407] &{0} 2023-10-21T04:18:21.770+05:30 [Info] Indexer::handleBuildIndex [14343533858105260407] 2023-10-21T04:18:21.770+05:30 [Info] bulkUpdateRState: Index instance 14343533858105260407 rstate moved to ACTIVE 2023-10-21T04:18:21.770+05:30 [Info] Indexer::handleBuildIndex Added Index: [14343533858105260407] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:18:21.771+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:21.771+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:21.771+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:21.771+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_1. OSO not supported for default scope/collection. 2023-10-21T04:18:21.772+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1 2023-10-21T04:18:21.772+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:21.772+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:21.772+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 42562 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:21.772+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:18:21.772+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:21.772+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:21.772+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 42562 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:21.772+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:18:21.772+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:21.772+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:21.772+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:18:21.772+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:18:21.772+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:21.772+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:21.772+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream INIT_STREAM 2023-10-21T04:18:21.772+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:11] INIT_STREAM:map[bucket_1:1]] 2023-10-21T04:18:21.772+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 14343533858105260407 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:18:21.772+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1 2023-10-21T04:18:21.772+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [0=17,1=19,2=20,3=13,4=18,5=28,6=27,7=27,8=24,9=19,10=24,11=21,12=19,13=15,14=26,15=23,16=24,17=21,18=26,19=20,20=19,21=23,22=17,23=17,24=24,25=26,26=17,27=18,28=18,29=19,30=29,31=24,32=23,33=21,34=23,35=21,36=22,37=25,38=21,39=20,40=20,41=20,42=22,43=20,44=21,45=24,46=23,47=22,48=23,49=13,50=24,51=19,52=22,53=24,54=21,55=23,56=28,57=24,58=18,59=27,60=24,61=20,62=21,63=23] IndexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 42563 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:21.773+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:18:21.773+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:18:21.774+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:21.774+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 4.282345ms 2023-10-21T04:18:21.774+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 4.32384ms 2023-10-21T04:18:21.774+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [14343533858105260407] 2023-10-21T04:18:21.774+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_INDEX elapsed 17.241691ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:21.778+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:18:21.780+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:21.780+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:21.782+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1 Check pending stream merge. 2023-10-21T04:18:21.795+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.701337ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:21.798+05:30 [Info] switched currmeta from 537 -> 539 force true 2023-10-21T04:18:21.808+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:21.808+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:21.815+05:30 [Info] switched currmeta from 539 -> 540 force true 2023-10-21T04:18:21.819+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:21.819+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:21.822+05:30 [Info] switched currmeta from 540 -> 540 force true 2023-10-21T04:18:21.832+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53200" doAuth() ... 2023-10-21T04:18:21.832+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53200" auth successful 2023-10-21T04:18:21.832+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:53200" +1 2023-10-21T04:18:22.126+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 0 186517626268899 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 2 148075163102199 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 1 178840478062605 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.138+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 3 215374859564337 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.139+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 4 28605411869539 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.142+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 7 191903564840858 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.142+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 5 39552632604610 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.142+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 6 242327631330054 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.145+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 9 225280996447309 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.145+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 8 50961953886801 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.146+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 10 274410200699189 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.149+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 11 66881023717063 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.153+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 12 4429827396609 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.160+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 13 183148377680315 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.163+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 15 127185710837163 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.168+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 17 55290066231400 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.170+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 16 257031483891250 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.172+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 19 210306088158507 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.176+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 25 280874278672777 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.176+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 23 216462437971699 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.177+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 20 230239359684242 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.177+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 21 209771427122473 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.177+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 24 160961818008256 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.180+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 28 102578907139247 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.180+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 27 184056281482149 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.181+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 29 45957426777107 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.185+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 32 82469662608686 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.186+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 33 136958029391999 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.192+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 31 217381278987664 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.192+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 35 70213628416850 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.194+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 14 226814108379280 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 37 253494845579900 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 18 242578104468179 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 36 253845164956592 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 34 28417669987133 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 26 219132834127088 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 30 99433130332778 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 22 260036488774725 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.196+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 38 188107826286720 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.199+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 39 222555662236724 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.201+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 40 5915864305121 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.202+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 42 252931432434966 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.202+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 41 259381390638966 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.206+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 43 146267938201125 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.212+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 44 158545118983293 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.212+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 45 12301831552869 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.218+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 49 215467668600630 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.218+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 47 271074273345339 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.218+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 48 227378777358533 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.218+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 46 242994859770444 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.221+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 50 207367350302991 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.223+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 51 200436537417141 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.227+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 53 204867593700407 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.227+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 54 279362078165491 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.231+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 55 170056388266420 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.234+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 57 129411666769864 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.234+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 56 264835102027079 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.234+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 52 188718367400378 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.235+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 59 125009072873642 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.238+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 60 75566139929862 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.238+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 58 16044440367455 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.241+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 61 106932309485550 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.244+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 63 133748675726909 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 62 10527297481171 0 42563 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:22.246+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1 InstanceIds [14343533858105260407] 2023-10-21T04:18:22.246+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1 Cid 0 SessionId 42563 2023-10-21T04:18:22.246+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:18:22.247+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:18:22.247+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:18:22.247+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 62 Crc64: 2113421542146274774 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 0 55 0 55} { 1 a2a78b2a980d 0 58 0 58} { 2 86ac6f7f2bf7 0 9 0 54} { 3 c3e1ddff7531 0 35 0 35} { 4 1a043769c363 0 18 0 58} { 5 23f9109dc7c2 0 27 0 52} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 0 19 0 63} { 8 2e5981289251 0 0 0 0} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} { 61 61411e5267ee 0 0 0 0} HWT bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 0 55 0 55} { 1 a2a78b2a980d 0 58 0 58} { 2 86ac6f7f2bf7 0 16 0 54} { 3 c3e1ddff7531 0 35 0 35} { 4 1a043769c363 0 18 0 58} { 5 23f9109dc7c2 0 27 0 52} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 0 19 0 63} { 8 2e5981289251 0 0 0 0} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} { 61 61411e5267ee 0 0 0 0} { 62 99313d3a9d3 0 0 0 0} { 63 79a4ca4e4e3d 0 0 0 0} 2023-10-21T04:18:22.248+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1 2023-10-21T04:18:22.248+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1 2023-10-21T04:18:22.248+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:22.249+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:22.249+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [0=17,1=19,2=20,3=13,4=18,5=28,6=27,7=27,8=24,9=19,10=24,11=21,12=19,13=15,14=26,15=23,16=24,17=21,18=26,19=20,20=19,21=23,22=17,23=17,24=24,25=26,26=17,27=18,28=18,29=19,30=29,31=24,32=23,33=21,34=23,35=21,36=22,37=25,38=21,39=20,40=20,41=20,42=22,43=20,44=21,45=24,46=23,47=22,48=23,49=13,50=24,51=19,52=22,53=24,54=21,55=23,56=28,57=24,58=18,59=27,60=24,61=20,62=21,63=23] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:22.765+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:22.771+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14343533858105260407 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:22.771+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14343533858105260407 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.859073ms SnapOpenDur 74.782µs 2023-10-21T04:18:22.771+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14343533858105260407, PartitionId 0 SnapshotId 1 Creating recovery point ... 2023-10-21T04:18:22.771+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 14343533858105260407 INDEX_STATE_CATCHUP 2023-10-21T04:18:22.773+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 14343533858105260407 true 2023-10-21T04:18:22.773+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 14343533858105260407 Stream: INIT_STREAM KeyspaceId: bucket_1 Session: 42563 BuildTS: [0=17,1=19,2=20,3=13,4=18,5=28,6=27,7=27,8=24,9=19,10=24,11=21,12=19,13=15,14=26,15=23,16=24,17=21,18=26,19=20,20=19,21=23,22=17,23=17,24=24,25=26,26=17,27=18,28=18,29=19,30=29,31=24,32=23,33=21,34=23,35=21,36=22,37=25,38=21,39=20,40=20,41=20,42=22,43=20,44=21,45=24,46=23,47=22,48=23,49=13,50=24,51=19,52=22,53=24,54=21,55=23,56=28,57=24,58=18,59=27,60=24,61=20,62=21,63=23] 2023-10-21T04:18:22.773+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1 Stream: INIT_STREAM SessionId: 42563 2023-10-21T04:18:22.773+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1 42563 2023-10-21T04:18:22.774+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:22.774+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:22.774+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:22.775+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:22.775+05:30 [Info] send progress stats to clients 2023-10-21T04:18:22.775+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1 set minMergeTs 2023-10-21T04:18:22.775+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:22.775+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:12] INIT_STREAM:map[bucket_1:1]] 2023-10-21T04:18:22.775+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 42563 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:22.778+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:22.789+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 2023-10-21T04:18:22.789+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1 SessionId 42563 2023-10-21T04:18:22.789+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 42563 2023-10-21T04:18:22.789+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1 set minMergeTs 2023-10-21T04:18:22.789+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:18:22.789+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 14343533858105260407 false 2023-10-21T04:18:22.789+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1 14343533858105260407 2023-10-21T04:18:22.789+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1. 2023-10-21T04:18:22.789+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 42563 2023-10-21T04:18:22.793+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:22.793+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:22.795+05:30 [Info] switched currmeta from 540 -> 541 force true 2023-10-21T04:18:22.807+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 31.27886ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:22.826+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:22.826+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:22.829+05:30 [Info] switched currmeta from 541 -> 542 force true 2023-10-21T04:18:22.853+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14343533858105260407, PartitionId 0 SnapshotId 1 Created recovery point (took 81.689197ms) 2023-10-21T04:18:23.249+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:23.249+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1 2023-10-21T04:18:23.249+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 14343533858105260407 Stream: INIT_STREAM KeyspaceId: bucket_1 SessionId: 42563, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 0 55 0 55} { 1 a2a78b2a980d 0 58 0 58} { 2 86ac6f7f2bf7 0 54 0 54} { 3 c3e1ddff7531 0 35 0 35} { 4 1a043769c363 0 58 0 58} { 5 23f9109dc7c2 0 52 0 52} { 6 dc654c627b06 0 62 0 62} { 7 ae8907d03f9a 0 63 0 63} { 8 2e5981289251 0 45 0 45} { 9 cce451c4944d 0 55 0 55} { 10 f9931a891135 0 55 0 55} { 11 3cd3f3a92ec7 0 51 0 51} { 12 4076649d401 0 42 0 42} { 13 a6928dfe29bb 0 46 0 46} { 14 ce49465c2890 0 72 0 72} { 15 73acbb1d6dab 0 50 0 50} { 16 e9c4ce4b1e32 0 59 0 59} { 17 324938c1e068 0 51 0 51} { 18 dc9f9dbf42d3 0 63 0 63} { 19 bf45b3adc52b 0 48 0 48} { 20 d166c73f4e92 0 50 0 50} { 21 bec93765cd29 0 48 0 48} { 22 ec80768ac445 0 51 0 51} { 23 c4df16b8f6f3 0 57 0 57} { 24 9264d7fc1ac0 0 59 0 59} { 25 ff74239b5989 0 71 0 71} { 26 c74cd6bc9cf0 0 52 0 52} { 27 a765f146c3a5 0 49 0 49} { 28 5d4b83524caf 0 58 0 58} { 29 29cc4c190c13 0 57 0 57} { 30 5a6f1469926a 0 58 0 58} { 31 c5b505ea3590 0 67 0 67} { 32 4b0176f1cd2e 0 60 0 60} { 33 7c9006b0e87f 0 65 0 65} { 34 19d8811ffb3d 0 58 0 58} { 35 3fdbe2615352 0 47 0 47} { 36 e6deeee29bb0 0 58 0 58} { 37 e68d5e39227c 0 54 0 54} { 38 ab15442a5080 0 60 0 60} { 39 ca69c730bc34 0 67 0 67} { 40 56164fd8de1 0 60 0 60} { 41 ebe7efa67f76 0 58 0 58} { 42 e60a302ecd16 0 46 0 46} { 43 8507a87f9e25 0 55 0 55} { 44 9032297f747d 0 67 0 67} { 45 b303e4cbf65 0 67 0 67} { 46 dd00a64c824c 0 54 0 54} { 47 f68a65c53f3b 0 53 0 53} { 48 ceccbf4100c5 0 59 0 59} { 49 c3f779d8f736 0 34 0 34} { 50 bc997940b90f 0 64 0 64} { 51 b64bc49cddb5 0 54 0 54} { 52 aba36b4061ba 0 56 0 56} { 53 ba5374260437 0 56 0 56} { 54 fe140d6ec9f3 0 55 0 55} { 55 9aaa569d49b4 0 70 0 70} { 56 f0ddba2e2d47 0 54 0 54} { 57 75b3006d77c8 0 55 0 55} { 58 e97a32b315f 0 53 0 53} { 59 71b1f161e0aa 0 53 0 53} { 60 44ba1ced7506 0 56 0 56} { 61 61411e5267ee 0 53 0 53} { 62 99313d3a9d3 0 57 0 57} { 63 79a4ca4e4e3d 0 56 0 56} 2023-10-21T04:18:23.250+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 14343533858105260407 INDEX_STATE_ACTIVE 2023-10-21T04:18:23.250+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1 State Changed to INACTIVE 2023-10-21T04:18:23.250+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:18:23.250+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:23.250+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:23.250+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1 Stream: INIT_STREAM SessionId: 42563 2023-10-21T04:18:23.250+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:23.251+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:23.251+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:23.252+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 42563 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:23.253+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 14343533858105260407 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:18:23.253+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:18:23.253+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:23.253+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:23.253+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 42563 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:23.253+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:23.253+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 total received key-versions: map[0:11 1:13 2:14 3:7 4:12 5:22 6:21 7:21 8:18 9:13 10:18 11:15 12:13 13:9 14:20 15:17 16:18 17:15 18:20 19:14 20:13 21:17 22:11 23:11 24:18 25:20 26:11 27:12 28:12 29:13 30:23 31:18 32:17 33:15 34:17 35:15 36:16 37:19 38:15 39:14 40:14 41:14 42:16 43:14 44:15 45:18 46:17 47:16 48:17 49:7 50:18 51:13 52:16 53:18 54:15 55:17 56:22 57:18 58:12 59:21 60:18 61:14 62:15 63:17] 2023-10-21T04:18:23.253+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 latest sequence numbers: map[0:17 1:19 2:20 3:13 4:18 5:28 6:27 7:27 8:24 9:19 10:24 11:21 12:19 13:15 14:26 15:23 16:24 17:21 18:26 19:20 20:19 21:23 22:17 23:17 24:24 25:26 26:17 27:18 28:18 29:19 30:29 31:24 32:23 33:21 34:23 35:21 36:22 37:25 38:21 39:20 40:20 41:20 42:22 43:20 44:21 45:24 46:23 47:22 48:23 49:13 50:24 51:19 52:22 53:24 54:21 55:23 56:28 57:24 58:18 59:27 60:24 61:20 62:21 63:23] 2023-10-21T04:18:23.253+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:23.253+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:53200: use of closed network connection 2023-10-21T04:18:23.254+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:53200" exit: read tcp4 127.0.0.1:9121->127.0.0.1:53200: use of closed network connection 2023-10-21T04:18:23.254+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53200" closed ! 2023-10-21T04:18:23.254+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:23.254+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:23.254+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:23.254+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:23.254+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:23.254+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:23.254+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:23.255+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1 Stream: INIT_STREAM SessionId 42563 2023-10-21T04:18:23.256+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1] 2023-10-21T04:18:23.273+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1]. Err genServer.closed 2023-10-21T04:18:23.273+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:23.273+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Treating genServer.closed As Success 2023-10-21T04:18:23.273+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1 SessionId 42563 2023-10-21T04:18:23.273+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 42563 2023-10-21T04:18:23.273+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:18:23.276+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.548629ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:23.300+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:23.301+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:23.304+05:30 [Info] switched currmeta from 542 -> 543 force true 2023-10-21T04:18:23.413+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:23.413+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:23.422+05:30 [Info] switched currmeta from 543 -> 544 force true 2023-10-21T04:18:23.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14343533858105260407:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:24.271+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:18:24.271+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:18:24.272+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:18:24.272+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:18:24.272+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:18:24.272+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:18:24.273+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11200171968618803270:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:24.793+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:18:24.797+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:24.798+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:24.800+05:30 [Info] switched currmeta from 544 -> 544 force true 2023-10-21T04:18:26.156+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:26.156+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:26.160+05:30 [Info] switched currmeta from 544 -> 545 force true 2023-10-21T04:18:26.185+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [9112972261750078043] &{0} 2023-10-21T04:18:26.185+05:30 [Info] Indexer::handleBuildIndex [9112972261750078043] 2023-10-21T04:18:26.186+05:30 [Info] bulkUpdateRState: Index instance 9112972261750078043 rstate moved to ACTIVE 2023-10-21T04:18:26.186+05:30 [Info] Indexer::handleBuildIndex Added Index: [9112972261750078043] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:18:26.186+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:26.187+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:26.187+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:26.187+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:18:26.187+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:26.187+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:26.187+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65896 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:26.187+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:18:26.187+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:26.187+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:26.187+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65896 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:26.188+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:18:26.188+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:26.188+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:26.188+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:18:26.188+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:18:26.188+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:26.188+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:26.188+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:18:26.188+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:12] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:18:26.188+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 9112972261750078043 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:18:26.188+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:18:26.188+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 65897 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:18:26.198+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:18:26.199+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:18:26.199+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:18:26.201+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:26.203+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 17.74869ms 2023-10-21T04:18:26.203+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 17.793955ms 2023-10-21T04:18:26.203+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [9112972261750078043] 2023-10-21T04:18:26.203+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_INDEX elapsed 32.704622ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:26.208+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 4.753121ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:26.211+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c1 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:18:26.223+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:26.223+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:26.228+05:30 [Info] switched currmeta from 545 -> 548 force true 2023-10-21T04:18:26.237+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:26.237+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:26.241+05:30 [Info] switched currmeta from 548 -> 548 force true 2023-10-21T04:18:26.255+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53304" doAuth() ... 2023-10-21T04:18:26.255+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53304" auth successful 2023-10-21T04:18:26.255+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:53304" +1 2023-10-21T04:18:26.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 186517626268899 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 178840478062605 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.596+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 28605411869539 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.596+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 148075163102199 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 215374859564337 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 242327631330054 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 39552632604610 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.602+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 191903564840858 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.602+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 50961953886801 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.604+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 274410200699189 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 225280996447309 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 226814108379280 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 66881023717063 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 4429827396609 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 183148377680315 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 127185710837163 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.615+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 55290066231400 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.615+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 257031483891250 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.615+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 242578104468179 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.616+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 210306088158507 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.621+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 230239359684242 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.626+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 160961818008256 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.626+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 260036488774725 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 219132834127088 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 209771427122473 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 280874278672777 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 45957426777107 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 99433130332778 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 217381278987664 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 102578907139247 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 216462437971699 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 82469662608686 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.644+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 28417669987133 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.644+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 184056281482149 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.646+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 253845164956592 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.652+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 70213628416850 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.652+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 136958029391999 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.655+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 188107826286720 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.656+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 253494845579900 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.656+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 5915864305121 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.661+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 146267938201125 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.661+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 222555662236724 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.661+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 259381390638966 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.666+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 12301831552869 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.666+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 158545118983293 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.666+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 242994859770444 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.666+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 252931432434966 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.671+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 271074273345339 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.671+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 227378777358533 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.673+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 207367350302991 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.673+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 215467668600630 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.676+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 200436537417141 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 279362078165491 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 188718367400378 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.683+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 264835102027079 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.685+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 170056388266420 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.686+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 204867593700407 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 129411666769864 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 16044440367455 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.691+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 125009072873642 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.692+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 75566139929862 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.692+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 106932309485550 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 133748675726909 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.698+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c1 InstanceIds [9112972261750078043] 2023-10-21T04:18:26.698+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 65897 2023-10-21T04:18:26.698+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:26.698+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:26.699+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_1:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:18:26.699+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 59 Crc64: 3108598613925939753 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 18 0 55} { 1 a2a78b2a980d 0 0 0 58} { 2 86ac6f7f2bf7 2 21 0 54} { 3 c3e1ddff7531 2 14 0 35} { 4 1a043769c363 0 0 0 0} { 5 23f9109dc7c2 0 0 0 0} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 0 0 0 0} { 8 2e5981289251 0 0 0 0} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} HWT bucket: bucket_1, scopeId: , collectionId: , vbuckets: 63 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 22 0 55} { 1 a2a78b2a980d 2 20 0 58} { 2 86ac6f7f2bf7 2 23 0 54} { 3 c3e1ddff7531 2 21 0 35} { 4 1a043769c363 0 0 0 0} { 5 23f9109dc7c2 0 0 0 0} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 0 0 0 0} { 8 2e5981289251 0 0 0 0} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} { 61 61411e5267ee 0 0 0 0} { 63 79a4ca4e4e3d 0 0 0 0} 2023-10-21T04:18:26.699+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:18:26.699+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:18:26.699+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:26.702+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:26.702+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 10527297481171 0 65897 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:26.988+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 4 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 26.247µs SnapOpenDur 86.231µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 5 Creating New Snapshot SnapshotInfo: count:117 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:117 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.154µs SnapOpenDur 37.628µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 6 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:137 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.901µs SnapOpenDur 36.34µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 7 Creating New Snapshot SnapshotInfo: count:102 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:102 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.137µs SnapOpenDur 44.143µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 8 Creating New Snapshot SnapshotInfo: count:148 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:148 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.371µs SnapOpenDur 40.049µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 1 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.906µs SnapOpenDur 39.917µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.397µs SnapOpenDur 39.497µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.862µs SnapOpenDur 37.934µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Creating New Snapshot SnapshotInfo: count:21 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4992762551574016313 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:21 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.615µs SnapOpenDur 58.354µs 2023-10-21T04:18:26.989+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13739166377675070800 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:26.989+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13739166377675070800 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.458µs SnapOpenDur 52.446µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9720186692804226822 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9720186692804226822 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.029µs SnapOpenDur 63.84µs 2023-10-21T04:18:26.990+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.990+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 265587992619548631, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1265729768899070816 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1265729768899070816 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.356µs SnapOpenDur 56.593µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 4 Creating New Snapshot SnapshotInfo: count:130 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:130 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.402µs SnapOpenDur 53.831µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 5 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.591µs SnapOpenDur 38.462µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 6 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:106 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.339µs SnapOpenDur 39.113µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 7 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.654µs SnapOpenDur 36.998µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 8 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.721µs SnapOpenDur 38.95µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 1 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:120 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.268µs SnapOpenDur 44.833µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 2 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.476µs SnapOpenDur 37.695µs 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:18:26.991+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.5µs SnapOpenDur 38.982µs 2023-10-21T04:18:26.990+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.405µs SnapOpenDur 67.639µs 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2296526085582506469 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:26.991+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2296526085582506469 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.099µs SnapOpenDur 303.797µs 2023-10-21T04:18:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.991+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11050396066566371125 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:26.991+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 8 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:18:26.992+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:124 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.794µs SnapOpenDur 904.573µs 2023-10-21T04:18:26.992+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 1 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:18:26.992+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.455µs SnapOpenDur 41.364µs 2023-10-21T04:18:26.992+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 2 Creating New Snapshot SnapshotInfo: count:110 committed:true 2023-10-21T04:18:26.992+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:110 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.266µs SnapOpenDur 41.573µs 2023-10-21T04:18:26.992+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 3 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:18:26.992+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11050396066566371125 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.871µs SnapOpenDur 309.235µs 2023-10-21T04:18:26.992+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.804µs SnapOpenDur 46.295µs 2023-10-21T04:18:26.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.993+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.992+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 4 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:18:26.993+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:140 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.218µs SnapOpenDur 1.290092ms 2023-10-21T04:18:26.993+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 5 Creating New Snapshot SnapshotInfo: count:116 committed:true 2023-10-21T04:18:26.993+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:116 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.068µs SnapOpenDur 320.378µs 2023-10-21T04:18:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.015µs SnapOpenDur 43.515µs 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 5 Creating recovery point ... 2023-10-21T04:18:26.990+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 265587992619548631 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 265587992619548631 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.589µs SnapOpenDur 5.080144ms 2023-10-21T04:18:27.046+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 265587992619548631, PartitionId 0 SnapshotId 3 Created recovery point (took 56.109498ms) 2023-10-21T04:18:27.059+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 SnapshotId 3 Created recovery point (took 68.674417ms) 2023-10-21T04:18:27.063+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:27.159+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:27.163+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9112972261750078043 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:27.163+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9112972261750078043 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.166µs SnapOpenDur 83.071µs 2023-10-21T04:18:27.163+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9112972261750078043, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:18:27.163+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 9112972261750078043 INDEX_STATE_CATCHUP 2023-10-21T04:18:27.163+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 9112972261750078043 true 2023-10-21T04:18:27.163+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 9112972261750078043 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 65897 BuildTS: [0=36,1=34,2=36,3=21,4=38,5=38,6=46,7=46,8=34,9=31,10=42,11=34,12=34,13=31,14=48,15=36,16=35,17=34,18=45,19=29,20=35,21=32,22=32,23=34,24=36,25=50,26=29,27=35,28=36,29=38,30=44,31=44,32=41,33=43,34=41,35=37,36=37,37=41,38=38,39=46,40=43,41=38,42=34,43=34,44=55,45=41,46=42,47=37,48=42,49=28,50=47,51=43,52=40,53=38,54=45,55=46,56=39,57=38,58=33,59=41,60=41,61=36,62=41,63=40] 2023-10-21T04:18:27.163+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 65897 2023-10-21T04:18:27.163+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 65897 2023-10-21T04:18:27.164+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:27.164+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:27.165+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:27.165+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:27.165+05:30 [Info] send progress stats to clients 2023-10-21T04:18:27.165+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:18:27.165+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:27.166+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:13] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:18:27.166+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 65897 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:27.171+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:27.179+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 2023-10-21T04:18:27.179+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65897 2023-10-21T04:18:27.179+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65897 2023-10-21T04:18:27.179+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:18:27.179+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:27.179+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 9112972261750078043 false 2023-10-21T04:18:27.179+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 9112972261750078043 2023-10-21T04:18:27.179+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:18:27.179+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 65897 2023-10-21T04:18:27.184+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.801248ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:27.219+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:27.219+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:27.226+05:30 [Info] switched currmeta from 548 -> 550 force true 2023-10-21T04:18:27.233+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:27.233+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:27.239+05:30 [Info] switched currmeta from 550 -> 550 force true 2023-10-21T04:18:27.254+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 SnapshotId 3 Created recovery point (took 261.740981ms) 2023-10-21T04:18:27.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:27.410+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 SnapshotId 3 Created recovery point (took 415.940816ms) 2023-10-21T04:18:27.412+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:27.432+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 5 Created recovery point (took 437.669833ms) 2023-10-21T04:18:27.434+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:18:27.458+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 SnapshotId 3 Created recovery point (took 468.473516ms) 2023-10-21T04:18:27.459+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:18:27.460+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:18:27.478+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 SnapshotId 3 Created recovery point (took 487.043751ms) 2023-10-21T04:18:27.479+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:18:27.479+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:18:27.491+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 SnapshotId 3 Created recovery point (took 500.216708ms) 2023-10-21T04:18:27.492+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:18:27.492+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:18:27.506+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 SnapshotId 3 Created recovery point (took 515.431406ms) 2023-10-21T04:18:27.507+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:18:27.508+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:18:27.517+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 SnapshotId 3 Created recovery point (took 525.969046ms) 2023-10-21T04:18:27.518+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:18:27.518+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:18:27.527+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 SnapshotId 3 Created recovery point (took 536.249596ms) 2023-10-21T04:18:27.529+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:18:27.529+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:18:27.538+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 SnapshotId 3 Created recovery point (took 546.575736ms) 2023-10-21T04:18:27.539+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:18:27.540+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:18:27.549+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 SnapshotId 3 Created recovery point (took 557.544502ms) 2023-10-21T04:18:27.551+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:18:27.551+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:18:27.573+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 SnapshotId 3 Created recovery point (took 581.978433ms) 2023-10-21T04:18:27.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:18:27.581+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:18:27.585+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 SnapshotId 3 Created recovery point (took 593.701182ms) 2023-10-21T04:18:27.586+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:18:27.587+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:18:27.609+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 SnapshotId 3 Created recovery point (took 616.670009ms) 2023-10-21T04:18:27.610+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:18:27.610+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:18:27.631+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 SnapshotId 3 Created recovery point (took 639.181645ms) 2023-10-21T04:18:27.632+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:18:27.633+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:18:27.647+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 SnapshotId 3 Created recovery point (took 655.299837ms) 2023-10-21T04:18:27.649+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:18:27.649+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:18:27.663+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 SnapshotId 3 Created recovery point (took 671.288153ms) 2023-10-21T04:18:27.664+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:18:27.665+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:18:27.678+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 SnapshotId 3 Created recovery point (took 684.888662ms) 2023-10-21T04:18:27.679+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:27.679+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:27.701+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:27.701+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:27.701+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 9112972261750078043 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 65897, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 2 55 0 55} { 1 a2a78b2a980d 2 58 0 58} { 2 86ac6f7f2bf7 2 54 0 54} { 3 c3e1ddff7531 2 35 0 35} { 4 1a043769c363 2 58 0 58} { 5 23f9109dc7c2 2 52 0 52} { 6 dc654c627b06 2 62 0 62} { 7 ae8907d03f9a 2 63 0 63} { 8 2e5981289251 2 45 0 45} { 9 cce451c4944d 2 55 0 55} { 10 f9931a891135 2 55 0 55} { 11 3cd3f3a92ec7 2 51 0 51} { 12 4076649d401 2 42 0 42} { 13 a6928dfe29bb 2 46 0 46} { 14 ce49465c2890 2 72 0 72} { 15 73acbb1d6dab 2 50 0 50} { 16 e9c4ce4b1e32 2 59 0 59} { 17 324938c1e068 2 51 0 51} { 18 dc9f9dbf42d3 2 63 0 63} { 19 bf45b3adc52b 2 48 0 48} { 20 d166c73f4e92 2 50 0 50} { 21 bec93765cd29 2 48 0 48} { 22 ec80768ac445 2 51 0 51} { 23 c4df16b8f6f3 2 58 0 58} { 24 9264d7fc1ac0 2 59 0 59} { 25 ff74239b5989 2 71 0 71} { 26 c74cd6bc9cf0 2 52 0 52} { 27 a765f146c3a5 2 50 0 50} { 28 5d4b83524caf 2 58 0 58} { 29 29cc4c190c13 2 57 0 57} { 30 5a6f1469926a 2 58 0 58} { 31 c5b505ea3590 2 67 0 67} { 32 4b0176f1cd2e 2 60 0 60} { 33 7c9006b0e87f 2 66 0 66} { 34 19d8811ffb3d 2 59 0 59} { 35 3fdbe2615352 2 47 0 47} { 36 e6deeee29bb0 2 58 0 58} { 37 e68d5e39227c 2 54 0 54} { 38 ab15442a5080 2 60 0 60} { 39 ca69c730bc34 2 67 0 67} { 40 56164fd8de1 2 60 0 60} { 41 ebe7efa67f76 2 58 0 58} { 42 e60a302ecd16 2 46 0 46} { 43 8507a87f9e25 2 55 0 55} { 44 9032297f747d 2 67 0 67} { 45 b303e4cbf65 2 67 0 67} { 46 dd00a64c824c 2 54 0 54} { 47 f68a65c53f3b 2 53 0 53} { 48 ceccbf4100c5 2 59 0 59} { 49 c3f779d8f736 2 34 0 34} { 50 bc997940b90f 2 64 0 64} { 51 b64bc49cddb5 2 54 0 54} { 52 aba36b4061ba 2 56 0 56} { 53 ba5374260437 2 56 0 56} { 54 fe140d6ec9f3 2 55 0 55} { 55 9aaa569d49b4 2 70 0 70} { 56 f0ddba2e2d47 2 54 0 54} { 57 75b3006d77c8 2 55 0 55} { 58 e97a32b315f 2 53 0 53} { 59 71b1f161e0aa 2 53 0 53} { 60 44ba1ced7506 2 56 0 56} { 61 61411e5267ee 2 53 0 53} { 62 99313d3a9d3 2 57 0 57} { 63 79a4ca4e4e3d 2 56 0 56} 2023-10-21T04:18:27.701+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 9112972261750078043 INDEX_STATE_ACTIVE 2023-10-21T04:18:27.702+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:18:27.702+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:18:27.702+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:27.702+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 1 New 0 2023-10-21T04:18:27.703+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 65897 2023-10-21T04:18:27.704+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:27.705+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:27.706+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:27.709+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:27.709+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:27.710+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65897 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:27.712+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 9112972261750078043 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:18:27.712+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:18:27.711+05:30 [Info] switched currmeta from 550 -> 551 force true 2023-10-21T04:18:27.714+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:27.714+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:27.714+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 65897 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:27.715+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:27.715+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:18 1:14 2:15 3:7 4:19 5:8 6:18 7:18 8:9 9:11 10:17 11:12 12:14 13:15 14:21 15:12 16:10 17:12 18:18 19:8 20:15 21:8 22:14 23:13 24:11 25:23 26:11 27:16 28:17 29:18 30:14 31:19 32:17 33:17 34:17 35:15 36:14 37:15 38:16 39:25 40:22 41:17 42:11 43:13 44:33 45:16 46:17 47:14 48:17 49:14 50:22 51:23 52:17 53:13 54:23 55:22 56:10 57:13 58:14 59:13 60:16 61:14 62:19 63:16] 2023-10-21T04:18:27.715+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:36 1:34 2:36 3:21 4:38 5:38 6:46 7:46 8:34 9:31 10:42 11:34 12:34 13:31 14:48 15:36 16:35 17:34 18:45 19:29 20:35 21:32 22:32 23:34 24:36 25:50 26:29 27:35 28:36 29:38 30:44 31:44 32:41 33:43 34:41 35:37 36:37 37:41 38:38 39:46 40:43 41:38 42:34 43:34 44:55 45:41 46:42 47:37 48:42 49:28 50:47 51:43 52:40 53:38 54:45 55:46 56:39 57:38 58:33 59:41 60:41 61:36 62:41 63:40] 2023-10-21T04:18:27.715+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:27.715+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:53304: use of closed network connection 2023-10-21T04:18:27.715+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:53304" exit: read tcp4 127.0.0.1:9121->127.0.0.1:53304: use of closed network connection 2023-10-21T04:18:27.715+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53304" closed ! 2023-10-21T04:18:27.715+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:27.715+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:53304" already gone 2023-10-21T04:18:27.716+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:27.716+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:27.716+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:27.716+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:27.716+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:27.716+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:27.716+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 65897 2023-10-21T04:18:27.720+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:18:27.730+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 SnapshotId 3 Created recovery point (took 735.848361ms) 2023-10-21T04:18:27.731+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:18:27.732+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:18:27.733+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.715913ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:27.745+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 SnapshotId 3 Created recovery point (took 750.82262ms) 2023-10-21T04:18:27.748+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:18:27.749+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:18:27.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:27.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:27.755+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 SnapshotId 3 Created recovery point (took 760.890498ms) 2023-10-21T04:18:27.756+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:18:27.757+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:18:27.759+05:30 [Info] switched currmeta from 551 -> 552 force true 2023-10-21T04:18:27.765+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 SnapshotId 3 Created recovery point (took 771.068918ms) 2023-10-21T04:18:27.766+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:18:27.767+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:18:27.773+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:18:27.773+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:27.773+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:18:27.774+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65897 2023-10-21T04:18:27.774+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 65897 2023-10-21T04:18:27.774+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:18:27.792+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 SnapshotId 3 Created recovery point (took 798.078342ms) 2023-10-21T04:18:27.793+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:18:27.794+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:18:27.807+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 SnapshotId 3 Created recovery point (took 812.911168ms) 2023-10-21T04:18:27.808+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:18:27.809+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:18:27.827+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 SnapshotId 3 Created recovery point (took 833.073885ms) 2023-10-21T04:18:27.829+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:18:27.829+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:18:27.847+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 SnapshotId 3 Created recovery point (took 852.520074ms) 2023-10-21T04:18:27.848+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:18:27.849+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:18:27.864+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 SnapshotId 3 Created recovery point (took 869.246613ms) 2023-10-21T04:18:27.864+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:18:27.865+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:18:27.874+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 SnapshotId 3 Created recovery point (took 879.574752ms) 2023-10-21T04:18:27.875+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:27.875+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:18:27.883+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 SnapshotId 3 Created recovery point (took 890.568653ms) 2023-10-21T04:18:27.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:18:27.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:18:27.893+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9112972261750078043, PartitionId 0 SnapshotId 2 Created recovery point (took 730.588772ms) 2023-10-21T04:18:28.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11796582816259958249:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:28.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11796582816259958249:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:28.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11796582816259958249:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:28.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11796582816259958249:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:28.699+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:28.790+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:28.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9112972261750078043:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:29.823+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:18:29.828+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:29.828+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:29.830+05:30 [Info] switched currmeta from 552 -> 552 force true 2023-10-21T04:18:36.810+05:30 [Info] builder: Processing build token /indexing/ddl/commandToken/build/6985049432225870291 2023-10-21T04:18:36.810+05:30 [Info] builder: Schedule index build for (bucket_1, _default, c2%, idx_secondary_defer). 2023-10-21T04:18:36.817+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:36.817+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:36.819+05:30 [Info] switched currmeta from 552 -> 553 force true 2023-10-21T04:18:36.829+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [4173530108555528677] &{0} 2023-10-21T04:18:36.829+05:30 [Info] Indexer::handleBuildIndex [4173530108555528677] 2023-10-21T04:18:36.832+05:30 [Info] bulkUpdateRState: Index instance 4173530108555528677 rstate moved to ACTIVE 2023-10-21T04:18:36.833+05:30 [Info] Indexer::handleBuildIndex Added Index: [4173530108555528677] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:18:36.834+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:36.834+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:36.835+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:36.835+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:18:36.835+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:36.835+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:36.835+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 95346 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:36.836+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:18:36.836+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:18:36.836+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:36.836+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 95346 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:36.836+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:18:36.836+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:36.837+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:36.837+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c2% reset bucket state to NIL 2023-10-21T04:18:36.837+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:18:36.837+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:36.837+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:36.837+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c2% Added for Stream INIT_STREAM 2023-10-21T04:18:36.837+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:13] INIT_STREAM:map[bucket_1:_default:c2%:1]] 2023-10-21T04:18:36.837+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4173530108555528677 stream INIT_STREAM keyspaceId bucket_1:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:18:36.837+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:18:36.837+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [0=55,1=53,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=68,15=50,16=55,17=51,18=62,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=56,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=57,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=52,59=53,60=56,61=53,62=57,63=56] IndexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 95347 CollectionId: d CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:18:36.838+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:18:36.838+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:18:36.839+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:36.840+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 10.561723ms 2023-10-21T04:18:36.840+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 10.746592ms 2023-10-21T04:18:36.840+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [4173530108555528677] 2023-10-21T04:18:36.840+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_INDEX elapsed 19.444297ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:36.846+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c2% Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:18:36.846+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.228789ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:36.847+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c2% Check pending stream merge. 2023-10-21T04:18:36.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:36.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:36.855+05:30 [Info] switched currmeta from 553 -> 556 force true 2023-10-21T04:18:36.873+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:36.873+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:36.890+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53552" doAuth() ... 2023-10-21T04:18:36.890+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53552" auth successful 2023-10-21T04:18:36.891+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:53552" +1 2023-10-21T04:18:36.893+05:30 [Info] switched currmeta from 556 -> 556 force true 2023-10-21T04:18:37.249+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 0 186517626268899 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.252+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 1 178840478062605 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.254+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 2 148075163102199 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.254+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 4 28605411869539 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.254+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 3 215374859564337 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.259+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 5 39552632604610 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.259+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 6 242327631330054 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.261+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 8 50961953886801 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.261+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 7 191903564840858 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 10 274410200699189 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.267+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 12 4429827396609 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.267+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 11 66881023717063 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 9 225280996447309 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.273+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 13 183148377680315 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.274+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 15 127185710837163 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.274+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 14 226814108379280 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.276+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 16 257031483891250 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.280+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 17 55290066231400 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.280+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 18 242578104468179 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.283+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 19 210306088158507 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 21 209771427122473 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 20 230239359684242 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.288+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 22 260036488774725 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.292+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 24 160961818008256 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.292+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 23 216462437971699 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.298+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 25 280874278672777 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.298+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 26 219132834127088 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.298+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 27 184056281482149 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.312+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 31 217381278987664 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.312+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 33 136958029391999 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.312+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 29 45957426777107 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.312+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 32 82469662608686 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.312+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 30 99433130332778 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.312+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 28 102578907139247 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.322+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 37 253494845579900 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.322+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 38 188107826286720 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.322+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 34 28417669987133 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.322+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 39 222555662236724 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.322+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 35 70213628416850 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.322+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 36 253845164956592 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.328+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 40 5915864305121 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.341+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 42 252931432434966 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.341+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 41 259381390638966 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.341+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 43 146267938201125 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 45 12301831552869 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.351+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 44 158545118983293 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 47 271074273345339 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 46 242994859770444 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.358+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 49 215467668600630 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.358+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 48 227378777358533 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.360+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 50 207367350302991 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 53 204867593700407 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 52 188718367400378 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.375+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 59 125009072873642 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.375+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 54 279362078165491 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 55 170056388266420 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 51 200436537417141 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 57 129411666769864 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 56 264835102027079 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 60 75566139929862 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.380+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 58 16044440367455 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.381+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 61 106932309485550 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.383+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 63 133748675726909 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.383+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c2% 62 10527297481171 0 95347 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:37.384+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_1:_default:c2% InstanceIds [4173530108555528677] 2023-10-21T04:18:37.385+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% Cid d SessionId 95347 2023-10-21T04:18:37.385+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:18:37.386+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:18:37.386+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_1:_default:c2%. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:18:37.386+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 60 Crc64: 3481855111340995622 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 0 55} { 1 a2a78b2a980d 3 59 0 59} { 2 86ac6f7f2bf7 3 38 0 54} { 3 c3e1ddff7531 3 35 0 35} { 4 1a043769c363 3 58 0 58} { 5 23f9109dc7c2 3 52 0 52} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 3 0 0 0} { 8 2e5981289251 3 40 0 45} { 9 cce451c4944d 0 0 0 0} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} HWT bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 0 55} { 1 a2a78b2a980d 3 59 0 59} { 2 86ac6f7f2bf7 3 38 0 54} { 3 c3e1ddff7531 3 35 0 35} { 4 1a043769c363 3 58 0 58} { 5 23f9109dc7c2 3 52 0 52} { 6 dc654c627b06 0 0 0 0} { 7 ae8907d03f9a 3 49 0 63} { 8 2e5981289251 3 40 0 45} { 9 cce451c4944d 3 33 0 55} { 10 f9931a891135 0 0 0 0} { 11 3cd3f3a92ec7 0 0 0 0} { 12 4076649d401 0 0 0 0} { 13 a6928dfe29bb 0 0 0 0} { 14 ce49465c2890 0 0 0 0} { 15 73acbb1d6dab 0 0 0 0} { 16 e9c4ce4b1e32 0 0 0 0} { 17 324938c1e068 0 0 0 0} { 18 dc9f9dbf42d3 0 0 0 0} { 19 bf45b3adc52b 0 0 0 0} { 20 d166c73f4e92 0 0 0 0} { 21 bec93765cd29 0 0 0 0} { 22 ec80768ac445 0 0 0 0} { 23 c4df16b8f6f3 0 0 0 0} { 24 9264d7fc1ac0 0 0 0 0} { 25 ff74239b5989 0 0 0 0} { 26 c74cd6bc9cf0 0 0 0 0} { 27 a765f146c3a5 0 0 0 0} { 28 5d4b83524caf 0 0 0 0} { 29 29cc4c190c13 0 0 0 0} { 30 5a6f1469926a 0 0 0 0} { 31 c5b505ea3590 0 0 0 0} { 32 4b0176f1cd2e 0 0 0 0} { 33 7c9006b0e87f 0 0 0 0} { 34 19d8811ffb3d 0 0 0 0} { 35 3fdbe2615352 0 0 0 0} { 36 e6deeee29bb0 0 0 0 0} { 37 e68d5e39227c 0 0 0 0} { 38 ab15442a5080 0 0 0 0} { 39 ca69c730bc34 0 0 0 0} { 40 56164fd8de1 0 0 0 0} { 41 ebe7efa67f76 0 0 0 0} { 42 e60a302ecd16 0 0 0 0} { 43 8507a87f9e25 0 0 0 0} { 44 9032297f747d 0 0 0 0} { 45 b303e4cbf65 0 0 0 0} { 46 dd00a64c824c 0 0 0 0} { 47 f68a65c53f3b 0 0 0 0} { 48 ceccbf4100c5 0 0 0 0} { 49 c3f779d8f736 0 0 0 0} { 50 bc997940b90f 0 0 0 0} { 51 b64bc49cddb5 0 0 0 0} { 52 aba36b4061ba 0 0 0 0} { 53 ba5374260437 0 0 0 0} { 54 fe140d6ec9f3 0 0 0 0} { 55 9aaa569d49b4 0 0 0 0} { 56 f0ddba2e2d47 0 0 0 0} { 57 75b3006d77c8 0 0 0 0} { 58 e97a32b315f 0 0 0 0} { 59 71b1f161e0aa 0 0 0 0} { 60 44ba1ced7506 0 0 0 0} { 61 61411e5267ee 0 0 0 0} { 62 99313d3a9d3 0 0 0 0} { 63 79a4ca4e4e3d 0 0 0 0} 2023-10-21T04:18:37.388+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:18:37.388+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:18:37.389+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:37.390+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:37.390+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [0=55,1=53,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=68,15=50,16=55,17=51,18=62,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=56,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=57,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=52,59=53,60=56,61=53,62=57,63=56] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:37.780+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c2% setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:37.790+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4173530108555528677 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:37.791+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4173530108555528677 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.862µs SnapOpenDur 621.848µs 2023-10-21T04:18:37.791+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 4173530108555528677 INDEX_STATE_CATCHUP 2023-10-21T04:18:37.791+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 4173530108555528677 true 2023-10-21T04:18:37.791+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 4173530108555528677 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% Session: 95347 BuildTS: [0=55,1=53,2=54,3=35,4=58,5=52,6=62,7=63,8=45,9=53,10=54,11=51,12=42,13=46,14=68,15=50,16=55,17=51,18=62,19=48,20=50,21=47,22=51,23=57,24=59,25=71,26=52,27=49,28=58,29=56,30=58,31=67,32=60,33=65,34=58,35=47,36=58,37=54,38=60,39=67,40=57,41=58,42=46,43=55,44=67,45=63,46=54,47=53,48=57,49=34,50=64,51=54,52=56,53=56,54=55,55=70,56=52,57=55,58=52,59=53,60=56,61=53,62=57,63=56] 2023-10-21T04:18:37.791+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4173530108555528677, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:37.795+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId: 95347 2023-10-21T04:18:37.795+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c2% 95347 2023-10-21T04:18:37.797+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:37.799+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:37.799+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:37.800+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:37.800+05:30 [Info] send progress stats to clients 2023-10-21T04:18:37.800+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c2% set minMergeTs 2023-10-21T04:18:37.800+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c2% StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:37.800+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:14] INIT_STREAM:map[bucket_1:_default:c2%:1]] 2023-10-21T04:18:37.800+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 95347 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:37.805+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:37.815+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_1 2023-10-21T04:18:37.815+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95347 2023-10-21T04:18:37.815+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95347 2023-10-21T04:18:37.815+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c2% set minMergeTs 2023-10-21T04:18:37.816+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:18:37.816+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c2% 4173530108555528677 false 2023-10-21T04:18:37.816+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c2% 4173530108555528677 2023-10-21T04:18:37.816+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c2%. 2023-10-21T04:18:37.816+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 95347 2023-10-21T04:18:37.819+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.898466ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:37.826+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:37.826+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:37.830+05:30 [Info] switched currmeta from 556 -> 558 force true 2023-10-21T04:18:37.836+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:37.837+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:37.842+05:30 [Info] switched currmeta from 558 -> 558 force true 2023-10-21T04:18:37.878+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4173530108555528677, PartitionId 0 SnapshotId 3 Created recovery point (took 85.995567ms) 2023-10-21T04:18:38.389+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c2% StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:38.391+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c2% 2023-10-21T04:18:38.392+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 4173530108555528677 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% SessionId: 95347, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 55 0 55} { 1 a2a78b2a980d 3 59 0 59} { 2 86ac6f7f2bf7 3 54 0 54} { 3 c3e1ddff7531 3 35 0 35} { 4 1a043769c363 3 58 0 58} { 5 23f9109dc7c2 3 52 0 52} { 6 dc654c627b06 3 62 0 62} { 7 ae8907d03f9a 3 63 0 63} { 8 2e5981289251 3 45 0 45} { 9 cce451c4944d 3 55 0 55} { 10 f9931a891135 3 55 0 55} { 11 3cd3f3a92ec7 3 51 0 51} { 12 4076649d401 3 42 0 42} { 13 a6928dfe29bb 3 47 0 47} { 14 ce49465c2890 3 72 0 72} { 15 73acbb1d6dab 3 50 0 50} { 16 e9c4ce4b1e32 3 59 0 59} { 17 324938c1e068 3 51 0 51} { 18 dc9f9dbf42d3 3 63 0 63} { 19 bf45b3adc52b 3 48 0 48} { 20 d166c73f4e92 3 50 0 50} { 21 bec93765cd29 3 48 0 48} { 22 ec80768ac445 3 51 0 51} { 23 c4df16b8f6f3 3 58 0 58} { 24 9264d7fc1ac0 3 59 0 59} { 25 ff74239b5989 3 71 0 71} { 26 c74cd6bc9cf0 3 52 0 52} { 27 a765f146c3a5 3 50 0 50} { 28 5d4b83524caf 3 58 0 58} { 29 29cc4c190c13 3 57 0 57} { 30 5a6f1469926a 3 58 0 58} { 31 c5b505ea3590 3 67 0 67} { 32 4b0176f1cd2e 3 60 0 60} { 33 7c9006b0e87f 3 66 0 66} { 34 19d8811ffb3d 3 59 0 59} { 35 3fdbe2615352 3 47 0 47} { 36 e6deeee29bb0 3 58 0 58} { 37 e68d5e39227c 3 54 0 54} { 38 ab15442a5080 3 60 0 60} { 39 ca69c730bc34 3 67 0 67} { 40 56164fd8de1 3 60 0 60} { 41 ebe7efa67f76 3 58 0 58} { 42 e60a302ecd16 3 46 0 46} { 43 8507a87f9e25 3 55 0 55} { 44 9032297f747d 3 67 0 67} { 45 b303e4cbf65 3 68 0 68} { 46 dd00a64c824c 3 54 0 54} { 47 f68a65c53f3b 3 53 0 53} { 48 ceccbf4100c5 3 59 0 59} { 49 c3f779d8f736 3 34 0 34} { 50 bc997940b90f 3 64 0 64} { 51 b64bc49cddb5 3 54 0 54} { 52 aba36b4061ba 3 56 0 56} { 53 ba5374260437 3 56 0 56} { 54 fe140d6ec9f3 3 55 0 55} { 55 9aaa569d49b4 3 71 0 71} { 56 f0ddba2e2d47 3 54 0 54} { 57 75b3006d77c8 3 55 0 55} { 58 e97a32b315f 3 53 0 53} { 59 71b1f161e0aa 3 53 0 53} { 60 44ba1ced7506 3 56 0 56} { 61 61411e5267ee 3 53 0 53} { 62 99313d3a9d3 3 57 0 57} { 63 79a4ca4e4e3d 3 56 0 56} 2023-10-21T04:18:38.395+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c2% 4173530108555528677 INDEX_STATE_ACTIVE 2023-10-21T04:18:38.396+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% State Changed to INACTIVE 2023-10-21T04:18:38.397+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:18:38.398+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:18:38.399+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 1 New 0 2023-10-21T04:18:38.399+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId: 95347 2023-10-21T04:18:38.397+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:38.406+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:38.408+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:38.409+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:38.409+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:38.409+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 95347 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:38.409+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4173530108555528677 from stream INIT_STREAM keyspaceId bucket_1:_default:c2% 2023-10-21T04:18:38.409+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c2% 2023-10-21T04:18:38.409+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:18:38.409+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:38.409+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 95347 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:38.410+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:38.410+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c2% total received key-versions: map[0:18 1:15 2:17 3:13 4:19 5:13 6:15 7:16 8:10 9:21 10:11 11:16 12:7 13:14 14:19 15:13 16:18 17:16 18:16 19:18 20:14 21:13 22:18 23:22 24:21 25:20 26:22 27:13 28:21 29:17 30:13 31:22 32:18 33:21 34:16 35:9 36:20 37:12 38:21 39:20 40:13 41:19 42:11 43:19 44:11 45:17 46:11 47:15 48:14 49:5 50:16 51:10 52:15 53:17 54:9 55:23 56:12 57:16 58:18 59:11 60:14 61:16 62:15 63:15] 2023-10-21T04:18:38.410+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c2% latest sequence numbers: map[0:55 1:53 2:54 3:35 4:58 5:52 6:62 7:63 8:45 9:53 10:54 11:51 12:42 13:46 14:68 15:50 16:55 17:51 18:62 19:48 20:50 21:47 22:51 23:57 24:59 25:71 26:52 27:49 28:58 29:56 30:58 31:67 32:60 33:65 34:58 35:47 36:58 37:54 38:60 39:67 40:57 41:58 42:46 43:55 44:67 45:63 46:54 47:53 48:57 49:34 50:64 51:54 52:56 53:56 54:55 55:70 56:52 57:55 58:52 59:53 60:56 61:53 62:57 63:56] 2023-10-21T04:18:38.412+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:38.414+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:53552: use of closed network connection 2023-10-21T04:18:38.414+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:53552" exit: read tcp4 127.0.0.1:9121->127.0.0.1:53552: use of closed network connection 2023-10-21T04:18:38.414+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53552" closed ! 2023-10-21T04:18:38.415+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:38.419+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:38.419+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:38.419+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:38.419+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:38.419+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:38.420+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:38.421+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c2% Stream: INIT_STREAM SessionId 95347 2023-10-21T04:18:38.426+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c2%] 2023-10-21T04:18:38.426+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 4.622303ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:38.430+05:30 [Info] switched currmeta from 558 -> 559 force true 2023-10-21T04:18:38.435+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:38.436+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:38.439+05:30 [Info] switched currmeta from 559 -> 560 force true 2023-10-21T04:18:38.459+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1:_default:c2%]. Err genServer.closed 2023-10-21T04:18:38.459+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c2%] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:38.459+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c2%] Treating genServer.closed As Success 2023-10-21T04:18:38.466+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95347 2023-10-21T04:18:38.467+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% SessionId 95347 2023-10-21T04:18:38.467+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c2% 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8177459947280813077:[0]]. New rollbacktime: 1697841977280443054, Old rollbacktime: 0 2023-10-21T04:18:39.795+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.799+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.801+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.801+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.802+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.802+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.803+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.803+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.804+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.804+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.804+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.806+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4173530108555528677:[0]]. New rollbacktime: 1697841976960500515, Old rollbacktime: 0 2023-10-21T04:18:39.897+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:18:39.904+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:39.904+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:39.907+05:30 [Info] switched currmeta from 560 -> 560 force true 2023-10-21T04:18:42.019+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:18:46.342+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [13624500265195250810] &{0} 2023-10-21T04:18:46.342+05:30 [Info] Indexer::handleBuildIndex [13624500265195250810] 2023-10-21T04:18:46.349+05:30 [Info] bulkUpdateRState: Index instance 13624500265195250810 rstate moved to ACTIVE 2023-10-21T04:18:46.349+05:30 [Info] Indexer::handleBuildIndex Added Index: [13624500265195250810] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:18:46.349+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:46.349+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:46.350+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:46.350+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_%2. OSO not supported for default scope/collection. 2023-10-21T04:18:46.350+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2 2023-10-21T04:18:46.350+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:46.350+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:46.350+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 84972 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:46.350+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:18:46.350+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:18:46.350+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:46.350+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 84972 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:46.350+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:18:46.350+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:46.350+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:46.350+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:18:46.350+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:18:46.350+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:46.350+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:46.350+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream INIT_STREAM 2023-10-21T04:18:46.351+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:11 bucket_1:14] INIT_STREAM:map[bucket_%2:1]] 2023-10-21T04:18:46.351+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 13624500265195250810 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:18:46.351+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2 2023-10-21T04:18:46.351+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [0=19,1=19,2=19,3=18,4=19,5=19,6=25,7=19,8=25,9=26,10=19,11=23,12=22,13=22,14=27,15=26,16=20,17=12,18=21,19=17,20=20,21=19,22=23,23=19,24=15,25=22,26=22,27=29,28=18,29=19,30=20,31=22,32=26,33=22,34=26,35=18,36=21,37=20,38=25,39=19,40=27,41=25,42=26,43=26,44=18,45=18,46=25,47=20,48=21,49=27,50=25,51=20,52=19,53=25,54=24,55=28,56=21,57=24,58=20,59=26,60=19,61=14,62=23,63=21] IndexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 84973 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:46.351+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:18:46.351+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:18:46.352+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:46.359+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 16.865094ms 2023-10-21T04:18:46.359+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 16.951755ms 2023-10-21T04:18:46.360+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [13624500265195250810] 2023-10-21T04:18:46.360+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_INDEX elapsed 24.650672ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:46.361+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2 Check pending stream merge. 2023-10-21T04:18:46.370+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:18:46.384+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:46.384+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:46.388+05:30 [Info] switched currmeta from 560 -> 562 force true 2023-10-21T04:18:46.399+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:46.399+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:46.403+05:30 [Info] switched currmeta from 562 -> 562 force true 2023-10-21T04:18:46.405+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 45.457786ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:46.417+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:46.417+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:46.421+05:30 [Info] switched currmeta from 562 -> 564 force true 2023-10-21T04:18:46.428+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:46.428+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:46.432+05:30 [Info] switched currmeta from 564 -> 564 force true 2023-10-21T04:18:46.493+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53782" doAuth() ... 2023-10-21T04:18:46.493+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53782" auth successful 2023-10-21T04:18:46.493+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:53782" +1 2023-10-21T04:18:46.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 1 176639880977228 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 0 263268055970459 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.818+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 4 159113185399719 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.818+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 3 215749515705641 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.818+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 5 3874519919278 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 2 35403626585217 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 7 170288557775498 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.823+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 6 261493429196201 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.823+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 8 264975169139761 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.823+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 9 236398693961229 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 12 116104661063478 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 11 234758907181738 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 13 102350060311343 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.833+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 10 206533515052028 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 19 208836052671833 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 14 140236896721046 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 16 200752780087280 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 15 162158720181435 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 17 98820809009123 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 18 207134287529256 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 20 92956357968546 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.846+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 21 27979933591532 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.846+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 22 192215088742243 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.848+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 23 154934968850942 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 24 271667630317209 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.855+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 26 120989102157727 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.855+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 25 168740244208559 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.857+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 27 5715487545729 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.862+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 29 130231725986856 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.862+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 28 236953356346174 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 30 60545336618717 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.868+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 32 67362620867705 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.868+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 31 73432771436458 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.875+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 33 278426329668882 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 34 51776324008509 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.880+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 35 24368176313355 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 36 43594649874772 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.884+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 37 83647976044922 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.888+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 38 181267443431019 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.889+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 39 100659545510465 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 40 153478150002150 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 42 45479338375128 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 41 116785907755771 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.893+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 44 166675095997741 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.893+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 43 249919763471169 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.893+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 45 49796766757346 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.893+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 46 124194574848641 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.896+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 47 149022976193937 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 50 199728167534433 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 48 19274425379955 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 49 184487117398647 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.900+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 51 196777743741606 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.903+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 52 158076890855634 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 59 140942236111856 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 60 56626109228904 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 57 191719959781498 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 53 138935694816848 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 54 24856128127173 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 56 29071415952852 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.913+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 55 61798873988101 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.913+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 58 103336286678362 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.915+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 62 177105814270799 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.916+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 61 119871837460216 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.917+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2 InstanceIds [13624500265195250810] 2023-10-21T04:18:46.918+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2 Cid 0 SessionId 84973 2023-10-21T04:18:46.918+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:18:46.918+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:18:46.918+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_%2. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:18:46.918+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 53 Crc64: 13149233316735112663 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 0 14 0 52} { 1 a0a72d5bdb4c 0 16 0 62} { 2 20330c961c81 0 50 0 50} { 3 c439193e7d29 0 16 0 45} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 0 0 0 0} { 6 edd3aef555a9 0 61 0 61} { 7 9ae064feda8a 0 0 0 0} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 47 87891d8f3591 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 49 a7ca41192277 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 51 b2f7e383f2a6 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} HWT bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 63 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 0 15 0 52} { 1 a0a72d5bdb4c 0 16 0 62} { 2 20330c961c81 0 50 0 50} { 3 c439193e7d29 0 16 0 45} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 0 0 0 0} { 6 edd3aef555a9 0 61 0 61} { 7 9ae064feda8a 0 0 0 0} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 47 87891d8f3591 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 49 a7ca41192277 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 51 b2f7e383f2a6 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} { 53 7e5c7cbc7650 0 0 0 0} { 54 169b44b000c5 0 0 0 0} { 55 3834abee7c05 0 0 0 0} { 56 1a70b76bc5d4 0 0 0 0} { 57 ae5e4819607a 0 0 0 0} { 58 5dfbdaa8ed5a 0 0 0 0} { 59 802fabead3f0 0 0 0 0} { 60 33804b202768 0 0 0 0} { 61 6d05d64a56f8 0 0 0 0} { 62 a113a925b34f 0 0 0 0} 2023-10-21T04:18:46.918+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:18:46.918+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:18:46.918+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:46.925+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 63 252537041863196 0 84973 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:46.928+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [0=19,1=19,2=19,3=18,4=19,5=19,6=25,7=19,8=25,9=26,10=19,11=23,12=22,13=22,14=27,15=26,16=20,17=12,18=21,19=17,20=20,21=19,22=23,23=19,24=15,25=22,26=22,27=29,28=18,29=19,30=20,31=22,32=26,33=22,34=26,35=18,36=21,37=20,38=25,39=19,40=27,41=25,42=26,43=26,44=18,45=18,46=25,47=20,48=21,49=27,50=25,51=20,52=19,53=25,54=24,55=28,56=21,57=24,58=20,59=26,60=19,61=14,62=23,63=21] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:47.271+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:47.276+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13624500265195250810 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:47.276+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13624500265195250810, PartitionId 0 SnapshotId 1 Creating recovery point ... 2023-10-21T04:18:47.276+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13624500265195250810 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.603271ms SnapOpenDur 180.162µs 2023-10-21T04:18:47.276+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 13624500265195250810 INDEX_STATE_CATCHUP 2023-10-21T04:18:47.276+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 13624500265195250810 true 2023-10-21T04:18:47.276+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 13624500265195250810 Stream: INIT_STREAM KeyspaceId: bucket_%2 Session: 84973 BuildTS: [0=19,1=19,2=19,3=18,4=19,5=19,6=25,7=19,8=25,9=26,10=19,11=23,12=22,13=22,14=27,15=26,16=20,17=12,18=21,19=17,20=20,21=19,22=23,23=19,24=15,25=22,26=22,27=29,28=18,29=19,30=20,31=22,32=26,33=22,34=26,35=18,36=21,37=20,38=25,39=19,40=27,41=25,42=26,43=26,44=18,45=18,46=25,47=20,48=21,49=27,50=25,51=20,52=19,53=25,54=24,55=28,56=21,57=24,58=20,59=26,60=19,61=14,62=23,63=21] 2023-10-21T04:18:47.277+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2 Stream: INIT_STREAM SessionId: 84973 2023-10-21T04:18:47.277+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2 84973 2023-10-21T04:18:47.277+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:47.277+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:47.278+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:47.278+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:47.278+05:30 [Info] send progress stats to clients 2023-10-21T04:18:47.278+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2 set minMergeTs 2023-10-21T04:18:47.278+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:47.278+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:12 bucket_1:14] INIT_STREAM:map[bucket_%2:1]] 2023-10-21T04:18:47.278+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 84973 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:47.282+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:47.307+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 2023-10-21T04:18:47.307+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2 SessionId 84973 2023-10-21T04:18:47.307+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 84973 2023-10-21T04:18:47.307+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2 set minMergeTs 2023-10-21T04:18:47.307+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:18:47.307+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 13624500265195250810 false 2023-10-21T04:18:47.307+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2 13624500265195250810 2023-10-21T04:18:47.307+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2. 2023-10-21T04:18:47.307+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 84973 2023-10-21T04:18:47.311+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 31.08712ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:47.322+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:47.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:47.328+05:30 [Info] switched currmeta from 564 -> 566 force true 2023-10-21T04:18:47.334+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:47.334+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:47.335+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13624500265195250810, PartitionId 0 SnapshotId 1 Created recovery point (took 58.252238ms) 2023-10-21T04:18:47.338+05:30 [Info] switched currmeta from 566 -> 566 force true 2023-10-21T04:18:47.911+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:47.911+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:47.914+05:30 [Info] switched currmeta from 566 -> 567 force true 2023-10-21T04:18:47.919+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:47.919+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:18:47.919+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 13624500265195250810 Stream: INIT_STREAM KeyspaceId: bucket_%2 SessionId: 84973, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 0 52 0 52} { 1 a0a72d5bdb4c 0 62 0 62} { 2 20330c961c81 0 50 0 50} { 3 c439193e7d29 0 45 0 45} { 4 90b66ce513a7 0 58 0 58} { 5 3861b622aae 0 54 0 54} { 6 edd3aef555a9 0 61 0 61} { 7 9ae064feda8a 0 52 0 52} { 8 f0fe56d4b431 0 56 0 56} { 9 d700dc2dce0d 0 62 0 62} { 10 bbd754cc83fc 0 50 0 50} { 11 d58311453aaa 0 56 0 56} { 12 6998b9145736 0 55 0 55} { 13 5d163afcd32f 0 53 0 53} { 14 7f8b7269d896 0 60 0 60} { 15 937b84e9e4bb 0 67 0 67} { 16 b69566250ff0 0 55 0 55} { 17 59e08337a7e3 0 46 0 46} { 18 bc6335a10528 0 46 0 46} { 19 bdef6ebc5d59 0 45 0 45} { 20 548b16b242a2 0 50 0 50} { 21 19729600a7ec 0 54 0 54} { 22 aed19015bf63 0 55 0 55} { 23 8ce99bc819fe 0 51 0 51} { 24 f7148c9a1e99 0 52 0 52} { 25 9977e650bbaf 0 61 0 61} { 26 6e09f874af9f 0 59 0 59} { 27 532bd9ad981 0 54 0 54} { 28 d78200a2233e 0 54 0 54} { 29 7671efc41428 0 49 0 49} { 30 3710cf482edd 0 46 0 46} { 31 42c966419faa 0 60 0 60} { 32 3d4415168479 0 67 0 67} { 33 fd3a2e79dd12 0 54 0 54} { 34 2f171d667a3d 0 60 0 60} { 35 1629a87e340b 0 50 0 50} { 36 27a62b9eb154 0 52 0 52} { 37 4c13cfe5f97a 0 59 0 59} { 38 a4dc9d93366b 0 53 0 53} { 39 5b8ca0741241 0 49 0 49} { 40 8b966a9c95e6 0 68 0 68} { 41 6a37568beafb 0 55 0 55} { 42 295cfbcef7d8 0 55 0 55} { 43 e34cfab95b41 0 66 0 66} { 44 979711e3cd2d 0 57 0 57} { 45 2d4a3697f1e2 0 62 0 62} { 46 70f44d845281 0 55 0 55} { 47 87891d8f3591 0 55 0 55} { 48 1187ad472c73 0 55 0 55} { 49 a7ca41192277 0 55 0 55} { 50 b5a6d67a3f61 0 66 0 66} { 51 b2f7e383f2a6 0 51 0 51} { 52 8fc524ed38d2 0 54 0 54} { 53 7e5c7cbc7650 0 61 0 61} { 54 169b44b000c5 0 60 0 60} { 55 3834abee7c05 0 67 0 67} { 56 1a70b76bc5d4 0 61 0 61} { 57 ae5e4819607a 0 63 0 63} { 58 5dfbdaa8ed5a 0 46 0 46} { 59 802fabead3f0 0 63 0 63} { 60 33804b202768 0 46 0 46} { 61 6d05d64a56f8 0 39 0 39} { 62 a113a925b34f 0 58 0 58} { 63 e5ae5cac421c 0 73 0 73} 2023-10-21T04:18:47.920+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2 Stream: INIT_STREAM SessionId: 84973 2023-10-21T04:18:47.921+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:47.921+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:47.921+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 13624500265195250810 INDEX_STATE_ACTIVE 2023-10-21T04:18:47.921+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2 State Changed to INACTIVE 2023-10-21T04:18:47.921+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:18:47.921+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:18:47.921+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 1 New 0 2023-10-21T04:18:47.921+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:47.922+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 84973 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:47.922+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 13624500265195250810 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:18:47.922+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:18:47.922+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:18:47.922+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:47.922+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 84973 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:47.922+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:47.922+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2 total received key-versions: map[0:13 1:13 2:13 3:12 4:13 5:13 6:19 7:13 8:19 9:20 10:13 11:17 12:16 13:16 14:21 15:20 16:14 17:6 18:15 19:11 20:14 21:13 22:17 23:13 24:9 25:16 26:16 27:23 28:12 29:13 30:14 31:16 32:20 33:16 34:20 35:12 36:15 37:14 38:19 39:13 40:21 41:19 42:20 43:20 44:12 45:12 46:19 47:14 48:15 49:21 50:19 51:14 52:13 53:19 54:18 55:22 56:15 57:18 58:14 59:20 60:13 61:8 62:17 63:15] 2023-10-21T04:18:47.922+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2 latest sequence numbers: map[0:19 1:19 2:19 3:18 4:19 5:19 6:25 7:19 8:25 9:26 10:19 11:23 12:22 13:22 14:27 15:26 16:20 17:12 18:21 19:17 20:20 21:19 22:23 23:19 24:15 25:22 26:22 27:29 28:18 29:19 30:20 31:22 32:26 33:22 34:26 35:18 36:21 37:20 38:25 39:19 40:27 41:25 42:26 43:26 44:18 45:18 46:25 47:20 48:21 49:27 50:25 51:20 52:19 53:25 54:24 55:28 56:21 57:24 58:20 59:26 60:19 61:14 62:23 63:21] 2023-10-21T04:18:47.923+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:47.923+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:53782: use of closed network connection 2023-10-21T04:18:47.923+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:53782" exit: read tcp4 127.0.0.1:9121->127.0.0.1:53782: use of closed network connection 2023-10-21T04:18:47.923+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53782" closed ! 2023-10-21T04:18:47.923+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:47.923+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:47.923+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:47.923+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:47.923+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:47.923+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:47.923+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:47.924+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2 Stream: INIT_STREAM SessionId 84973 2023-10-21T04:18:47.924+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2] 2023-10-21T04:18:47.938+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.926921ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:47.950+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:47.950+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:47.953+05:30 [Info] switched currmeta from 567 -> 568 force true 2023-10-21T04:18:47.954+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2]. Err genServer.closed 2023-10-21T04:18:47.954+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:47.954+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2] Treating genServer.closed As Success 2023-10-21T04:18:47.954+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2 SessionId 84973 2023-10-21T04:18:47.954+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 84973 2023-10-21T04:18:47.954+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:18:48.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.505+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5282181291074303797:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13624500265195250810:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:48.919+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:49.943+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:18:49.948+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:49.948+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:49.950+05:30 [Info] switched currmeta from 568 -> 568 force true 2023-10-21T04:18:51.757+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [8606677778651310689] &{0} 2023-10-21T04:18:51.757+05:30 [Info] Indexer::handleBuildIndex [8606677778651310689] 2023-10-21T04:18:51.773+05:30 [Info] bulkUpdateRState: Index instance 8606677778651310689 rstate moved to ACTIVE 2023-10-21T04:18:51.774+05:30 [Info] Indexer::handleBuildIndex Added Index: [8606677778651310689] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:18:51.774+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:51.774+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:51.778+05:30 [Info] switched currmeta from 568 -> 569 force true 2023-10-21T04:18:51.782+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:51.782+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:51.782+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:51.783+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:51.784+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:51.784+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:51.784+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79188 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:51.784+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:51.784+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:51.785+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:51.785+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79188 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:51.785+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:18:51.785+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:51.785+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:51.785+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:18:51.785+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:18:51.786+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:51.786+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:51.786+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:18:51.786+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:12 bucket_1:14] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:18:51.786+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 8606677778651310689 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:18:51.786+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:51.786+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 79189 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:18:51.787+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:18:51.787+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:18:51.788+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:51.793+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 35.951078ms 2023-10-21T04:18:51.794+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c1 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:18:51.794+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [8606677778651310689] 2023-10-21T04:18:51.794+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 37.062852ms 2023-10-21T04:18:51.798+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:18:51.799+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_INDEX elapsed 54.502655ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:51.844+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 43.548231ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:51.848+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:51.849+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:51.855+05:30 [Info] switched currmeta from 569 -> 572 force true 2023-10-21T04:18:51.861+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53954" doAuth() ... 2023-10-21T04:18:51.861+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53954" auth successful 2023-10-21T04:18:51.861+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:51.861+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:51.862+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:53954" +1 2023-10-21T04:18:51.866+05:30 [Info] switched currmeta from 572 -> 572 force true 2023-10-21T04:18:52.188+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 263268055970459 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.189+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 176639880977228 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.198+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 215749515705641 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.202+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 159113185399719 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.202+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 35403626585217 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.213+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 116104661063478 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.213+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 261493429196201 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.214+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 170288557775498 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.214+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 3874519919278 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.215+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 264975169139761 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.215+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 236398693961229 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.215+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 206533515052028 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.215+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 234758907181738 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.215+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 102350060311343 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.217+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 162158720181435 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.218+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 140236896721046 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.224+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 98820809009123 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.224+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 200752780087280 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.225+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 207134287529256 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.229+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 92956357968546 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.229+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 208836052671833 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.230+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 27979933591532 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.232+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 154934968850942 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.233+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 192215088742243 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.235+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 168740244208559 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.235+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 271667630317209 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.237+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 120989102157727 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.239+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5715487545729 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.239+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 236953356346174 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.241+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 130231725986856 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 67362620867705 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 60545336618717 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 73432771436458 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.251+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 24368176313355 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.257+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 181267443431019 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.257+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 43594649874772 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.264+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 278426329668882 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 153478150002150 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 51776324008509 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 100659545510465 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 83647976044922 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 249919763471169 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 116785907755771 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 49796766757346 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 166675095997741 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 124194574848641 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.278+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 19274425379955 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.278+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 45479338375128 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.281+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 199728167534433 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.284+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 158076890855634 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.288+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 24856128127173 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.291+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 29071415952852 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.293+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 103336286678362 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.297+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 56626109228904 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.300+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 177105814270799 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.304+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c1 InstanceIds [8606677778651310689] 2023-10-21T04:18:52.304+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 79189 2023-10-21T04:18:52.304+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:52.304+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:52.304+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_%2:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:18:52.304+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 53 Crc64: 3346159608272835459 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 20 0 52} { 1 a0a72d5bdb4c 2 62 0 62} { 2 20330c961c81 0 0 0 50} { 3 c439193e7d29 0 0 0 0} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 2 20 0 54} { 6 edd3aef555a9 0 0 0 0} { 7 9ae064feda8a 0 0 0 0} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} { 54 169b44b000c5 0 0 0 0} { 56 1a70b76bc5d4 0 0 0 0} { 58 5dfbdaa8ed5a 0 0 0 0} HWT bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 55 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 20 0 52} { 1 a0a72d5bdb4c 2 62 0 62} { 2 20330c961c81 0 0 0 50} { 3 c439193e7d29 0 0 0 0} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 2 20 0 54} { 6 edd3aef555a9 0 0 0 0} { 7 9ae064feda8a 0 0 0 0} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} { 54 169b44b000c5 0 0 0 0} { 56 1a70b76bc5d4 0 0 0 0} { 58 5dfbdaa8ed5a 0 0 0 0} { 60 33804b202768 0 0 0 0} { 62 a113a925b34f 0 0 0 0} 2023-10-21T04:18:52.305+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:18:52.305+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:18:52.305+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:52.305+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:52.355+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 61798873988101 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.357+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 140942236111856 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.359+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 252537041863196 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 149022976193937 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.362+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 196777743741606 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.363+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 138935694816848 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 119871837460216 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 184487117398647 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.365+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 191719959781498 0 79189 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:52.648+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:52.666+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:52.666+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:52.673+05:30 [Info] switched currmeta from 572 -> 573 force true 2023-10-21T04:18:52.676+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8606677778651310689 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:52.676+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8606677778651310689 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.043µs SnapOpenDur 129.068µs 2023-10-21T04:18:52.676+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 8606677778651310689 INDEX_STATE_CATCHUP 2023-10-21T04:18:52.676+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 8606677778651310689 true 2023-10-21T04:18:52.676+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8606677778651310689, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:18:52.676+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 8606677778651310689 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 79189 BuildTS: [0=37,1=38,2=35,3=33,4=41,5=34,6=47,7=33,8=42,9=41,10=34,11=37,12=35,13=37,14=38,15=47,16=39,17=27,18=34,19=32,20=33,21=36,22=37,23=40,24=33,25=33,26=38,27=47,28=39,29=36,30=32,31=40,32=47,33=38,34=45,35=39,36=33,37=33,38=39,39=29,40=45,41=41,42=46,43=41,44=34,45=34,46=42,47=38,48=38,49=42,50=44,51=36,52=37,53=50,54=41,55=47,56=36,57=41,58=32,59=46,60=37,61=30,62=43,63=50] 2023-10-21T04:18:52.676+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 79189 2023-10-21T04:18:52.677+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 79189 2023-10-21T04:18:52.677+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:52.677+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:52.677+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:52.678+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:52.678+05:30 [Info] send progress stats to clients 2023-10-21T04:18:52.678+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:18:52.678+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:52.678+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:13 bucket_1:14] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:18:52.678+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 79189 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:52.682+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:52.690+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.093593ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:52.704+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 2023-10-21T04:18:52.704+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79189 2023-10-21T04:18:52.704+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79189 2023-10-21T04:18:52.704+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:18:52.704+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:52.704+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 8606677778651310689 false 2023-10-21T04:18:52.704+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 8606677778651310689 2023-10-21T04:18:52.704+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:18:52.704+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 79189 2023-10-21T04:18:52.708+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:52.709+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:52.712+05:30 [Info] switched currmeta from 573 -> 574 force true 2023-10-21T04:18:52.718+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8606677778651310689, PartitionId 0 SnapshotId 3 Created recovery point (took 41.436875ms) 2023-10-21T04:18:53.305+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:53.305+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:53.306+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 8606677778651310689 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 79189, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 2 52 0 52} { 1 a0a72d5bdb4c 2 62 0 62} { 2 20330c961c81 2 50 0 50} { 3 c439193e7d29 2 45 0 45} { 4 90b66ce513a7 2 58 0 58} { 5 3861b622aae 2 54 0 54} { 6 edd3aef555a9 2 61 0 61} { 7 9ae064feda8a 2 52 0 52} { 8 f0fe56d4b431 2 56 0 56} { 9 d700dc2dce0d 2 62 0 62} { 10 bbd754cc83fc 2 50 0 50} { 11 d58311453aaa 2 56 0 56} { 12 6998b9145736 2 55 0 55} { 13 5d163afcd32f 2 53 0 53} { 14 7f8b7269d896 2 60 0 60} { 15 937b84e9e4bb 2 67 0 67} { 16 b69566250ff0 2 55 0 55} { 17 59e08337a7e3 2 46 0 46} { 18 bc6335a10528 2 46 0 46} { 19 bdef6ebc5d59 2 45 0 45} { 20 548b16b242a2 2 51 0 51} { 21 19729600a7ec 2 54 0 54} { 22 aed19015bf63 2 55 0 55} { 23 8ce99bc819fe 2 52 0 52} { 24 f7148c9a1e99 2 52 0 52} { 25 9977e650bbaf 2 61 0 61} { 26 6e09f874af9f 2 59 0 59} { 27 532bd9ad981 2 55 0 55} { 28 d78200a2233e 2 54 0 54} { 29 7671efc41428 2 49 0 49} { 30 3710cf482edd 2 46 0 46} { 31 42c966419faa 2 60 0 60} { 32 3d4415168479 2 67 0 67} { 33 fd3a2e79dd12 2 55 0 55} { 34 2f171d667a3d 2 60 0 60} { 35 1629a87e340b 2 50 0 50} { 36 27a62b9eb154 2 52 0 52} { 37 4c13cfe5f97a 2 59 0 59} { 38 a4dc9d93366b 2 53 0 53} { 39 5b8ca0741241 2 49 0 49} { 40 8b966a9c95e6 2 68 0 68} { 41 6a37568beafb 2 55 0 55} { 42 295cfbcef7d8 2 55 0 55} { 43 e34cfab95b41 2 66 0 66} { 44 979711e3cd2d 2 57 0 57} { 45 2d4a3697f1e2 2 62 0 62} { 46 70f44d845281 2 55 0 55} { 47 87891d8f3591 2 55 0 55} { 48 1187ad472c73 2 55 0 55} { 49 a7ca41192277 2 55 0 55} { 50 b5a6d67a3f61 2 66 0 66} { 51 b2f7e383f2a6 2 51 0 51} { 52 8fc524ed38d2 2 54 0 54} { 53 7e5c7cbc7650 2 61 0 61} { 54 169b44b000c5 2 60 0 60} { 55 3834abee7c05 2 67 0 67} { 56 1a70b76bc5d4 2 61 0 61} { 57 ae5e4819607a 2 63 0 63} { 58 5dfbdaa8ed5a 2 46 0 46} { 59 802fabead3f0 2 63 0 63} { 60 33804b202768 2 46 0 46} { 61 6d05d64a56f8 2 39 0 39} { 62 a113a925b34f 2 58 0 58} { 63 e5ae5cac421c 2 73 0 73} 2023-10-21T04:18:53.306+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 8606677778651310689 INDEX_STATE_ACTIVE 2023-10-21T04:18:53.306+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:18:53.306+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:53.306+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:53.306+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:53.306+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 79189 2023-10-21T04:18:53.306+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:53.307+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:53.307+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:18:53.307+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79189 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:53.307+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 8606677778651310689 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:18:53.307+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:18:53.307+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:18:53.307+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:53.307+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 79189 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:53.308+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:53.308+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:17 1:18 2:15 3:14 4:21 5:14 6:21 7:13 8:16 9:14 10:14 11:13 12:12 13:14 14:10 15:20 16:18 17:14 18:12 19:13 20:12 21:16 22:13 23:17 24:16 25:10 26:15 27:17 28:20 29:16 30:11 31:17 32:20 33:12 34:18 35:20 36:11 37:12 38:12 39:9 40:17 41:15 42:19 43:14 44:15 45:15 46:16 47:17 48:15 49:14 50:18 51:15 52:17 53:24 54:16 55:18 56:14 57:16 58:11 59:19 60:17 61:14 62:19 63:28] 2023-10-21T04:18:53.308+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:37 1:38 2:35 3:33 4:41 5:34 6:47 7:33 8:42 9:41 10:34 11:37 12:35 13:37 14:38 15:47 16:39 17:27 18:34 19:32 20:33 21:36 22:37 23:40 24:33 25:33 26:38 27:47 28:39 29:36 30:32 31:40 32:47 33:38 34:45 35:39 36:33 37:33 38:39 39:29 40:45 41:41 42:46 43:41 44:34 45:34 46:42 47:38 48:38 49:42 50:44 51:36 52:37 53:50 54:41 55:47 56:36 57:41 58:32 59:46 60:37 61:30 62:43 63:50] 2023-10-21T04:18:53.308+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:53.308+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:53954: use of closed network connection 2023-10-21T04:18:53.308+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:53954" exit: read tcp4 127.0.0.1:9121->127.0.0.1:53954: use of closed network connection 2023-10-21T04:18:53.308+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:53954" closed ! 2023-10-21T04:18:53.308+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:53.309+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:53.309+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:53.309+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:53.309+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:53.309+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:53.309+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:53.309+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 79189 2023-10-21T04:18:53.310+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:18:53.321+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.35892ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:53.347+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:53.347+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:53.353+05:30 [Info] switched currmeta from 574 -> 576 force true 2023-10-21T04:18:53.362+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:53.363+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:53.368+05:30 [Info] switched currmeta from 576 -> 576 force true 2023-10-21T04:18:53.391+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:18:53.392+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:53.393+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:18:53.393+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79189 2023-10-21T04:18:53.394+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 79189 2023-10-21T04:18:53.394+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:53.793+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8606677778651310689:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:54.273+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:18:54.273+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:18:54.274+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:18:54.274+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:18:54.274+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:18:54.274+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:18:54.275+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:18:54.306+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.504+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1408113278413491434:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:54.519+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 4 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 50 to 0, changed reclaimSize flush threshold from 10240 KB to 0 KB. 2023-10-21T04:18:54.987+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:18:54.993+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:54.993+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:54.996+05:30 [Info] switched currmeta from 576 -> 576 force true 2023-10-21T04:18:55.990+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:18:56.009+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:56.009+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:56.018+05:30 [Info] switched currmeta from 576 -> 576 force true 2023-10-21T04:18:56.207+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:56.207+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:56.210+05:30 [Info] switched currmeta from 576 -> 578 force true 2023-10-21T04:18:56.215+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:56.215+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:56.218+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [15425360657090425347] &{0} 2023-10-21T04:18:56.218+05:30 [Info] Indexer::handleBuildIndex [15425360657090425347] 2023-10-21T04:18:56.219+05:30 [Info] bulkUpdateRState: Index instance 15425360657090425347 rstate moved to ACTIVE 2023-10-21T04:18:56.219+05:30 [Info] Indexer::handleBuildIndex Added Index: [15425360657090425347] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:18:56.219+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:56.219+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:56.220+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:56.220+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:18:56.220+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:56.220+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:56.220+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 88713 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:56.220+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:18:56.220+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:18:56.220+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:56.220+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 88713 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:56.220+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:18:56.220+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:56.220+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:56.220+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c2% reset bucket state to NIL 2023-10-21T04:18:56.220+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:18:56.221+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:56.221+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:56.222+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c2% Added for Stream INIT_STREAM 2023-10-21T04:18:56.222+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:13 bucket_1:14] INIT_STREAM:map[bucket_%2:_default:c2%:1]] 2023-10-21T04:18:56.222+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15425360657090425347 stream INIT_STREAM keyspaceId bucket_%2:_default:c2% state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:18:56.222+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:18:56.222+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [0=52,1=57,2=50,3=44,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=56,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=65,41=55,42=55,43=64,44=56,45=58,46=54,47=55,48=55,49=54,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 88714 CollectionId: d CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:18:56.223+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:18:56.224+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:18:56.226+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:56.227+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 8.647636ms 2023-10-21T04:18:56.227+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 8.82963ms 2023-10-21T04:18:56.227+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [15425360657090425347] 2023-10-21T04:18:56.227+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_INDEX elapsed 21.784069ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:56.231+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c2% Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:18:56.233+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c2% Check pending stream merge. 2023-10-21T04:18:56.244+05:30 [Info] switched currmeta from 578 -> 579 force true 2023-10-21T04:18:56.256+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 28.782362ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:56.260+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:56.260+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:56.272+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:54068" doAuth() ... 2023-10-21T04:18:56.280+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:54068" auth successful 2023-10-21T04:18:56.280+05:30 [Info] switched currmeta from 579 -> 580 force true 2023-10-21T04:18:56.281+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:54068" +1 2023-10-21T04:18:56.287+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:56.287+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:56.294+05:30 [Info] switched currmeta from 580 -> 580 force true 2023-10-21T04:18:56.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 0 263268055970459 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 1 176639880977228 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.761+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 2 35403626585217 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.761+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 3 215749515705641 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.771+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 9 236398693961229 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 6 261493429196201 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 14 140236896721046 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 4 159113185399719 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 5 3874519919278 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 12 116104661063478 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 13 102350060311343 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 11 234758907181738 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 10 206533515052028 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 8 264975169139761 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 15 162158720181435 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 17 98820809009123 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 16 200752780087280 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 7 170288557775498 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.776+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 19 208836052671833 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.776+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 18 207134287529256 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.779+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 21 27979933591532 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.779+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 20 92956357968546 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.781+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 22 192215088742243 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.786+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 25 168740244208559 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.787+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 23 154934968850942 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.787+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 24 271667630317209 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 27 5715487545729 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 26 120989102157727 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.802+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 31 73432771436458 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.802+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 28 236953356346174 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.804+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 29 130231725986856 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.804+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 30 60545336618717 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 32 67362620867705 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 33 278426329668882 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 34 51776324008509 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 35 24368176313355 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.814+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 37 83647976044922 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.814+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 38 181267443431019 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.814+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 36 43594649874772 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.817+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 39 100659545510465 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 42 45479338375128 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 40 153478150002150 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 41 116785907755771 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 44 166675095997741 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 43 249919763471169 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 46 124194574848641 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 45 49796766757346 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 47 149022976193937 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 48 19274425379955 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 49 184487117398647 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 51 196777743741606 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 52 158076890855634 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 50 199728167534433 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 53 138935694816848 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 54 24856128127173 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.841+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 55 61798873988101 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 58 103336286678362 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 56 29071415952852 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 57 191719959781498 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.854+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 60 56626109228904 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.854+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 61 119871837460216 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.854+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 59 140942236111856 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.854+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 62 177105814270799 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c2% 63 252537041863196 0 88714 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:18:56.876+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 bucket_%2:_default:c2% InstanceIds [15425360657090425347] 2023-10-21T04:18:56.876+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% Cid d SessionId 88714 2023-10-21T04:18:56.876+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:56.876+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:56.876+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_%2:_default:c2%. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:18:56.876+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 38 0 53} { 1 a0a72d5bdb4c 3 42 0 63} { 2 20330c961c81 0 0 0 0} { 3 c439193e7d29 3 45 0 45} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 0 0 0 0} { 6 edd3aef555a9 0 0 0 0} { 7 9ae064feda8a 0 0 0 52} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 47 87891d8f3591 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 49 a7ca41192277 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 51 b2f7e383f2a6 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} { 53 7e5c7cbc7650 0 0 0 0} { 54 169b44b000c5 0 0 0 0} { 55 3834abee7c05 0 0 0 0} { 56 1a70b76bc5d4 0 0 0 0} { 57 ae5e4819607a 0 0 0 0} { 58 5dfbdaa8ed5a 0 0 0 0} { 59 802fabead3f0 0 0 0 0} { 60 33804b202768 0 0 0 0} { 61 6d05d64a56f8 0 0 0 0} { 62 a113a925b34f 0 0 0 0} { 63 e5ae5cac421c 0 0 0 0} HWT bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 39 0 53} { 1 a0a72d5bdb4c 3 42 0 63} { 2 20330c961c81 0 0 0 0} { 3 c439193e7d29 3 45 0 45} { 4 90b66ce513a7 0 0 0 0} { 5 3861b622aae 0 0 0 0} { 6 edd3aef555a9 0 0 0 0} { 7 9ae064feda8a 0 0 0 52} { 8 f0fe56d4b431 0 0 0 0} { 9 d700dc2dce0d 0 0 0 0} { 10 bbd754cc83fc 0 0 0 0} { 11 d58311453aaa 0 0 0 0} { 12 6998b9145736 0 0 0 0} { 13 5d163afcd32f 0 0 0 0} { 14 7f8b7269d896 0 0 0 0} { 15 937b84e9e4bb 0 0 0 0} { 16 b69566250ff0 0 0 0 0} { 17 59e08337a7e3 0 0 0 0} { 18 bc6335a10528 0 0 0 0} { 19 bdef6ebc5d59 0 0 0 0} { 20 548b16b242a2 0 0 0 0} { 21 19729600a7ec 0 0 0 0} { 22 aed19015bf63 0 0 0 0} { 23 8ce99bc819fe 0 0 0 0} { 24 f7148c9a1e99 0 0 0 0} { 25 9977e650bbaf 0 0 0 0} { 26 6e09f874af9f 0 0 0 0} { 27 532bd9ad981 0 0 0 0} { 28 d78200a2233e 0 0 0 0} { 29 7671efc41428 0 0 0 0} { 30 3710cf482edd 0 0 0 0} { 31 42c966419faa 0 0 0 0} { 32 3d4415168479 0 0 0 0} { 33 fd3a2e79dd12 0 0 0 0} { 34 2f171d667a3d 0 0 0 0} { 35 1629a87e340b 0 0 0 0} { 36 27a62b9eb154 0 0 0 0} { 37 4c13cfe5f97a 0 0 0 0} { 38 a4dc9d93366b 0 0 0 0} { 39 5b8ca0741241 0 0 0 0} { 40 8b966a9c95e6 0 0 0 0} { 41 6a37568beafb 0 0 0 0} { 42 295cfbcef7d8 0 0 0 0} { 43 e34cfab95b41 0 0 0 0} { 44 979711e3cd2d 0 0 0 0} { 45 2d4a3697f1e2 0 0 0 0} { 46 70f44d845281 0 0 0 0} { 47 87891d8f3591 0 0 0 0} { 48 1187ad472c73 0 0 0 0} { 49 a7ca41192277 0 0 0 0} { 50 b5a6d67a3f61 0 0 0 0} { 51 b2f7e383f2a6 0 0 0 0} { 52 8fc524ed38d2 0 0 0 0} { 53 7e5c7cbc7650 0 0 0 0} { 54 169b44b000c5 0 0 0 0} { 55 3834abee7c05 0 0 0 0} { 56 1a70b76bc5d4 0 0 0 0} { 57 ae5e4819607a 0 0 0 0} { 58 5dfbdaa8ed5a 0 0 0 0} { 59 802fabead3f0 0 0 0 0} { 60 33804b202768 0 0 0 0} { 61 6d05d64a56f8 0 0 0 0} { 62 a113a925b34f 0 0 0 0} { 63 e5ae5cac421c 0 0 0 0} 2023-10-21T04:18:56.876+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:56.876+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:56.877+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:18:56.877+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:18:56.877+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [0=52,1=57,2=50,3=44,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=56,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=65,41=55,42=55,43=64,44=56,45=58,46=54,47=55,48=55,49=54,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:57.333+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c2% setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:18:57.337+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15425360657090425347 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:18:57.337+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15425360657090425347 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.954µs SnapOpenDur 115.266µs 2023-10-21T04:18:57.337+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 15425360657090425347 INDEX_STATE_CATCHUP 2023-10-21T04:18:57.337+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 15425360657090425347 true 2023-10-21T04:18:57.337+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 15425360657090425347 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% Session: 88714 BuildTS: [0=52,1=57,2=50,3=44,4=58,5=54,6=60,7=52,8=56,9=62,10=50,11=56,12=55,13=53,14=56,15=67,16=51,17=44,18=46,19=45,20=50,21=54,22=55,23=51,24=52,25=59,26=59,27=54,28=54,29=49,30=46,31=60,32=67,33=54,34=60,35=50,36=52,37=59,38=53,39=49,40=65,41=55,42=55,43=64,44=56,45=58,46=54,47=55,48=55,49=54,50=66,51=51,52=54,53=61,54=60,55=67,56=61,57=63,58=46,59=63,60=46,61=39,62=58,63=73] 2023-10-21T04:18:57.337+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15425360657090425347, PartitionId 0 SnapshotId 1 Creating recovery point ... 2023-10-21T04:18:57.339+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c2% Stream: INIT_STREAM SessionId: 88714 2023-10-21T04:18:57.339+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c2% 88714 2023-10-21T04:18:57.339+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:57.343+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:57.344+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:57.344+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:57.344+05:30 [Info] send progress stats to clients 2023-10-21T04:18:57.344+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c2% set minMergeTs 2023-10-21T04:18:57.345+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c2% StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:18:57.346+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:14 bucket_1:14] INIT_STREAM:map[bucket_%2:_default:c2%:1]] 2023-10-21T04:18:57.346+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 88714 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:18:57.353+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 Instances [indexInstance:([`company`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:18:57.375+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:57.375+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:57.378+05:30 [Info] switched currmeta from 580 -> 581 force true 2023-10-21T04:18:57.381+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId bucket_%2 2023-10-21T04:18:57.381+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 88714 2023-10-21T04:18:57.381+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 88714 2023-10-21T04:18:57.381+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c2% set minMergeTs 2023-10-21T04:18:57.381+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:57.381+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c2% 15425360657090425347 false 2023-10-21T04:18:57.381+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c2% 15425360657090425347 2023-10-21T04:18:57.381+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c2%. 2023-10-21T04:18:57.381+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 88714 2023-10-21T04:18:57.382+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 37.380246ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:57.408+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:57.408+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:57.413+05:30 [Info] switched currmeta from 581 -> 582 force true 2023-10-21T04:18:57.431+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15425360657090425347, PartitionId 0 SnapshotId 1 Created recovery point (took 94.205901ms) 2023-10-21T04:18:57.520+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 6 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 0 to 0, changed reclaimSize flush threshold from 0 KB to 0 KB. 2023-10-21T04:18:57.886+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c2% StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:18:57.886+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:57.887+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 15425360657090425347 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% SessionId: 88714, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 3 53 0 53} { 1 a0a72d5bdb4c 3 63 0 63} { 2 20330c961c81 3 50 0 50} { 3 c439193e7d29 3 45 0 45} { 4 90b66ce513a7 3 58 0 58} { 5 3861b622aae 3 54 0 54} { 6 edd3aef555a9 3 61 0 61} { 7 9ae064feda8a 3 52 0 52} { 8 f0fe56d4b431 3 56 0 56} { 9 d700dc2dce0d 3 62 0 62} { 10 bbd754cc83fc 3 50 0 50} { 11 d58311453aaa 3 56 0 56} { 12 6998b9145736 3 55 0 55} { 13 5d163afcd32f 3 54 0 54} { 14 7f8b7269d896 3 60 0 60} { 15 937b84e9e4bb 3 67 0 67} { 16 b69566250ff0 3 55 0 55} { 17 59e08337a7e3 3 46 0 46} { 18 bc6335a10528 3 46 0 46} { 19 bdef6ebc5d59 3 45 0 45} { 20 548b16b242a2 3 51 0 51} { 21 19729600a7ec 3 54 0 54} { 22 aed19015bf63 3 55 0 55} { 23 8ce99bc819fe 3 52 0 52} { 24 f7148c9a1e99 3 52 0 52} { 25 9977e650bbaf 3 61 0 61} { 26 6e09f874af9f 3 59 0 59} { 27 532bd9ad981 3 55 0 55} { 28 d78200a2233e 3 54 0 54} { 29 7671efc41428 3 49 0 49} { 30 3710cf482edd 3 46 0 46} { 31 42c966419faa 3 60 0 60} { 32 3d4415168479 3 67 0 67} { 33 fd3a2e79dd12 3 55 0 55} { 34 2f171d667a3d 3 60 0 60} { 35 1629a87e340b 3 50 0 50} { 36 27a62b9eb154 3 52 0 52} { 37 4c13cfe5f97a 3 59 0 59} { 38 a4dc9d93366b 3 53 0 53} { 39 5b8ca0741241 3 49 0 49} { 40 8b966a9c95e6 3 68 0 68} { 41 6a37568beafb 3 55 0 55} { 42 295cfbcef7d8 3 55 0 55} { 43 e34cfab95b41 3 66 0 66} { 44 979711e3cd2d 3 57 0 57} { 45 2d4a3697f1e2 3 63 0 63} { 46 70f44d845281 3 55 0 55} { 47 87891d8f3591 3 55 0 55} { 48 1187ad472c73 3 55 0 55} { 49 a7ca41192277 3 55 0 55} { 50 b5a6d67a3f61 3 66 0 66} { 51 b2f7e383f2a6 3 51 0 51} { 52 8fc524ed38d2 3 54 0 54} { 53 7e5c7cbc7650 3 61 0 61} { 54 169b44b000c5 3 60 0 60} { 55 3834abee7c05 3 67 0 67} { 56 1a70b76bc5d4 3 61 0 61} { 57 ae5e4819607a 3 63 0 63} { 58 5dfbdaa8ed5a 3 46 0 46} { 59 802fabead3f0 3 63 0 63} { 60 33804b202768 3 46 0 46} { 61 6d05d64a56f8 3 39 0 39} { 62 a113a925b34f 3 58 0 58} { 63 e5ae5cac421c 3 73 0 73} 2023-10-21T04:18:57.887+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c2% 15425360657090425347 INDEX_STATE_ACTIVE 2023-10-21T04:18:57.887+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% State Changed to INACTIVE 2023-10-21T04:18:57.887+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:18:57.887+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:18:57.887+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 1 New 0 2023-10-21T04:18:57.888+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c2% Stream: INIT_STREAM SessionId: 88714 2023-10-21T04:18:57.888+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:57.889+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:18:57.889+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:18:57.889+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 88714 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:57.889+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15425360657090425347 from stream INIT_STREAM keyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:57.889+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c2% 2023-10-21T04:18:57.889+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c2% Deleted from Stream INIT_STREAM 2023-10-21T04:18:57.889+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:18:57.889+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c2% BuildTS: [] IndexList: [] Async: false SessionId: 88714 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:18:57.890+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:18:57.890+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c2% total received key-versions: map[0:14 1:15 2:14 3:10 4:16 5:19 6:11 7:18 8:13 9:20 10:15 11:18 12:19 13:15 14:17 15:19 16:10 17:16 18:11 19:12 20:16 21:17 22:17 23:10 24:18 25:25 26:20 27:6 28:14 29:12 30:13 31:19 32:19 33:15 34:14 35:10 36:17 37:25 38:13 39:19 40:19 41:13 42:8 43:21 44:21 45:20 46:11 47:16 48:16 49:10 50:21 51:14 52:16 53:10 54:18 55:19 56:24 57:21 58:13 59:16 60:8 61:8 62:14 63:22] 2023-10-21T04:18:57.890+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c2% latest sequence numbers: map[0:52 1:57 2:50 3:44 4:58 5:54 6:60 7:52 8:56 9:62 10:50 11:56 12:55 13:53 14:56 15:67 16:51 17:44 18:46 19:45 20:50 21:54 22:55 23:51 24:52 25:59 26:59 27:54 28:54 29:49 30:46 31:60 32:67 33:54 34:60 35:50 36:52 37:59 38:53 39:49 40:65 41:55 42:55 43:64 44:56 45:58 46:54 47:55 48:55 49:54 50:66 51:51 52:54 53:61 54:60 55:67 56:61 57:63 58:46 59:63 60:46 61:39 62:58 63:73] 2023-10-21T04:18:57.890+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:18:57.890+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:54068: use of closed network connection 2023-10-21T04:18:57.890+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:54068" exit: read tcp4 127.0.0.1:9121->127.0.0.1:54068: use of closed network connection 2023-10-21T04:18:57.890+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:54068" closed ! 2023-10-21T04:18:57.890+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:18:57.890+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:54068" already gone 2023-10-21T04:18:57.891+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:57.891+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:57.891+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:18:57.891+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:18:57.891+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:18:57.891+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:18:57.891+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c2% Stream: INIT_STREAM SessionId 88714 2023-10-21T04:18:57.893+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c2%] 2023-10-21T04:18:57.903+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.32044ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:18:57.910+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:57.910+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:57.915+05:30 [Info] switched currmeta from 582 -> 584 force true 2023-10-21T04:18:57.922+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:18:57.922+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:18:57.924+05:30 [Info] switched currmeta from 584 -> 584 force true 2023-10-21T04:18:57.941+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2:_default:c2%]. Err genServer.closed 2023-10-21T04:18:57.941+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c2%] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:18:57.941+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c2%] Treating genServer.closed As Success 2023-10-21T04:18:57.941+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 88714 2023-10-21T04:18:57.941+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% SessionId 88714 2023-10-21T04:18:57.941+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c2% 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.503+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2432386300762706992:[0]]. New rollbacktime: 1697841977132147563, Old rollbacktime: 0 2023-10-21T04:18:58.520+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 5 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 0 to 0, changed reclaimSize flush threshold from 0 KB to 0 KB. 2023-10-21T04:18:58.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.791+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:58.792+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15425360657090425347:[0]]. New rollbacktime: 1697841977675544947, Old rollbacktime: 0 2023-10-21T04:18:59.521+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 4 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 0 to 0, changed reclaimSize flush threshold from 0 KB to 0 KB. 2023-10-21T04:19:00.010+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:19:00.016+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:00.016+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:00.019+05:30 [Info] switched currmeta from 584 -> 584 force true 2023-10-21T04:19:01.520+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 4 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 0 to 0, changed reclaimSize flush threshold from 0 KB to 0 KB. 2023-10-21T04:19:02.520+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 4 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 0 to 0, changed reclaimSize flush threshold from 0 KB to 0 KB. 2023-10-21T04:19:02.889+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_1 Forcing Overdue Commit 2023-10-21T04:19:02.890+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7582778950442222621 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.890+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7582778950442222621 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.76µs SnapOpenDur 100.737µs 2023-10-21T04:19:02.892+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15667952562388529486 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.892+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15667952562388529486 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.272µs SnapOpenDur 77.859µs 2023-10-21T04:19:02.892+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4173530108555528677 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4173530108555528677 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.081µs SnapOpenDur 95.82µs 2023-10-21T04:19:02.893+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4173530108555528677, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 8 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.28µs SnapOpenDur 106.571µs 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.161µs SnapOpenDur 33.821µs 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 2 Creating New Snapshot SnapshotInfo: count:113 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:113 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.133µs SnapOpenDur 32.433µs 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.079µs SnapOpenDur 31.139µs 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 4 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:124 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.038µs SnapOpenDur 34.466µs 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 5 Creating New Snapshot SnapshotInfo: count:111 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:111 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.019µs SnapOpenDur 31.98µs 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 6 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:140 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.92µs SnapOpenDur 39.843µs 2023-10-21T04:19:02.893+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9912003609070511414 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:19:02.893+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9912003609070511414 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.18µs SnapOpenDur 45.951µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5004576769623010783 PartitionId 0 Creating New Snapshot SnapshotInfo: count:27 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5004576769623010783 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:27 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.844µs SnapOpenDur 47.861µs 2023-10-21T04:19:02.894+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0 SnapshotId 7 Creating recovery point ... 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 1 Creating New Snapshot SnapshotInfo: count:134 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:134 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.989µs SnapOpenDur 59.405µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 2 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:120 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.536µs SnapOpenDur 35.165µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 3 Creating New Snapshot SnapshotInfo: count:139 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:139 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.929µs SnapOpenDur 36.82µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 4 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.967µs SnapOpenDur 33.983µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 5 Creating New Snapshot SnapshotInfo: count:133 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:133 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.319µs SnapOpenDur 37.844µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 6 Creating New Snapshot SnapshotInfo: count:121 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:121 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.719µs SnapOpenDur 26.738µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 7 Creating New Snapshot SnapshotInfo: count:119 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:119 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.022µs SnapOpenDur 31.946µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4440544141961093904 PartitionId 8 Creating New Snapshot SnapshotInfo: count:122 committed:true 2023-10-21T04:19:02.894+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4440544141961093904 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:122 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.786µs SnapOpenDur 32.066µs 2023-10-21T04:19:02.894+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14343533858105260407 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.895+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14343533858105260407 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.988µs SnapOpenDur 92.478µs 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14343533858105260407, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3100906924534408756 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.895+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3100906924534408756 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.867µs SnapOpenDur 59.638µs 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.899+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.899+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.900+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 6261854707981462523 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.900+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 6261854707981462523 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.253µs SnapOpenDur 465.091µs 2023-10-21T04:19:02.902+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:02.903+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.903+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 9112972261750078043 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.904+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 9112972261750078043 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.079µs SnapOpenDur 918.467µs 2023-10-21T04:19:02.904+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9112972261750078043, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.904+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 962468716067699903 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.904+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 962468716067699903 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.724µs SnapOpenDur 59.869µs 2023-10-21T04:19:02.905+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.905+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 6 Creating New Snapshot SnapshotInfo: count:115 committed:true 2023-10-21T04:19:02.905+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 6 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:115 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.208µs SnapOpenDur 193.042µs 2023-10-21T04:19:02.905+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 7 Creating New Snapshot SnapshotInfo: count:109 committed:true 2023-10-21T04:19:02.905+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 7 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:109 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.7µs SnapOpenDur 121.498µs 2023-10-21T04:19:02.905+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 8 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:19:02.905+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 8 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.224µs SnapOpenDur 38.032µs 2023-10-21T04:19:02.906+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 1 Creating New Snapshot SnapshotInfo: count:135 committed:true 2023-10-21T04:19:02.906+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 1 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:135 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.022µs SnapOpenDur 341.893µs 2023-10-21T04:19:02.906+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 2 Creating New Snapshot SnapshotInfo: count:107 committed:true 2023-10-21T04:19:02.906+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 2 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:107 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 55.461µs SnapOpenDur 194.597µs 2023-10-21T04:19:02.906+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 3 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:19:02.907+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 3 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 59.698µs SnapOpenDur 189.254µs 2023-10-21T04:19:02.907+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 4 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:19:02.907+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 4 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:137 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.272µs SnapOpenDur 187.786µs 2023-10-21T04:19:02.907+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 5744899830895973442 PartitionId 5 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:19:02.907+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 5744899830895973442 PartitionId: 5 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:131 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.201µs SnapOpenDur 194.882µs 2023-10-21T04:19:02.908+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7494249569853685720 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:02.908+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7494249569853685720 PartitionId: 0 SliceId: 0 Crc64: 2946622789168712077 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.027µs SnapOpenDur 228.816µs 2023-10-21T04:19:02.908+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.908+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.909+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.909+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.909+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.909+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.910+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.910+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.910+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.910+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0 SnapshotId 3 Created recovery point (took 7.023144ms) 2023-10-21T04:19:02.911+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:02.912+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:19:02.931+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0 SnapshotId 7 Created recovery point (took 37.553496ms) 2023-10-21T04:19:02.933+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:19:03.047+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 SnapshotId 3 Created recovery point (took 151.853429ms) 2023-10-21T04:19:03.048+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:19:03.049+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:19:03.082+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 SnapshotId 3 Created recovery point (took 186.831211ms) 2023-10-21T04:19:03.083+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:19:03.083+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:19:03.187+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 SnapshotId 3 Created recovery point (took 291.070686ms) 2023-10-21T04:19:03.188+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:19:03.188+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:19:03.281+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 SnapshotId 3 Created recovery point (took 384.87431ms) 2023-10-21T04:19:03.282+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:19:03.283+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:19:03.360+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4173530108555528677, PartitionId 0 SnapshotId 4 Created recovery point (took 467.207983ms) 2023-10-21T04:19:03.493+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0 SnapshotId 3 Created recovery point (took 583.112893ms) 2023-10-21T04:19:03.494+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.520+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 5 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 0 to 0, changed reclaimSize flush threshold from 0 KB to 0 KB. 2023-10-21T04:19:03.526+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0 SnapshotId 3 Created recovery point (took 614.122477ms) 2023-10-21T04:19:03.527+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.547+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9112972261750078043, PartitionId 0 SnapshotId 3 Created recovery point (took 643.108915ms) 2023-10-21T04:19:03.563+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14343533858105260407, PartitionId 0 SnapshotId 2 Created recovery point (took 668.54238ms) 2023-10-21T04:19:03.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 SnapshotId 3 Created recovery point (took 680.299325ms) 2023-10-21T04:19:03.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:19:03.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:121 committed:false. num RPs 3. 2023-10-21T04:19:03.591+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 SnapshotId 3 Created recovery point (took 696.072011ms) 2023-10-21T04:19:03.592+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:19:03.592+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:19:03.603+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 SnapshotId 3 Created recovery point (took 707.618849ms) 2023-10-21T04:19:03.603+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:19:03.604+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:113 committed:false. num RPs 3. 2023-10-21T04:19:03.628+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 SnapshotId 3 Created recovery point (took 732.437658ms) 2023-10-21T04:19:03.629+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:19:03.629+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:111 committed:false. num RPs 3. 2023-10-21T04:19:03.647+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 SnapshotId 3 Created recovery point (took 751.908337ms) 2023-10-21T04:19:03.648+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:19:03.649+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:19:03.668+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 SnapshotId 3 Created recovery point (took 772.029035ms) 2023-10-21T04:19:03.669+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:19:03.669+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:19:03.692+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 SnapshotId 3 Created recovery point (took 796.299391ms) 2023-10-21T04:19:03.693+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:19:03.693+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:134 committed:false. num RPs 3. 2023-10-21T04:19:03.714+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 SnapshotId 3 Created recovery point (took 817.889137ms) 2023-10-21T04:19:03.715+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:19:03.715+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:139 committed:false. num RPs 3. 2023-10-21T04:19:03.732+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 SnapshotId 3 Created recovery point (took 836.139747ms) 2023-10-21T04:19:03.733+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:19:03.734+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:19:03.755+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 SnapshotId 3 Created recovery point (took 858.581218ms) 2023-10-21T04:19:03.755+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:19:03.756+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:133 committed:false. num RPs 3. 2023-10-21T04:19:03.774+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 SnapshotId 3 Created recovery point (took 878.238502ms) 2023-10-21T04:19:03.776+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:19:03.776+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:119 committed:false. num RPs 3. 2023-10-21T04:19:03.791+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 SnapshotId 3 Created recovery point (took 891.658368ms) 2023-10-21T04:19:03.792+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:19:03.792+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:19:03.801+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 SnapshotId 3 Created recovery point (took 901.967401ms) 2023-10-21T04:19:03.803+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.803+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.818+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 SnapshotId 4 Created recovery point (took 915.381862ms) 2023-10-21T04:19:03.819+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.819+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.837+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 SnapshotId 3 Created recovery point (took 932.590255ms) 2023-10-21T04:19:03.838+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.839+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 962468716067699903, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:03.854+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 SnapshotId 3 Created recovery point (took 945.886285ms) 2023-10-21T04:19:03.855+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:19:03.856+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:115 committed:false. num RPs 3. 2023-10-21T04:19:03.879+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 SnapshotId 3 Created recovery point (took 970.772275ms) 2023-10-21T04:19:03.880+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:19:03.881+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:109 committed:false. num RPs 3. 2023-10-21T04:19:03.895+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 SnapshotId 3 Created recovery point (took 986.460083ms) 2023-10-21T04:19:03.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:19:03.896+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:19:03.908+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 SnapshotId 3 Created recovery point (took 998.611133ms) 2023-10-21T04:19:03.909+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:19:03.909+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:135 committed:false. num RPs 3. 2023-10-21T04:19:03.921+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 SnapshotId 3 Created recovery point (took 1.011564493s) 2023-10-21T04:19:03.922+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:19:03.922+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:107 committed:false. num RPs 3. 2023-10-21T04:19:03.935+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 SnapshotId 3 Created recovery point (took 1.02524372s) 2023-10-21T04:19:03.936+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:19:03.936+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:19:03.951+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 SnapshotId 3 Created recovery point (took 1.041425521s) 2023-10-21T04:19:03.952+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:19:03.953+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:19:03.972+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 SnapshotId 3 Created recovery point (took 1.062107084s) 2023-10-21T04:19:03.973+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:19:03.973+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:19:04.520+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: SMR reclaim pending is higher than expected: pending = 298 KB (expected = 7 KB), wCtxCnt = 5, objCnt 40, changed reclaimList flush threshold from 0 to 0, changed reclaimSize flush threshold from 0 KB to 1 KB. 2023-10-21T04:19:05.509+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:57962 closed 2023-10-21T04:19:05.563+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60872 closed 2023-10-21T04:19:05.624+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60880 closed 2023-10-21T04:19:05.694+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60890 closed 2023-10-21T04:19:05.711+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60896 closed 2023-10-21T04:19:05.717+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60898 closed 2023-10-21T04:19:05.769+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60904 closed 2023-10-21T04:19:05.798+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60908 closed 2023-10-21T04:19:05.858+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60916 closed 2023-10-21T04:19:05.983+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60918 closed 2023-10-21T04:19:06.012+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60934 closed 2023-10-21T04:19:06.130+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60944 closed 2023-10-21T04:19:06.194+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60948 closed 2023-10-21T04:19:06.257+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60962 closed 2023-10-21T04:19:06.259+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60968 closed 2023-10-21T04:19:06.405+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60976 closed 2023-10-21T04:19:06.427+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60984 closed 2023-10-21T04:19:06.433+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60988 closed 2023-10-21T04:19:06.486+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:60994 closed 2023-10-21T04:19:06.542+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32812 closed 2023-10-21T04:19:06.583+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32818 closed 2023-10-21T04:19:06.604+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32824 closed 2023-10-21T04:19:06.621+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32828 closed 2023-10-21T04:19:06.646+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32830 closed 2023-10-21T04:19:06.811+05:30 [Warn] builder: Index (6985049432225870291, bucket_1) has no instances in READY state. Skipping. 2023-10-21T04:19:06.830+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32848 closed 2023-10-21T04:19:06.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32858 closed 2023-10-21T04:19:06.889+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32862 closed 2023-10-21T04:19:06.898+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32864 closed 2023-10-21T04:19:06.959+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32868 closed 2023-10-21T04:19:07.028+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32874 closed 2023-10-21T04:19:07.119+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32876 closed 2023-10-21T04:19:07.184+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32888 closed 2023-10-21T04:19:07.187+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32898 closed 2023-10-21T04:19:07.251+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32904 closed 2023-10-21T04:19:07.261+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32914 closed 2023-10-21T04:19:07.318+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32922 closed 2023-10-21T04:19:07.336+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32924 closed 2023-10-21T04:19:07.406+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32926 closed 2023-10-21T04:19:07.444+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32932 closed 2023-10-21T04:19:07.460+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32934 closed 2023-10-21T04:19:07.484+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32942 closed 2023-10-21T04:19:07.488+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32944 closed 2023-10-21T04:19:07.533+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32946 closed 2023-10-21T04:19:07.538+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32954 closed 2023-10-21T04:19:07.571+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32956 closed 2023-10-21T04:19:07.612+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32962 closed 2023-10-21T04:19:07.640+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32966 closed 2023-10-21T04:19:07.695+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32970 closed 2023-10-21T04:19:07.732+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32974 closed 2023-10-21T04:19:07.746+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:32982 closed 2023-10-21T04:19:12.013+05:30 [Info] memstats {"Alloc":308896288, "TotalAlloc":3741991792, "Sys":384422232, "Lookups":0, "Mallocs":43172185,"Frees":41705199, "HeapAlloc":308896288, "HeapSys":345669632, "HeapIdle":28516352, "HeapInuse":317153280,"HeapReleased":1572864, "HeapObjects":1466986,"MSpanInuse":2711680, "MSpanSys": 3100800, "StackInuse": 15040512,"GCSys":17376528, "LastGC":1697842088379834470,"PauseTotalNs":50440555, "PauseNs":[2166023, 161078, 178894, 124421, 152129, 183557, 85035, 90428, 240327, 83963, 161551, 398545, 222920, 196154], "NumGC":30} 2023-10-21T04:19:12.024+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:19:12.179+05:30 [Info] janitor: running cleanup. 2023-10-21T04:19:12.179+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[5465835793930010953 12195715103909237684 2048006213723315482 249887442666643345 14889401051926000176 4138047918192025493 4535197604682045916 4860549750398470299]} 2023-10-21T04:19:12.324+05:30 [Info] connected with 2 indexers 2023-10-21T04:19:12.325+05:30 [Info] client stats current counts: current: 76, not current: 0 2023-10-21T04:19:12.437+05:30 [Info] num concurrent scans {0} 2023-10-21T04:19:12.437+05:30 [Info] average scan response {0 ms} 2023-10-21T04:19:12.762+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:19:12.787+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:19:14.087+05:30 [Info] Indexer::ReadMemstats Time Taken 98.85µs 2023-10-21T04:19:14.846+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33010 closed 2023-10-21T04:19:14.866+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33250 closed 2023-10-21T04:19:15.148+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33254 closed 2023-10-21T04:19:15.247+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33312 closed 2023-10-21T04:19:15.286+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33320 closed 2023-10-21T04:19:15.321+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33324 closed 2023-10-21T04:19:15.343+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33326 closed 2023-10-21T04:19:15.380+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33328 closed 2023-10-21T04:19:15.437+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33332 closed 2023-10-21T04:19:15.512+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33334 closed 2023-10-21T04:19:15.560+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33340 closed 2023-10-21T04:19:15.566+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33346 closed 2023-10-21T04:19:15.628+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33352 closed 2023-10-21T04:19:15.648+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33354 closed 2023-10-21T04:19:15.748+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33368 closed 2023-10-21T04:19:15.755+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33372 closed 2023-10-21T04:19:15.914+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33374 closed 2023-10-21T04:19:15.921+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33388 closed 2023-10-21T04:19:15.924+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33390 closed 2023-10-21T04:19:15.975+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33392 closed 2023-10-21T04:19:16.016+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33400 closed 2023-10-21T04:19:16.056+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33408 closed 2023-10-21T04:19:16.074+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33410 closed 2023-10-21T04:19:16.134+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33412 closed 2023-10-21T04:19:16.154+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33426 closed 2023-10-21T04:19:16.158+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33430 closed 2023-10-21T04:19:16.184+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33436 closed 2023-10-21T04:19:16.305+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33442 closed 2023-10-21T04:19:16.306+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33446 closed 2023-10-21T04:19:16.309+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33448 closed 2023-10-21T04:19:16.327+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33450 closed 2023-10-21T04:19:16.384+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33464 closed 2023-10-21T04:19:16.385+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33458 closed 2023-10-21T04:19:16.398+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33468 closed 2023-10-21T04:19:16.403+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33470 closed 2023-10-21T04:19:16.406+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33472 closed 2023-10-21T04:19:16.440+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33480 closed 2023-10-21T04:19:16.467+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33482 closed 2023-10-21T04:19:16.481+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33484 closed 2023-10-21T04:19:16.570+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33490 closed 2023-10-21T04:19:16.578+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33500 closed 2023-10-21T04:19:16.594+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33502 closed 2023-10-21T04:19:16.610+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33506 closed 2023-10-21T04:19:16.646+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33510 closed 2023-10-21T04:19:16.663+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33518 closed 2023-10-21T04:19:16.693+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33522 closed 2023-10-21T04:19:16.745+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33528 closed 2023-10-21T04:19:16.772+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33542 closed 2023-10-21T04:19:16.805+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33544 closed 2023-10-21T04:19:18.425+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3100906924534408756 &{0} 2023-10-21T04:19:18.425+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3100906924534408756 2023-10-21T04:19:18.426+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:18.426+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.426+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3100906924534408756 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:18.427+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3100906924534408756] 2023-10-21T04:19:18.427+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3100906924534408756] 2023-10-21T04:19:18.427+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.427+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3100906924534408756] 2023-10-21T04:19:18.428+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0, IndexDefnId 14292583894719865678. 2023-10-21T04:19:18.428+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.428+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.428+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex] 2023-10-21T04:19:18.428+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex 2023-10-21T04:19:18.428+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex] 2023-10-21T04:19:18.428+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.428+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex] 2023-10-21T04:19:18.428+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex 2023-10-21T04:19:18.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex stopped 2023-10-21T04:19:18.428+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.428+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.428+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.428+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.429+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 3100906924534408756 Defn: DefnId: 14292583894719865678 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.430+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.594283ms 2023-10-21T04:19:18.430+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3100906924534408756 2023-10-21T04:19:18.430+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [3100906924534408756] 2023-10-21T04:19:18.435+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:18.435+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:19:18.436+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 5.676505ms 2023-10-21T04:19:18.436+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 10.669651ms 2023-10-21T04:19:18.441+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex] 2023-10-21T04:19:18.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex closed 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex] 2023-10-21T04:19:18.442+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex 2023-10-21T04:19:18.442+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex] 2023-10-21T04:19:18.442+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex] 2023-10-21T04:19:18.442+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex 2023-10-21T04:19:18.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex stopped 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.442+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.456+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 50.619398ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:18.472+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.472+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.473+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex] 2023-10-21T04:19:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex closed 2023-10-21T04:19:18.473+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.473+05:30 [Info] Indexer::cleanupIndexData IndexInst 3100906924534408756 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex ... 2023-10-21T04:19:18.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 1 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex ... 2023-10-21T04:19:18.477+05:30 [Info] switched currmeta from 584 -> 587 force true 2023-10-21T04:19:18.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:19:18.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:18.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex ... 2023-10-21T04:19:18.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 1 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex ... 2023-10-21T04:19:18.489+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.489+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:19:18.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3100906924534408756_0.index destroyed 2023-10-21T04:19:18.492+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3100906924534408756, PartitionId 0, IndexDefnId 14292583894719865678. 2023-10-21T04:19:18.492+05:30 [Info] Indexer::cleanupIndexData IndexInst 3100906924534408756 Partition 0 Destroy Done 2023-10-21T04:19:18.494+05:30 [Info] switched currmeta from 587 -> 587 force true 2023-10-21T04:19:18.540+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.540+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.543+05:30 [Info] switched currmeta from 587 -> 588 force true 2023-10-21T04:19:18.548+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.548+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.551+05:30 [Info] switched currmeta from 588 -> 588 force true 2023-10-21T04:19:18.598+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 14343533858105260407 &{0} 2023-10-21T04:19:18.598+05:30 [Info] Indexer::handleDropIndex - IndexInstId 14343533858105260407 2023-10-21T04:19:18.599+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:18.599+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.599+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14343533858105260407 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:18.600+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [14343533858105260407] 2023-10-21T04:19:18.611+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [14343533858105260407] 2023-10-21T04:19:18.611+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.611+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [14343533858105260407] 2023-10-21T04:19:18.612+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14343533858105260407, PartitionId 0, IndexDefnId 15945117906390072636. 2023-10-21T04:19:18.611+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.617+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.621+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.621+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14343533858105260407 Defn: DefnId: 15945117906390072636 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.622+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 23.217577ms 2023-10-21T04:19:18.622+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 14343533858105260407 2023-10-21T04:19:18.622+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [14343533858105260407] 2023-10-21T04:19:18.626+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:18.626+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:19:18.626+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.852681ms 2023-10-21T04:19:18.626+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 28.375401ms 2023-10-21T04:19:18.632+05:30 [Info] switched currmeta from 588 -> 589 force true 2023-10-21T04:19:18.635+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 46.960729ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:18.639+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.639+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.639+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:19:18.639+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:19:18.639+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:19:18.639+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:19:18.639+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex 2023-10-21T04:19:18.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex stopped 2023-10-21T04:19:18.639+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.639+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.639+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.639+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.641+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.642+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.649+05:30 [Info] switched currmeta from 589 -> 590 force true 2023-10-21T04:19:18.655+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex] 2023-10-21T04:19:18.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex closed 2023-10-21T04:19:18.657+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#14343533858105260407:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.673+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.673+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.673+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:19:18.674+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:19:18.660+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.674+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.674+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:19:18.674+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:19:18.674+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex 2023-10-21T04:19:18.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex stopped 2023-10-21T04:19:18.674+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.674+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.674+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.674+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.688+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex] 2023-10-21T04:19:18.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex closed 2023-10-21T04:19:18.689+05:30 [Info] switched currmeta from 590 -> 591 force true 2023-10-21T04:19:18.690+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#14343533858105260407:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.690+05:30 [Info] Indexer::cleanupIndexData IndexInst 14343533858105260407 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:18.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex ... 2023-10-21T04:19:18.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 147 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex ... 2023-10-21T04:19:18.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.700+05:30 [Info] switched currmeta from 591 -> 591 force true 2023-10-21T04:19:18.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:19:18.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:18.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex ... 2023-10-21T04:19:18.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 163 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex ... 2023-10-21T04:19:18.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:19:18.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_14343533858105260407_0.index destroyed 2023-10-21T04:19:18.717+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14343533858105260407, PartitionId 0, IndexDefnId 15945117906390072636. 2023-10-21T04:19:18.717+05:30 [Info] Indexer::cleanupIndexData IndexInst 14343533858105260407 Partition 0 Destroy Done 2023-10-21T04:19:18.729+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.729+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.731+05:30 [Info] switched currmeta from 591 -> 592 force true 2023-10-21T04:19:18.769+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 6261854707981462523 &{0} 2023-10-21T04:19:18.770+05:30 [Info] Indexer::handleDropIndex - IndexInstId 6261854707981462523 2023-10-21T04:19:18.771+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:19:18.772+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.773+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 6261854707981462523 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:19:18.778+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [6261854707981462523] 2023-10-21T04:19:18.780+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [6261854707981462523] 2023-10-21T04:19:18.781+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.781+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [6261854707981462523] 2023-10-21T04:19:18.782+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0, IndexDefnId 8504686614261377811. 2023-10-21T04:19:18.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex] 2023-10-21T04:19:18.782+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex 2023-10-21T04:19:18.782+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex] 2023-10-21T04:19:18.782+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex] 2023-10-21T04:19:18.782+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex 2023-10-21T04:19:18.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex stopped 2023-10-21T04:19:18.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.782+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.783+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.786+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 6261854707981462523 Defn: DefnId: 8504686614261377811 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.786+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 16.570825ms 2023-10-21T04:19:18.787+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex] 2023-10-21T04:19:18.787+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 6261854707981462523 2023-10-21T04:19:18.787+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [6261854707981462523] 2023-10-21T04:19:18.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex closed 2023-10-21T04:19:18.794+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:18.796+05:30 [Info] bucket_1/idx_secondary/Mainstore#6261854707981462523:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.838+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.838+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.838+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex] 2023-10-21T04:19:18.839+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex 2023-10-21T04:19:18.819+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 54.755511ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:18.825+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.839+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.839+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex] 2023-10-21T04:19:18.845+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex] 2023-10-21T04:19:18.845+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex 2023-10-21T04:19:18.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex stopped 2023-10-21T04:19:18.846+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.847+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.847+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.847+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.850+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:19:18.851+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 64.350334ms 2023-10-21T04:19:18.851+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 81.50838ms 2023-10-21T04:19:18.852+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex] 2023-10-21T04:19:18.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex closed 2023-10-21T04:19:18.858+05:30 [Info] switched currmeta from 592 -> 596 force true 2023-10-21T04:19:18.859+05:30 [Info] bucket_1/idx_secondary/Backstore#6261854707981462523:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.860+05:30 [Info] Indexer::cleanupIndexData IndexInst 6261854707981462523 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:18.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex ... 2023-10-21T04:19:18.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 166 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex ... 2023-10-21T04:19:18.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:19:18.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:18.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex ... 2023-10-21T04:19:18.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 185 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex ... 2023-10-21T04:19:18.869+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:18.869+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:18.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:19:18.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_6261854707981462523_0.index destroyed 2023-10-21T04:19:18.873+05:30 [Info] switched currmeta from 596 -> 596 force true 2023-10-21T04:19:18.875+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 6261854707981462523, PartitionId 0, IndexDefnId 8504686614261377811. 2023-10-21T04:19:18.875+05:30 [Info] Indexer::cleanupIndexData IndexInst 6261854707981462523 Partition 0 Destroy Done 2023-10-21T04:19:18.974+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 9112972261750078043 &{0} 2023-10-21T04:19:18.974+05:30 [Info] Indexer::handleDropIndex - IndexInstId 9112972261750078043 2023-10-21T04:19:18.975+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:18.975+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.975+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9112972261750078043 Name: idx_secondary_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:18.976+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [9112972261750078043] 2023-10-21T04:19:18.976+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [9112972261750078043] 2023-10-21T04:19:18.976+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:18.977+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [9112972261750078043] 2023-10-21T04:19:18.977+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.979+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9112972261750078043, PartitionId 0, IndexDefnId 18323460706027642575. 2023-10-21T04:19:18.979+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.979+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.979+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:19:18.979+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:19:18.979+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:19:18.979+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:19:18.979+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex 2023-10-21T04:19:18.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex stopped 2023-10-21T04:19:18.979+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.979+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.979+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.979+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.982+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex] 2023-10-21T04:19:18.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex closed 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#9112972261750078043:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:18.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:19:18.983+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:19:18.983+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:19:18.983+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:19:18.983+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex 2023-10-21T04:19:18.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex stopped 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:18.983+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:18.977+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9112972261750078043 Defn: DefnId: 18323460706027642575 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:18.986+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 11.326836ms 2023-10-21T04:19:18.986+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [9112972261750078043] 2023-10-21T04:19:18.986+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 9112972261750078043 2023-10-21T04:19:18.989+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:18.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex] 2023-10-21T04:19:18.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex closed 2023-10-21T04:19:18.989+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#9112972261750078043:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:18.989+05:30 [Info] Indexer::cleanupIndexData IndexInst 9112972261750078043 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:18.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex ... 2023-10-21T04:19:18.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 148 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex ... 2023-10-21T04:19:18.993+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:18.993+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:19:18.993+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 7.513132ms 2023-10-21T04:19:18.993+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 19.221455ms 2023-10-21T04:19:18.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:19:18.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:18.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex ... 2023-10-21T04:19:18.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 165 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex ... 2023-10-21T04:19:19.012+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.012+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.015+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 54.291093ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:19.018+05:30 [Info] switched currmeta from 596 -> 598 force true 2023-10-21T04:19:19.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_9112972261750078043_0.index destroyed 2023-10-21T04:19:19.024+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9112972261750078043, PartitionId 0, IndexDefnId 18323460706027642575. 2023-10-21T04:19:19.024+05:30 [Info] Indexer::cleanupIndexData IndexInst 9112972261750078043 Partition 0 Destroy Done 2023-10-21T04:19:19.048+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.048+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.051+05:30 [Info] switched currmeta from 598 -> 599 force true 2023-10-21T04:19:19.056+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.056+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.060+05:30 [Info] switched currmeta from 599 -> 599 force true 2023-10-21T04:19:19.078+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.078+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.083+05:30 [Info] switched currmeta from 599 -> 600 force true 2023-10-21T04:19:19.253+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 962468716067699903 &{0} 2023-10-21T04:19:19.253+05:30 [Info] Indexer::handleDropIndex - IndexInstId 962468716067699903 2023-10-21T04:19:19.253+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:19.254+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.254+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 962468716067699903 Name: idx_secondary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:19.255+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [962468716067699903] 2023-10-21T04:19:19.255+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [962468716067699903] 2023-10-21T04:19:19.255+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.255+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [962468716067699903] 2023-10-21T04:19:19.255+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.256+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 962468716067699903 Defn: DefnId: 12140458060295645238 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.256+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.910107ms 2023-10-21T04:19:19.257+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 962468716067699903, PartitionId 0, IndexDefnId 12140458060295645238. 2023-10-21T04:19:19.257+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.257+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.257+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex] 2023-10-21T04:19:19.257+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex 2023-10-21T04:19:19.257+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 962468716067699903 2023-10-21T04:19:19.257+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex] 2023-10-21T04:19:19.257+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex] 2023-10-21T04:19:19.257+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex 2023-10-21T04:19:19.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex stopped 2023-10-21T04:19:19.257+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.257+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.257+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.257+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.257+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [962468716067699903] 2023-10-21T04:19:19.264+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:19.264+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:19:19.264+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 7.819317ms 2023-10-21T04:19:19.264+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 10.808073ms 2023-10-21T04:19:19.264+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex] 2023-10-21T04:19:19.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex closed 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex] 2023-10-21T04:19:19.265+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex 2023-10-21T04:19:19.265+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex] 2023-10-21T04:19:19.265+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex] 2023-10-21T04:19:19.265+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex 2023-10-21T04:19:19.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex stopped 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.270+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex] 2023-10-21T04:19:19.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex closed 2023-10-21T04:19:19.270+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.270+05:30 [Info] Indexer::cleanupIndexData IndexInst 962468716067699903 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:19.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex ... 2023-10-21T04:19:19.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex ... 2023-10-21T04:19:19.273+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 24.902239ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:19.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:19.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex ... 2023-10-21T04:19:19.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex ... 2023-10-21T04:19:19.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_962468716067699903_0.index destroyed 2023-10-21T04:19:19.281+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 962468716067699903, PartitionId 0, IndexDefnId 12140458060295645238. 2023-10-21T04:19:19.281+05:30 [Info] Indexer::cleanupIndexData IndexInst 962468716067699903 Partition 0 Destroy Done 2023-10-21T04:19:19.282+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.282+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.287+05:30 [Info] switched currmeta from 600 -> 603 force true 2023-10-21T04:19:19.300+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.300+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.310+05:30 [Info] switched currmeta from 603 -> 603 force true 2023-10-21T04:19:19.356+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.356+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.358+05:30 [Info] switched currmeta from 603 -> 604 force true 2023-10-21T04:19:19.365+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.365+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.368+05:30 [Info] switched currmeta from 604 -> 604 force true 2023-10-21T04:19:19.419+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.419+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.424+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4173530108555528677 &{0} 2023-10-21T04:19:19.429+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4173530108555528677 2023-10-21T04:19:19.431+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:19:19.432+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.433+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4173530108555528677 Name: idx_secondary_defer Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:19:19.435+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4173530108555528677] 2023-10-21T04:19:19.436+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4173530108555528677] 2023-10-21T04:19:19.436+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.437+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4173530108555528677] 2023-10-21T04:19:19.438+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4173530108555528677, PartitionId 0, IndexDefnId 6985049432225870291. 2023-10-21T04:19:19.438+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.438+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.438+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:19:19.438+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:19:19.438+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:19:19.438+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:19:19.438+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex 2023-10-21T04:19:19.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex stopped 2023-10-21T04:19:19.438+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.438+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.438+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.438+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.438+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.440+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4173530108555528677 Defn: DefnId: 6985049432225870291 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.440+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 15.649626ms 2023-10-21T04:19:19.440+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4173530108555528677 2023-10-21T04:19:19.441+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [4173530108555528677] 2023-10-21T04:19:19.445+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex] 2023-10-21T04:19:19.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex closed 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#4173530108555528677:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:19:19.446+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:19:19.446+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:19:19.446+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:19:19.446+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex 2023-10-21T04:19:19.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex stopped 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.446+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.447+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:19.447+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:19:19.447+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.739632ms 2023-10-21T04:19:19.447+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 22.453282ms 2023-10-21T04:19:19.453+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex] 2023-10-21T04:19:19.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex closed 2023-10-21T04:19:19.454+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#4173530108555528677:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.454+05:30 [Info] Indexer::cleanupIndexData IndexInst 4173530108555528677 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:19.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex ... 2023-10-21T04:19:19.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 165 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex ... 2023-10-21T04:19:19.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:19.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex ... 2023-10-21T04:19:19.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 183 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex ... 2023-10-21T04:19:19.458+05:30 [Info] switched currmeta from 604 -> 607 force true 2023-10-21T04:19:19.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_4173530108555528677_0.index destroyed 2023-10-21T04:19:19.460+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4173530108555528677, PartitionId 0, IndexDefnId 6985049432225870291. 2023-10-21T04:19:19.460+05:30 [Info] Indexer::cleanupIndexData IndexInst 4173530108555528677 Partition 0 Destroy Done 2023-10-21T04:19:19.471+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 53.456013ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:19.496+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.496+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.502+05:30 [Info] switched currmeta from 607 -> 608 force true 2023-10-21T04:19:19.523+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.523+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.526+05:30 [Info] switched currmeta from 608 -> 608 force true 2023-10-21T04:19:19.576+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 9720186692804226822 &{0} 2023-10-21T04:19:19.577+05:30 [Info] Indexer::handleDropIndex - IndexInstId 9720186692804226822 2023-10-21T04:19:19.577+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:19.581+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.582+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9720186692804226822 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:19.583+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [9720186692804226822] 2023-10-21T04:19:19.583+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [9720186692804226822] 2023-10-21T04:19:19.583+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.583+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [9720186692804226822] 2023-10-21T04:19:19.584+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.584+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0, IndexDefnId 12027054384338714804. 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.584+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9720186692804226822 Defn: DefnId: 12027054384338714804 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex] 2023-10-21T04:19:19.584+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 7.556826ms 2023-10-21T04:19:19.584+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex] 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex] 2023-10-21T04:19:19.584+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 9720186692804226822 2023-10-21T04:19:19.584+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex 2023-10-21T04:19:19.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex stopped 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.584+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.585+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [9720186692804226822] 2023-10-21T04:19:19.589+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:19.589+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:19:19.589+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.592738ms 2023-10-21T04:19:19.589+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 12.216936ms 2023-10-21T04:19:19.591+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex] 2023-10-21T04:19:19.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex closed 2023-10-21T04:19:19.591+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.591+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.592+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex] 2023-10-21T04:19:19.592+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex 2023-10-21T04:19:19.592+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex] 2023-10-21T04:19:19.592+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex] 2023-10-21T04:19:19.592+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex 2023-10-21T04:19:19.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex stopped 2023-10-21T04:19:19.592+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.593+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.593+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.593+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.593+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.596+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex] 2023-10-21T04:19:19.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex closed 2023-10-21T04:19:19.597+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.597+05:30 [Info] Indexer::cleanupIndexData IndexInst 9720186692804226822 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex ... 2023-10-21T04:19:19.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 1 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex ... 2023-10-21T04:19:19.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:19.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex ... 2023-10-21T04:19:19.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 1 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex ... 2023-10-21T04:19:19.604+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 32.829999ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:19.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_9720186692804226822_0.index destroyed 2023-10-21T04:19:19.605+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9720186692804226822, PartitionId 0, IndexDefnId 12027054384338714804. 2023-10-21T04:19:19.605+05:30 [Info] Indexer::cleanupIndexData IndexInst 9720186692804226822 Partition 0 Destroy Done 2023-10-21T04:19:19.640+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.640+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.647+05:30 [Info] switched currmeta from 608 -> 612 force true 2023-10-21T04:19:19.654+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.654+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.657+05:30 [Info] switched currmeta from 612 -> 612 force true 2023-10-21T04:19:19.834+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 13624500265195250810 &{0} 2023-10-21T04:19:19.834+05:30 [Info] Indexer::handleDropIndex - IndexInstId 13624500265195250810 2023-10-21T04:19:19.834+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:19:19.834+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.835+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13624500265195250810 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:19:19.835+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [13624500265195250810] 2023-10-21T04:19:19.835+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [13624500265195250810] 2023-10-21T04:19:19.836+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.836+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [13624500265195250810] 2023-10-21T04:19:19.837+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13624500265195250810, PartitionId 0, IndexDefnId 18046371353728341033. 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:19:19.837+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:19:19.837+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex 2023-10-21T04:19:19.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex stopped 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.837+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.839+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex] 2023-10-21T04:19:19.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex closed 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#13624500265195250810:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:19:19.840+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:19:19.840+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex 2023-10-21T04:19:19.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex stopped 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.840+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.836+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.843+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex] 2023-10-21T04:19:19.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex closed 2023-10-21T04:19:19.843+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 13624500265195250810 Defn: DefnId: 18046371353728341033 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.844+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 9.92651ms 2023-10-21T04:19:19.844+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 13624500265195250810 2023-10-21T04:19:19.844+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [13624500265195250810] 2023-10-21T04:19:19.844+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#13624500265195250810:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.844+05:30 [Info] Indexer::cleanupIndexData IndexInst 13624500265195250810 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:19.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex ... 2023-10-21T04:19:19.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 148 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex ... 2023-10-21T04:19:19.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:19.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex ... 2023-10-21T04:19:19.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 165 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex ... 2023-10-21T04:19:19.848+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:19.848+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:19:19.848+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.794522ms 2023-10-21T04:19:19.848+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 14.792739ms 2023-10-21T04:19:19.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:19:19.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_13624500265195250810_0.index destroyed 2023-10-21T04:19:19.853+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13624500265195250810, PartitionId 0, IndexDefnId 18046371353728341033. 2023-10-21T04:19:19.854+05:30 [Info] Indexer::cleanupIndexData IndexInst 13624500265195250810 Partition 0 Destroy Done 2023-10-21T04:19:19.855+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 28.990003ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:19.866+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.880+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.886+05:30 [Info] switched currmeta from 612 -> 615 force true 2023-10-21T04:19:19.906+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.907+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.915+05:30 [Info] switched currmeta from 615 -> 616 force true 2023-10-21T04:19:19.919+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:19.919+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:19.922+05:30 [Info] switched currmeta from 616 -> 616 force true 2023-10-21T04:19:19.987+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 265587992619548631 &{0} 2023-10-21T04:19:19.987+05:30 [Info] Indexer::handleDropIndex - IndexInstId 265587992619548631 2023-10-21T04:19:19.987+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:19:19.988+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.988+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 265587992619548631 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:19:19.989+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [265587992619548631] 2023-10-21T04:19:19.989+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [265587992619548631] 2023-10-21T04:19:19.989+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:19.989+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [265587992619548631] 2023-10-21T04:19:19.990+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 265587992619548631, PartitionId 0, IndexDefnId 7392796230534034390. 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.989+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex] 2023-10-21T04:19:19.990+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex] 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex] 2023-10-21T04:19:19.990+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex 2023-10-21T04:19:19.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex stopped 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.990+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 265587992619548631 Defn: DefnId: 7392796230534034390 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:19.990+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:19.990+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.013365ms 2023-10-21T04:19:19.990+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 265587992619548631 2023-10-21T04:19:19.991+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [265587992619548631] 2023-10-21T04:19:19.995+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:19.996+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:19.997+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:19:19.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex] 2023-10-21T04:19:19.997+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.916114ms 2023-10-21T04:19:19.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex closed 2023-10-21T04:19:19.997+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 9.99776ms 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Mainstore#265587992619548631:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:19.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex] 2023-10-21T04:19:19.998+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex] 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex] 2023-10-21T04:19:19.998+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex 2023-10-21T04:19:19.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex stopped 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:19.998+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:20.011+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:20.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex] 2023-10-21T04:19:20.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex closed 2023-10-21T04:19:20.013+05:30 [Info] bucket_%2/idx_secondary/Backstore#265587992619548631:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:20.013+05:30 [Info] Indexer::cleanupIndexData IndexInst 265587992619548631 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:20.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex ... 2023-10-21T04:19:20.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 166 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex ... 2023-10-21T04:19:20.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:20.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex ... 2023-10-21T04:19:20.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 185 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex ... 2023-10-21T04:19:20.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_265587992619548631_0.index destroyed 2023-10-21T04:19:20.030+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 265587992619548631, PartitionId 0, IndexDefnId 7392796230534034390. 2023-10-21T04:19:20.030+05:30 [Info] Indexer::cleanupIndexData IndexInst 265587992619548631 Partition 0 Destroy Done 2023-10-21T04:19:20.033+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 54.248577ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:20.063+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.063+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.074+05:30 [Info] switched currmeta from 616 -> 620 force true 2023-10-21T04:19:20.099+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.101+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.108+05:30 [Info] switched currmeta from 620 -> 620 force true 2023-10-21T04:19:20.146+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 8606677778651310689 &{0} 2023-10-21T04:19:20.146+05:30 [Info] Indexer::handleDropIndex - IndexInstId 8606677778651310689 2023-10-21T04:19:20.146+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:20.147+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:20.147+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8606677778651310689 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:20.148+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [8606677778651310689] 2023-10-21T04:19:20.148+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [8606677778651310689] 2023-10-21T04:19:20.148+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:20.148+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [8606677778651310689] 2023-10-21T04:19:20.148+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:20.149+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8606677778651310689, PartitionId 0, IndexDefnId 15968368593314043432. 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:20.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:19:20.149+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:19:20.149+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex 2023-10-21T04:19:20.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex stopped 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:20.149+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:20.149+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 8606677778651310689 Defn: DefnId: 15968368593314043432 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:20.152+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 5.891989ms 2023-10-21T04:19:20.152+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 8606677778651310689 2023-10-21T04:19:20.152+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [8606677778651310689] 2023-10-21T04:19:20.158+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:20.159+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:19:20.159+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.829121ms 2023-10-21T04:19:20.159+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 12.786582ms 2023-10-21T04:19:20.159+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:20.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex] 2023-10-21T04:19:20.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex closed 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#8606677778651310689:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:20.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:19:20.161+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:19:20.161+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex 2023-10-21T04:19:20.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex stopped 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:20.161+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:20.164+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:20.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex] 2023-10-21T04:19:20.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex closed 2023-10-21T04:19:20.165+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#8606677778651310689:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:20.165+05:30 [Info] Indexer::cleanupIndexData IndexInst 8606677778651310689 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:20.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex ... 2023-10-21T04:19:20.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 165 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex ... 2023-10-21T04:19:20.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:20.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex ... 2023-10-21T04:19:20.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 183 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex ... 2023-10-21T04:19:20.172+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 31.794077ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:20.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_8606677778651310689_0.index destroyed 2023-10-21T04:19:20.181+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8606677778651310689, PartitionId 0, IndexDefnId 15968368593314043432. 2023-10-21T04:19:20.181+05:30 [Info] Indexer::cleanupIndexData IndexInst 8606677778651310689 Partition 0 Destroy Done 2023-10-21T04:19:20.210+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.210+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.218+05:30 [Info] switched currmeta from 620 -> 624 force true 2023-10-21T04:19:20.235+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.235+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.240+05:30 [Info] switched currmeta from 624 -> 624 force true 2023-10-21T04:19:20.260+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 11050396066566371125 &{0} 2023-10-21T04:19:20.260+05:30 [Info] Indexer::handleDropIndex - IndexInstId 11050396066566371125 2023-10-21T04:19:20.261+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:20.261+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:20.261+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11050396066566371125 Name: idx_secondary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:19:20.262+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [11050396066566371125] 2023-10-21T04:19:20.262+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [11050396066566371125] 2023-10-21T04:19:20.262+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:20.262+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [11050396066566371125] 2023-10-21T04:19:20.263+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:20.263+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 11050396066566371125 Defn: DefnId: 14741949230702299196 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:20.263+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.222869ms 2023-10-21T04:19:20.263+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0, IndexDefnId 14741949230702299196. 2023-10-21T04:19:20.263+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:20.263+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:20.263+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:20.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex] 2023-10-21T04:19:20.264+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex 2023-10-21T04:19:20.264+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex] 2023-10-21T04:19:20.264+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex] 2023-10-21T04:19:20.264+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex 2023-10-21T04:19:20.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex stopped 2023-10-21T04:19:20.264+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:20.264+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:20.264+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:20.264+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:20.264+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 11050396066566371125 2023-10-21T04:19:20.267+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [11050396066566371125] 2023-10-21T04:19:20.268+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:20.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex] 2023-10-21T04:19:20.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex closed 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:20.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex] 2023-10-21T04:19:20.269+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex] 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex] 2023-10-21T04:19:20.269+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex 2023-10-21T04:19:20.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex stopped 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:20.269+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:20.271+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:20.271+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:19:20.271+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 7.733929ms 2023-10-21T04:19:20.271+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 11.027724ms 2023-10-21T04:19:20.274+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:20.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex] 2023-10-21T04:19:20.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex closed 2023-10-21T04:19:20.275+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:20.275+05:30 [Info] Indexer::cleanupIndexData IndexInst 11050396066566371125 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:20.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex ... 2023-10-21T04:19:20.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex ... 2023-10-21T04:19:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex ... 2023-10-21T04:19:20.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex ... 2023-10-21T04:19:20.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_11050396066566371125_0.index destroyed 2023-10-21T04:19:20.292+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11050396066566371125, PartitionId 0, IndexDefnId 14741949230702299196. 2023-10-21T04:19:20.292+05:30 [Info] Indexer::cleanupIndexData IndexInst 11050396066566371125 Partition 0 Destroy Done 2023-10-21T04:19:20.293+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 38.168039ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:20.347+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:19:20.366+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.366+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.371+05:30 [Info] switched currmeta from 624 -> 627 force true 2023-10-21T04:19:20.382+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.382+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.397+05:30 [Info] switched currmeta from 627 -> 628 force true 2023-10-21T04:19:20.430+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.430+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.436+05:30 [Info] switched currmeta from 628 -> 628 force true 2023-10-21T04:19:20.457+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15425360657090425347 &{0} 2023-10-21T04:19:20.457+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15425360657090425347 2023-10-21T04:19:20.458+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:19:20.458+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:20.458+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15425360657090425347 Name: idx_secondary_defer Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:19:20.459+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15425360657090425347] 2023-10-21T04:19:20.459+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15425360657090425347] 2023-10-21T04:19:20.459+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:19:20.459+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15425360657090425347] 2023-10-21T04:19:20.459+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:20.460+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15425360657090425347 Defn: DefnId: 8669649359784409226 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`company`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:19:20.460+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.900433ms 2023-10-21T04:19:20.461+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15425360657090425347, PartitionId 0, IndexDefnId 8669649359784409226. 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:20.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:19:20.461+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:19:20.461+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex 2023-10-21T04:19:20.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex stopped 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:20.461+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:20.462+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [15425360657090425347] 2023-10-21T04:19:20.462+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15425360657090425347 2023-10-21T04:19:20.469+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:20.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex] 2023-10-21T04:19:20.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex closed 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:19:20.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:19:20.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:19:20.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex 2023-10-21T04:19:20.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex stopped 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:19:20.470+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:19:20.473+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:19:20.473+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:19:20.473+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 12.776203ms 2023-10-21T04:19:20.473+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 15.774128ms 2023-10-21T04:19:20.477+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:19:20.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex] 2023-10-21T04:19:20.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex closed 2023-10-21T04:19:20.477+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:19:20.477+05:30 [Info] Indexer::cleanupIndexData IndexInst 15425360657090425347 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:19:20.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex ... 2023-10-21T04:19:20.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 139 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex ... 2023-10-21T04:19:20.487+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 43.996789ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:19:20.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/docIndex sucessfully destroyed 2023-10-21T04:19:20.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex ... 2023-10-21T04:19:20.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 155 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex ... 2023-10-21T04:19:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:19:20.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_15425360657090425347_0.index destroyed 2023-10-21T04:19:20.499+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15425360657090425347, PartitionId 0, IndexDefnId 8669649359784409226. 2023-10-21T04:19:20.500+05:30 [Info] Indexer::cleanupIndexData IndexInst 15425360657090425347 Partition 0 Destroy Done 2023-10-21T04:19:20.507+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33546 closed 2023-10-21T04:19:20.521+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33666 closed 2023-10-21T04:19:20.526+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33668 closed 2023-10-21T04:19:20.529+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33670 closed 2023-10-21T04:19:20.539+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.539+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.542+05:30 [Info] switched currmeta from 628 -> 632 force true 2023-10-21T04:19:20.552+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:20.552+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:20.558+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:19:20.560+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33672 closed 2023-10-21T04:19:20.604+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33680 closed 2023-10-21T04:19:20.607+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33686 closed 2023-10-21T04:19:20.611+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33688 closed 2023-10-21T04:19:20.625+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33692 closed 2023-10-21T04:19:20.656+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33696 closed 2023-10-21T04:19:20.685+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33702 closed 2023-10-21T04:19:20.697+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33706 closed 2023-10-21T04:19:20.706+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33708 closed 2023-10-21T04:19:20.709+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33710 closed 2023-10-21T04:19:20.713+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33712 closed 2023-10-21T04:19:20.733+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33714 closed 2023-10-21T04:19:20.745+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33718 closed 2023-10-21T04:19:20.760+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33720 closed 2023-10-21T04:19:20.784+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33730 closed 2023-10-21T04:19:20.838+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33734 closed 2023-10-21T04:19:20.842+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33740 closed 2023-10-21T04:19:20.860+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33744 closed 2023-10-21T04:19:20.870+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33748 closed 2023-10-21T04:19:20.874+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33750 closed 2023-10-21T04:19:20.882+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33752 closed 2023-10-21T04:19:20.899+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33758 closed 2023-10-21T04:19:20.903+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33760 closed 2023-10-21T04:19:20.914+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33762 closed 2023-10-21T04:19:20.922+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33766 closed 2023-10-21T04:19:20.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33768 closed 2023-10-21T04:19:20.956+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33772 closed 2023-10-21T04:19:20.960+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33774 closed 2023-10-21T04:19:20.993+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33776 closed 2023-10-21T04:19:21.113+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33784 closed 2023-10-21T04:19:21.124+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33792 closed 2023-10-21T04:19:21.134+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33794 closed 2023-10-21T04:19:21.164+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33796 closed 2023-10-21T04:19:21.180+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33800 closed 2023-10-21T04:19:21.201+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33804 closed 2023-10-21T04:19:21.224+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33808 closed 2023-10-21T04:19:21.228+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33810 closed 2023-10-21T04:19:21.255+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33812 closed 2023-10-21T04:19:21.270+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33816 closed 2023-10-21T04:19:21.275+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33818 closed 2023-10-21T04:19:21.280+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33820 closed 2023-10-21T04:19:21.364+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33826 closed 2023-10-21T04:19:21.412+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33842 closed 2023-10-21T04:19:21.423+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33846 closed 2023-10-21T04:19:21.437+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33850 closed 2023-10-21T04:19:21.460+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33854 closed 2023-10-21T04:19:21.465+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33856 closed 2023-10-21T04:19:21.547+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33858 closed 2023-10-21T04:19:21.582+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33872 closed 2023-10-21T04:19:21.622+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33886 closed 2023-10-21T04:19:21.636+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33892 closed 2023-10-21T04:19:21.660+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33894 closed 2023-10-21T04:19:21.688+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33898 closed 2023-10-21T04:19:24.275+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:19:24.275+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:19:24.276+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:19:24.276+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:19:24.276+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:19:24.276+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:19:24.277+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:19:25.219+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:19:25.223+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:19:25.224+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:19:25.228+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:19:25.484+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33904 closed 2023-10-21T04:19:25.561+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33962 closed 2023-10-21T04:19:25.565+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33968 closed 2023-10-21T04:19:25.578+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33970 closed 2023-10-21T04:19:25.590+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33972 closed 2023-10-21T04:19:25.618+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33976 closed 2023-10-21T04:19:25.745+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33982 closed 2023-10-21T04:19:25.757+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33994 closed 2023-10-21T04:19:25.762+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:33996 closed 2023-10-21T04:19:25.773+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34000 closed 2023-10-21T04:19:25.777+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34002 closed 2023-10-21T04:19:25.791+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34008 closed 2023-10-21T04:19:25.800+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34012 closed 2023-10-21T04:19:25.850+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34014 closed 2023-10-21T04:19:25.856+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34022 closed 2023-10-21T04:19:25.926+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34030 closed 2023-10-21T04:19:25.932+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34036 closed 2023-10-21T04:19:25.943+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34038 closed 2023-10-21T04:19:25.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34040 closed 2023-10-21T04:19:25.978+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34042 closed 2023-10-21T04:19:26.057+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34046 closed 2023-10-21T04:19:26.151+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34054 closed 2023-10-21T04:19:26.181+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34072 closed 2023-10-21T04:19:26.206+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34074 closed 2023-10-21T04:19:26.247+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34078 closed 2023-10-21T04:19:26.264+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34086 closed 2023-10-21T04:19:26.314+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34092 closed 2023-10-21T04:19:26.320+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34100 closed 2023-10-21T04:19:26.332+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34102 closed 2023-10-21T04:19:26.343+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34104 closed 2023-10-21T04:19:26.347+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34106 closed 2023-10-21T04:19:26.358+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34108 closed 2023-10-21T04:19:26.379+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34112 closed 2023-10-21T04:19:26.383+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34114 closed 2023-10-21T04:19:26.389+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34116 closed 2023-10-21T04:19:26.418+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34128 closed 2023-10-21T04:19:26.426+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34130 closed 2023-10-21T04:19:26.431+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34132 closed 2023-10-21T04:19:26.446+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34140 closed 2023-10-21T04:19:26.499+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34146 closed 2023-10-21T04:19:26.503+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34150 closed 2023-10-21T04:19:26.536+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34156 closed 2023-10-21T04:19:26.542+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34158 closed 2023-10-21T04:19:26.595+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34160 closed 2023-10-21T04:19:26.619+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34172 closed 2023-10-21T04:19:26.639+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34176 closed 2023-10-21T04:19:26.993+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:19:26.993+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 6 Creating New Snapshot SnapshotInfo: count:137 committed:true 2023-10-21T04:19:26.993+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:137 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.706µs SnapOpenDur 110.117µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 7 Creating New Snapshot SnapshotInfo: count:102 committed:true 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Creating New Snapshot SnapshotInfo: count:27 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:102 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.653µs SnapOpenDur 39.684µs 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 4992762551574016313 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:27 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.763µs SnapOpenDur 75.034µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 8 Creating New Snapshot SnapshotInfo: count:148 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:148 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.286µs SnapOpenDur 39.466µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 1 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.617µs SnapOpenDur 36.99µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 4 Creating New Snapshot SnapshotInfo: count:130 committed:true 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:130 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.976µs SnapOpenDur 92.825µs 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 5 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.379µs SnapOpenDur 34.403µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 6 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:106 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.203µs SnapOpenDur 33.632µs 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 7 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.207µs SnapOpenDur 36.353µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 8 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1265729768899070816 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.158µs SnapOpenDur 107.782µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13739166377675070800 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1265729768899070816 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.984µs SnapOpenDur 69.089µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 1 Creating New Snapshot SnapshotInfo: count:120 committed:true 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:120 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.616µs SnapOpenDur 37.937µs 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 2 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.992µs SnapOpenDur 88.744µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 3 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.364µs SnapOpenDur 52.964µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14579235200590321485 PartitionId 3 Creating New Snapshot SnapshotInfo: count:132 committed:true 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14579235200590321485 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:132 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.198µs SnapOpenDur 35.619µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 4 Creating New Snapshot SnapshotInfo: count:140 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:140 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.665µs SnapOpenDur 39.4µs 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 5 Creating New Snapshot SnapshotInfo: count:116 committed:true 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:116 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.899µs SnapOpenDur 32.774µs 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 6 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.403µs SnapOpenDur 33.721µs 2023-10-21T04:19:26.994+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2296526085582506469 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 7 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2296526085582506469 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.644µs SnapOpenDur 1.00553ms 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 7 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.314µs SnapOpenDur 43.993µs 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 8 Creating New Snapshot SnapshotInfo: count:124 committed:true 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 8 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:124 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.542µs SnapOpenDur 33µs 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 1 Creating New Snapshot SnapshotInfo: count:129 committed:true 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13739166377675070800 PartitionId: 0 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.396µs SnapOpenDur 52.297µs 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 1 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:129 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.377µs SnapOpenDur 34.727µs 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 11460826864060385936 PartitionId 2 Creating New Snapshot SnapshotInfo: count:110 committed:true 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 11460826864060385936 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:110 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.457µs SnapOpenDur 30.734µs 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 2 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.08µs SnapOpenDur 37.462µs 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 3 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.934µs SnapOpenDur 42.863µs 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 4 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 4 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.542µs SnapOpenDur 102.738µs 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17357232428819395534 PartitionId 5 Creating New Snapshot SnapshotInfo: count:117 committed:true 2023-10-21T04:19:26.995+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17357232428819395534 PartitionId: 5 SliceId: 0 Crc64: 18394135563695078153 (SnapshotInfo: count:117 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.245µs SnapOpenDur 38.467µs 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 11 Creating recovery point ... 2023-10-21T04:19:26.994+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:26.995+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:19:27.005+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 SnapshotId 4 Created recovery point (took 10.904601ms) 2023-10-21T04:19:27.006+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:27.032+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 SnapshotId 4 Created recovery point (took 37.861881ms) 2023-10-21T04:19:27.034+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:27.040+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 SnapshotId 4 Created recovery point (took 45.613929ms) 2023-10-21T04:19:27.041+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:19:27.205+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 SnapshotId 4 Created recovery point (took 209.787739ms) 2023-10-21T04:19:27.206+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:19:27.207+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:19:27.287+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 SnapshotId 11 Created recovery point (took 291.504744ms) 2023-10-21T04:19:27.288+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:21 committed:false. num RPs 3. 2023-10-21T04:19:27.353+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 SnapshotId 4 Created recovery point (took 359.593919ms) 2023-10-21T04:19:27.354+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:19:27.355+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:19:27.363+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 SnapshotId 4 Created recovery point (took 368.515314ms) 2023-10-21T04:19:27.363+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:19:27.364+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:137 committed:false. num RPs 3. 2023-10-21T04:19:27.371+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 SnapshotId 4 Created recovery point (took 376.33575ms) 2023-10-21T04:19:27.372+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:19:27.372+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:102 committed:false. num RPs 3. 2023-10-21T04:19:27.380+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 SnapshotId 4 Created recovery point (took 385.909187ms) 2023-10-21T04:19:27.381+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:19:27.382+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:19:27.393+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 SnapshotId 4 Created recovery point (took 398.812503ms) 2023-10-21T04:19:27.394+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:19:27.395+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:19:27.416+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 SnapshotId 4 Created recovery point (took 421.549852ms) 2023-10-21T04:19:27.417+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:19:27.418+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:19:27.435+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 SnapshotId 4 Created recovery point (took 440.510959ms) 2023-10-21T04:19:27.436+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:19:27.436+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:19:27.453+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 SnapshotId 4 Created recovery point (took 457.897481ms) 2023-10-21T04:19:27.455+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:19:27.455+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:120 committed:false. num RPs 3. 2023-10-21T04:19:27.466+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 SnapshotId 4 Created recovery point (took 470.721701ms) 2023-10-21T04:19:27.466+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:19:27.467+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:19:27.481+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 SnapshotId 4 Created recovery point (took 486.192332ms) 2023-10-21T04:19:27.482+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:19:27.482+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:130 committed:false. num RPs 3. 2023-10-21T04:19:27.492+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 SnapshotId 4 Created recovery point (took 496.578256ms) 2023-10-21T04:19:27.493+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:19:27.493+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:129 committed:false. num RPs 3. 2023-10-21T04:19:27.497+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 SnapshotId 4 Created recovery point (took 502.005532ms) 2023-10-21T04:19:27.498+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:19:27.498+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:19:27.506+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 SnapshotId 4 Created recovery point (took 510.665504ms) 2023-10-21T04:19:27.507+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:19:27.507+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:140 committed:false. num RPs 3. 2023-10-21T04:19:27.517+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 SnapshotId 4 Created recovery point (took 521.437206ms) 2023-10-21T04:19:27.517+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:19:27.518+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:124 committed:false. num RPs 3. 2023-10-21T04:19:27.528+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 SnapshotId 4 Created recovery point (took 532.697182ms) 2023-10-21T04:19:27.529+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:19:27.530+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:116 committed:false. num RPs 3. 2023-10-21T04:19:27.536+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 SnapshotId 4 Created recovery point (took 540.613439ms) 2023-10-21T04:19:27.538+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:19:27.538+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:19:27.551+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 SnapshotId 4 Created recovery point (took 555.404862ms) 2023-10-21T04:19:27.553+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:19:27.553+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:132 committed:false. num RPs 3. 2023-10-21T04:19:27.571+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 SnapshotId 4 Created recovery point (took 575.929087ms) 2023-10-21T04:19:27.574+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:19:27.574+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:110 committed:false. num RPs 3. 2023-10-21T04:19:27.593+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 SnapshotId 4 Created recovery point (took 597.466738ms) 2023-10-21T04:19:27.594+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:19:27.594+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:148 committed:false. num RPs 3. 2023-10-21T04:19:27.619+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 SnapshotId 4 Created recovery point (took 623.035471ms) 2023-10-21T04:19:27.619+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:19:27.620+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:19:27.630+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 SnapshotId 4 Created recovery point (took 634.405736ms) 2023-10-21T04:19:27.631+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:19:27.631+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:19:27.642+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 SnapshotId 4 Created recovery point (took 645.993504ms) 2023-10-21T04:19:27.643+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:19:27.643+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:19:27.655+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 SnapshotId 4 Created recovery point (took 658.478005ms) 2023-10-21T04:19:27.656+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:19:27.656+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:117 committed:false. num RPs 3. 2023-10-21T04:19:30.605+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34180 closed 2023-10-21T04:19:30.651+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34284 closed 2023-10-21T04:19:30.677+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34294 closed 2023-10-21T04:19:30.694+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34300 closed 2023-10-21T04:19:30.737+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34306 closed 2023-10-21T04:19:30.743+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34308 closed 2023-10-21T04:19:30.764+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34312 closed 2023-10-21T04:19:30.824+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34318 closed 2023-10-21T04:19:30.855+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34322 closed 2023-10-21T04:19:30.876+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34328 closed 2023-10-21T04:19:30.903+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34330 closed 2023-10-21T04:19:30.923+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34334 closed 2023-10-21T04:19:30.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34338 closed 2023-10-21T04:19:30.970+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34344 closed 2023-10-21T04:19:30.980+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34346 closed 2023-10-21T04:19:30.983+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34348 closed 2023-10-21T04:19:31.111+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34350 closed 2023-10-21T04:19:31.123+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34366 closed 2023-10-21T04:19:31.127+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34368 closed 2023-10-21T04:19:31.143+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34372 closed 2023-10-21T04:19:31.180+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34376 closed 2023-10-21T04:19:31.226+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34378 closed 2023-10-21T04:19:31.238+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34384 closed 2023-10-21T04:19:31.262+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34388 closed 2023-10-21T04:19:31.276+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34392 closed 2023-10-21T04:19:31.319+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34394 closed 2023-10-21T04:19:31.334+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34404 closed 2023-10-21T04:19:31.391+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34408 closed 2023-10-21T04:19:31.439+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34418 closed 2023-10-21T04:19:31.467+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34426 closed 2023-10-21T04:19:31.502+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34428 closed 2023-10-21T04:19:31.539+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34430 closed 2023-10-21T04:19:31.546+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34432 closed 2023-10-21T04:19:31.603+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34442 closed 2023-10-21T04:19:31.638+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34448 closed 2023-10-21T04:19:35.345+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34452 closed 2023-10-21T04:19:35.399+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34512 closed 2023-10-21T04:19:35.412+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34514 closed 2023-10-21T04:19:35.414+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34516 closed 2023-10-21T04:19:35.426+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34518 closed 2023-10-21T04:19:35.429+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34520 closed 2023-10-21T04:19:35.432+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34522 closed 2023-10-21T04:19:35.440+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34524 closed 2023-10-21T04:19:35.458+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34526 closed 2023-10-21T04:19:35.468+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34530 closed 2023-10-21T04:19:35.471+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34532 closed 2023-10-21T04:19:35.488+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34534 closed 2023-10-21T04:19:35.511+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34540 closed 2023-10-21T04:19:35.516+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34542 closed 2023-10-21T04:19:35.548+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34548 closed 2023-10-21T04:19:35.562+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34554 closed 2023-10-21T04:19:35.564+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34556 closed 2023-10-21T04:19:35.568+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34558 closed 2023-10-21T04:19:35.586+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34562 closed 2023-10-21T04:19:35.594+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34564 closed 2023-10-21T04:19:35.604+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34566 closed 2023-10-21T04:19:35.612+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34568 closed 2023-10-21T04:19:35.614+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34570 closed 2023-10-21T04:19:35.625+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34574 closed 2023-10-21T04:19:35.628+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34576 closed 2023-10-21T04:19:35.631+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34578 closed 2023-10-21T04:19:35.636+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34580 closed 2023-10-21T04:19:35.656+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34584 closed 2023-10-21T04:19:35.664+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34588 closed 2023-10-21T04:19:35.668+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34590 closed 2023-10-21T04:19:35.736+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34604 closed 2023-10-21T04:19:35.766+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34606 closed 2023-10-21T04:19:35.793+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34608 closed 2023-10-21T04:19:35.804+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34612 closed 2023-10-21T04:19:35.805+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34616 closed 2023-10-21T04:19:35.808+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34618 closed 2023-10-21T04:19:35.828+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34622 closed 2023-10-21T04:19:35.859+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34624 closed 2023-10-21T04:19:35.878+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34626 closed 2023-10-21T04:19:35.907+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34630 closed 2023-10-21T04:19:35.922+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34638 closed 2023-10-21T04:19:35.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34642 closed 2023-10-21T04:19:35.940+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34644 closed 2023-10-21T04:19:35.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34650 closed 2023-10-21T04:19:35.958+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34652 closed 2023-10-21T04:19:35.963+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34654 closed 2023-10-21T04:19:35.976+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34658 closed 2023-10-21T04:19:35.995+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34662 closed 2023-10-21T04:19:36.000+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34664 closed 2023-10-21T04:19:36.034+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34666 closed 2023-10-21T04:19:36.056+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34670 closed 2023-10-21T04:19:36.092+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34672 closed 2023-10-21T04:19:36.111+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34674 closed 2023-10-21T04:19:36.118+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34680 closed 2023-10-21T04:19:36.140+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34682 closed 2023-10-21T04:19:36.175+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34688 closed 2023-10-21T04:19:36.192+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34696 closed 2023-10-21T04:19:36.231+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34700 closed 2023-10-21T04:19:36.235+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34704 closed 2023-10-21T04:19:36.239+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34706 closed 2023-10-21T04:19:36.256+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34708 closed 2023-10-21T04:19:36.310+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34710 closed 2023-10-21T04:19:36.342+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34718 closed 2023-10-21T04:19:36.347+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34724 closed 2023-10-21T04:19:36.352+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34726 closed 2023-10-21T04:19:36.370+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34728 closed 2023-10-21T04:19:36.391+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34730 closed 2023-10-21T04:19:36.400+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34732 closed 2023-10-21T04:19:36.405+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34734 closed 2023-10-21T04:19:36.431+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34736 closed 2023-10-21T04:19:36.448+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34740 closed 2023-10-21T04:19:36.466+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34746 closed 2023-10-21T04:19:36.471+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34748 closed 2023-10-21T04:19:36.494+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34756 closed 2023-10-21T04:19:42.027+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:19:54.277+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:19:54.277+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:19:54.278+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:19:54.278+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:19:54.278+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:19:54.278+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:19:54.280+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:20:03.039+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 220] aggregate storage info: map[bucket_%2:map[kv:5814464 index:68413658] bucket_1:map[kv:6539456 index:61011522]] 2023-10-21T04:20:12.025+05:30 [Info] memstats {"Alloc":275873504, "TotalAlloc":5345633128, "Sys":392810840, "Lookups":0, "Mallocs":61977179,"Frees":60358770, "HeapAlloc":275873504, "HeapSys":354156544, "HeapIdle":64290816, "HeapInuse":289865728,"HeapReleased":778240, "HeapObjects":1618409,"MSpanInuse":2374720, "MSpanSys": 3100800, "StackInuse": 14942208,"GCSys":17376528, "LastGC":1697842152408914227,"PauseTotalNs":52301446, "PauseNs":[188871, 174011, 73749, 228419, 310558, 126167, 194931, 196032, 171937, 196216], "NumGC":40} 2023-10-21T04:20:12.032+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:20:12.179+05:30 [Info] janitor: running cleanup. 2023-10-21T04:20:12.179+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[12027054384338714804 8669649359784409226 249887442666643345 8504686614261377811 12140458060295645238 6985049432225870291 15968368593314043432 2048006213723315482 14889401051926000176 14292583894719865678 7392796230534034390 4860549750398470299 15945117906390072636 18323460706027642575 4535197604682045916 18046371353728341033 14741949230702299196 5465835793930010953 12195715103909237684 4138047918192025493]} 2023-10-21T04:20:12.308+05:30 [Info] connected with 2 indexers 2023-10-21T04:20:12.308+05:30 [Info] client stats current counts: current: 52, not current: 0 2023-10-21T04:20:12.437+05:30 [Info] num concurrent scans {0} 2023-10-21T04:20:12.437+05:30 [Info] average scan response {0 ms} 2023-10-21T04:20:12.800+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:20:12.820+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:20:14.350+05:30 [Info] Indexer::ReadMemstats Time Taken 168.545µs 2023-10-21T04:20:24.279+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:20:24.279+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:20:24.280+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:20:24.280+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:20:24.280+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:20:24.280+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:20:24.281+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:20:42.039+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:20:54.281+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:20:54.281+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:20:54.281+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:20:54.281+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:20:54.281+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:20:54.284+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:20:54.284+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:21:11.132+05:30 [Info] Periodic Aggregated StorageStats: { "num_tenant": 2, "num_controller": 2, "num_shard": 4, "num_shard_altid": 0, "num_instances": 206, "num_lssctx": 8, "num_sctxs": 60, "num_active_sctxs": 32, "num_free_sctxs": 28, "num_trim_sctxs": 4, "num_wctxs": 495, "num_free_wctxs": 289, "num_active_wctxs": 206, "num_trim_wctxs": 0, "num_readers": 54, "num_writers": 5, "cpu": 0.17600, "total_core": 4, "threads": 16, "avail_thread_barrier": 8000, "assigned_quota": 1415577600, "current_quota": 1415577600, "tenant_quota_assigned": 18631977, "tenant_quota_mandatory": 18631977, "tenant_quota_discretionary": 0, "tenant_quota_mutation": 17927563, "tenant_working_set": 0, "heap_limit": max, "memory_resident": 285622272, "memory_jemalloc": 17543168, "memory_jemalloc_active": 9166848, "memory_jemalloc_allocated": 6600904, "memory_jemalloc_dirty": 3481600, "estimated_mem_frag": 16.37847, "memory_stats_size_total": 1259906, "memory_stats_size_global": 1009477, "memory_stats_size_page": 1005553, "memory_stats_size_index": 3924, "memory_stats_size_bloom": 0, "memory_stats_size_delta": 394432, "reclaim_pending": 0, "reclaim_pending_global": 0, "buf_memused": 250429, "lss_fbuf_size": 16777216, "under_swap": false, "num_freeOS": 0, "num_smr_ctx ": 3, "num_smr_worker ": 24, "num_active_smr_worker ": 0, "num_smr_queued ": 0, "num_total_smr": 206, "num_active_smr": 0, "num_pending_smr": 0, "num_pages": 255, "items_count": 18054, "total_records": 18060, "avg_item_size": 52, "total_ops": 17, "compacts": 2, "purges": 0, "merges_emptypages": 0, "lss_data_size": 4638101, "lss_used_space": 64577536, "lss_disk_size": 64577536, "lss_open_filedesc": 16, "max_open_filedesc": 200000, "lss_fragmentation": 92, "lss_num_reads": 521, "lss_read_bs": 11740869, "lss_blk_read_bs": 18411520, "rlss_num_reads": 341, "lss_rdr_reads_bs": 203423, "lss_blk_rdr_reads_bs": 1449984, "lookup_num_reads": 54, "lookup_reads_bs": 10640, "lookup_blk_reads_bs": 217088, "bytes_written": 20025344, "bytes_incoming": 832261, "write_amp_avg": 12.31, "num_swapper_ctx": 2, "avg_sweep_interval": 5m0s, "avg_run_interval": 100ms, "avg_clock_interval": 0s, "num_swapperWorker": 4, "cached_records": 12060, "resident_ratio": 0.66777, "num_rec_compressed": 0, "num_rec_compressible": 12052, "compressed_percent": 0, "compressed_compressible_perc": 0, "compression_ratio": 2.93865, "num_burst_visits": 0, "num_periodic_visits": 9, "num_evicted": 0, "num_evictable": 0, "cleaner_num_reads": 0, "cleaner_read_bs": 0, "cleaner_blk_read_bs": 0, "recovery_num_lssctx": 4, "recovery_num_sctxs": 20, "recovery_num_active_sctxs": 16, "recovery_num_free_sctxs": 4, "recovery_num_trim_sctxs": 4, "recovery_buf_memused": 0, "recovery_fbuf_size": 8388608, "recovery_data_size": 2177206, "recovery_used_space": 31653888, "recovery_disk_size": 31653888, "recovery_lss_open_filedesc": 8, "recovery_lss_blk_read_bs": 16105472, "recovery_lss_read_bs": 11399906, "recovery_lss__num_reads": 0, "recovery_bytes_written": 9768960, "recovery_lss_frag": 93, "recovery_cleaner_num_reads": 0, "recovery_cleaner_read_bs": 0, "recovery_cleaner_blk_read_bs": 0, "mvcc_purge_ratio": 1.00033, "reader_purge_count": 0, "reader_emptypage_count": 0, "fbuf_tuner_status": , "active_copy_ctx": 0, "num_shard_upload": 0 (0), "num_shard_restored": 4 (4), "upload_bytes": 0, "avg_upload_bw": 0, "download_bytes": 44652032, "avg_download_bw": 206066184, "num_upload_cleanups": 0 (0), "num_staging_cleanups": 0 (0) } 2023-10-21T04:21:11.135+05:30 [Info] Periodic TenantStats: { "bucket_%2_7007f0ab442ac00e52d057a30cd22644": { "num_shards": 2, "num_instances": 103, "num_controllers": 1, "quota": 9315692, "mandatory_quota": 9315692, "max_mandatory_quota": 9315692, "discretionary_quota": 0, "resident_quota": 351912, "mutation_quota": 8963780, "max_mutation_quota": 8963780, "idle_quota": 348856, "derived_quota": 115456, "working_set": 0, "mem_in_use": 505000, "mem_index": 1989, "total_records": 9031, "avg_key_size": 52, "avg_mem_per_key": 83, "resident_ratio": 0.66781, "idle": false, "total_ops": 1999, "read_ops": 1972, "new_ops": 0, "new_read_ops": 0, "compacts": 1, "compresses": 0, "cache_hits": 1959, "cache_misses": 40, "cache_hit_ratio": 0.00000, "lookup_dur": "0s", "writer_dur": "0s", "writer_throttle_dur": "0s", "controller": { "build": [false], "swap": [true], "instances": [103], "pages": [128], "quota": [9315692], "mem_in_use": [505000], "mem_used_global": [505000], "mandatory_quota": [9315692], "discretionary_quota": [0], "resident_quota": [351912], "derived_quota": [115456], "mutation_quota": [8963780], "idle_quota": [348856], "new_evict_cycle": [0], "new_evict_visit": [0], "new_evicted": [0], "swapper_instances": [[103]], "sweep_duration": [["0s"]], "swapper_pages": [[128]], "swapper_cycles": [[0]], "evict_visit": [[1]], "evict_burst": [[0]], "evict_periodic": [[1]], "evictable": [[0]], "evicted": [[0]], "compressed": [[0]], "resident": [[0.66781]] } }, "bucket_1_db19c9050b4ab1232e6bc6654011618f": { "num_shards": 2, "num_instances": 103, "num_controllers": 1, "quota": 9316285, "mandatory_quota": 9316285, "max_mandatory_quota": 9316285, "discretionary_quota": 0, "resident_quota": 352502, "mutation_quota": 8963783, "max_mutation_quota": 8963783, "idle_quota": 349384, "derived_quota": 115684, "working_set": 0, "mem_in_use": 504477, "mem_index": 1935, "total_records": 9029, "avg_key_size": 52, "avg_mem_per_key": 83, "resident_ratio": 0.66774, "idle": false, "total_ops": 1906, "read_ops": 1881, "new_ops": 0, "new_read_ops": 0, "compacts": 1, "compresses": 0, "cache_hits": 1850, "cache_misses": 56, "cache_hit_ratio": 0.00000, "lookup_dur": "0s", "writer_dur": "0s", "writer_throttle_dur": "0s", "controller": { "build": [false], "swap": [true], "instances": [103], "pages": [127], "quota": [9316285], "mem_in_use": [504477], "mem_used_global": [504477], "mandatory_quota": [9316285], "discretionary_quota": [0], "resident_quota": [352502], "derived_quota": [115684], "mutation_quota": [8963783], "idle_quota": [349384], "new_evict_cycle": [0], "new_evict_visit": [0], "new_evicted": [0], "swapper_instances": [[103]], "sweep_duration": [["0s"]], "swapper_pages": [[127]], "swapper_cycles": [[0]], "evict_visit": [[8]], "evict_burst": [[0]], "evict_periodic": [[8]], "evictable": [[0]], "evicted": [[0]], "compressed": [[0]], "resident": [[0.66774]] } } } 2023-10-21T04:21:11.169+05:30 DEBU REGU.impl.dumpStats() at aggrecorder.go:364 [id 68] aggRecorder: [bucket_1] recorded 89477 aborted 0 refunded 0 2023-10-21T04:21:11.169+05:30 DEBU REGU.impl.dumpStats() at aggrecorder.go:364 [id 68] aggRecorder: [bucket_%2] recorded 89153 aborted 0 refunded 0 2023-10-21T04:21:11.946+05:30 [Info] DDLServiceMgr checking create token progress 2023-10-21T04:21:11.961+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9118 2023-10-21T04:21:11.961+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:21:11.967+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56322 2023-10-21T04:21:11.971+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:21:11.991+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 11.597744ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:11.991+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:21:11.991+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9124 2023-10-21T04:21:11.991+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:21:11.996+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:21:12.014+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:21:12.016+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:21:12.016+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:21:12.016+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:21:12.023+05:30 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/15945117906390072636 2023-10-21T04:21:12.025+05:30 [Info] memstats {"Alloc":188022416, "TotalAlloc":6698474520, "Sys":397005144, "Lookups":0, "Mallocs":77998557,"Frees":77299133, "HeapAlloc":188022416, "HeapSys":362807296, "HeapIdle":148340736, "HeapInuse":214466560,"HeapReleased":59146240, "HeapObjects":699424,"MSpanInuse":1707360, "MSpanSys": 3100800, "StackInuse": 10485760,"GCSys":17384720, "LastGC":1697842211371976970,"PauseTotalNs":54036216, "PauseNs":[180267, 357158, 243305, 129251, 135635, 135553, 89072, 249157, 132429, 82943], "NumGC":50} 2023-10-21T04:21:12.027+05:30 [Info] DDLServiceMgr: Remove build index token /indexing/ddl/commandToken/build/15945117906390072636. 2023-10-21T04:21:12.027+05:30 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/15968368593314043432 2023-10-21T04:21:12.028+05:30 [Info] DDLServiceMgr: Remove build index token /indexing/ddl/commandToken/build/15968368593314043432. 2023-10-21T04:21:12.028+05:30 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/18046371353728341033 2023-10-21T04:21:12.040+05:30 [Info] DDLServiceMgr: Remove build index token /indexing/ddl/commandToken/build/18046371353728341033. 2023-10-21T04:21:12.040+05:30 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/18323460706027642575 2023-10-21T04:21:12.044+05:30 [Info] DDLServiceMgr: Remove build index token /indexing/ddl/commandToken/build/18323460706027642575. 2023-10-21T04:21:12.044+05:30 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/6985049432225870291 2023-10-21T04:21:12.044+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:21:12.048+05:30 [Info] DDLServiceMgr: Remove build index token /indexing/ddl/commandToken/build/6985049432225870291. 2023-10-21T04:21:12.048+05:30 [Info] DDLServiceMgr: processing build index token /indexing/ddl/commandToken/build/8669649359784409226 2023-10-21T04:21:12.056+05:30 [Info] DDLServiceMgr: Remove build index token /indexing/ddl/commandToken/build/8669649359784409226. 2023-10-21T04:21:12.071+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12027054384338714804 2023-10-21T04:21:12.091+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12140458060295645238 2023-10-21T04:21:12.115+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12195715103909237684 2023-10-21T04:21:12.144+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14292583894719865678 2023-10-21T04:21:12.179+05:30 [Info] janitor: running cleanup. 2023-10-21T04:21:12.179+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[8669649359784409226 249887442666643345 7392796230534034390 4535197604682045916 6985049432225870291 12140458060295645238 18323460706027642575 14292583894719865678 12195715103909237684 8504686614261377811 5465835793930010953 15945117906390072636 15968368593314043432 12027054384338714804 4860549750398470299 18046371353728341033 14889401051926000176 4138047918192025493 14741949230702299196 2048006213723315482]} 2023-10-21T04:21:12.180+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14741949230702299196 2023-10-21T04:21:12.215+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14889401051926000176 2023-10-21T04:21:12.234+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15945117906390072636 2023-10-21T04:21:12.254+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56338 2023-10-21T04:21:12.274+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15968368593314043432 2023-10-21T04:21:12.308+05:30 [Info] connected with 2 indexers 2023-10-21T04:21:12.308+05:30 [Info] client stats current counts: current: 52, not current: 0 2023-10-21T04:21:12.315+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 52.518769ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:12.352+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18046371353728341033 2023-10-21T04:21:12.436+05:30 [Info] num concurrent scans {0} 2023-10-21T04:21:12.436+05:30 [Info] average scan response {0 ms} 2023-10-21T04:21:12.446+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18323460706027642575 2023-10-21T04:21:12.493+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:21:12.536+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/2048006213723315482 2023-10-21T04:21:12.574+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/249887442666643345 2023-10-21T04:21:12.607+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4138047918192025493 2023-10-21T04:21:12.667+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4535197604682045916 2023-10-21T04:21:12.760+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4860549750398470299 2023-10-21T04:21:12.816+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:21:12.825+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/5465835793930010953 2023-10-21T04:21:12.837+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:21:12.858+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:21:12.863+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/6985049432225870291 2023-10-21T04:21:12.920+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7392796230534034390 2023-10-21T04:21:12.933+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8504686614261377811 2023-10-21T04:21:12.959+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8669649359784409226 2023-10-21T04:21:13.006+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:21:13.006+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:21:13.006+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:21:13.007+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:56322->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:21:13.007+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:52698->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:21:13.007+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:56322. Reason = EOF. Cleaning Pipe. 2023-10-21T04:21:13.007+05:30 [Info] messageListener.start(): message channel closed. Remove peer df:33:1e:93:3c:b:63:1f as follower. 2023-10-21T04:21:13.196+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:56338. Reason = EOF. Cleaning Pipe. 2023-10-21T04:21:13.197+05:30 [Info] messageListener.start(): message channel closed. Remove peer b3:f9:f2:5:e3:fc:8c:d9 as follower. 2023-10-21T04:21:14.852+05:30 [Info] Indexer::ReadMemstats Time Taken 217.738µs 2023-10-21T04:21:24.284+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:21:24.284+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4c6e4b64d0c01f9d6431a65ac36fc024 458c2b8d760618b2a2f6a998f4c93e74] IsBalanced:true Messages:[]} 2023-10-21T04:21:24.284+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:21:24.284+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:21:24.285+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:21:24.286+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:21:24.286+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:21:28.937+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:21:29.017+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 1 2023-10-21T04:21:29.017+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:29.017+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:29.028+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:29.045+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:29.108+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:29.108+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:29.111+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:30.159+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:30.173+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:21:30.175+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:30.175+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:30.180+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:30.209+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:30.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:30.216+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:39.217+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:39.233+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:39.233+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:39.236+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:40.347+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:21:40.401+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 2 2023-10-21T04:21:40.401+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:40.401+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:40.411+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:40.434+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:40.452+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:40.452+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:40.461+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:41.200+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:21:41.247+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:41.247+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:41.249+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:41.266+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:41.278+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:41.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:41.283+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:42.048+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:21:50.454+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:50.460+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:50.460+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:50.464+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:50.793+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:21:50.887+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:50.887+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:50.896+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:50.907+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:51.016+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:51.016+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:51.019+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:51.034+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:51.040+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 226] updated cluster rebalancing status to true 2023-10-21T04:21:51.040+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 226] updating rebalancing status to true 2023-10-21T04:21:51.043+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:51.043+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:51.045+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:51.458+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:21:51.525+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:21:51.525+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:51.525+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:51.527+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:51.596+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:51.645+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:51.645+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:51.649+05:30 [Info] switched currmeta from 632 -> 632 force true 2023-10-21T04:21:51.992+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9804447946734076, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:21:52.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9701432738709755, throttleDelayMs (new, old, change): (5, 3, 2) 2023-10-21T04:21:53.644+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 221] /regulator/report/s/index/217f03d04907d0ef82070fe4f41363c1 has 0-length value 2023-10-21T04:21:53.834+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {88b52e29e23a9bc9a1f18d56ee3fbbd4 [] topology-change-rebalance [{{bada28845870208460ab46b431032b01 7060000 } recovery-full} {{f22df9bc00cfefa0906def8153acae8a 7060000 } recovery-full}] [{4c6e4b64d0c01f9d6431a65ac36fc024 7060000 } {458c2b8d760618b2a2f6a998f4c93e74 7060000 }]} 2023-10-21T04:21:53.841+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 22.87µs 2023-10-21T04:21:53.841+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:21:53.841+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:21:53.841+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:21:53.851+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:21:53.851+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:21:53.851+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 10.171288ms 2023-10-21T04:21:53.851+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:21:53.851+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:21:53.851+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.666µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:53.851+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/88b52e29e23a9bc9a1f18d56ee3fbbd4 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:88b52e29e23a9bc9a1f18d56ee3fbbd4]}]} 2023-10-21T04:21:53.851+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[bada28845870208460ab46b431032b01 f22df9bc00cfefa0906def8153acae8a] IsBalanced:true Messages:[]} 2023-10-21T04:21:53.857+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:21:53.858+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:21:53.874+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:21:53.935+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:53.935+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:53.935+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:21:53.936+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:21:53.992+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9593497936995874, throttleDelayMs (new, old, change): (6, 5, 1) 2023-10-21T04:21:54.009+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.009+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.031+05:30 [Warn] watcher:ClientAuth cluster Ver (9)/Internal Version () not yet initialised 2023-10-21T04:21:54.038+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:21:54.163+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:21:54.163+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:21:54.171+05:30 [Warn] watcher:ClientAuth cluster Ver (9)/Internal Version () not yet initialised 2023-10-21T04:21:54.184+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:21:54.208+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.208+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.224+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57998 2023-10-21T04:21:54.267+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.44743ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:54.272+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 55.701µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:54.311+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:21:54.316+05:30 [Info] switched currmeta from 632 -> 634 force true 2023-10-21T04:21:54.324+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:21:54.325+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:21:54.324+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:54.327+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:54.326+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:21:54.330+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:21:54.344+05:30 [Info] switched currmeta from 634 -> 634 force true 2023-10-21T04:21:54.365+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:21:54.365+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:21:54.365+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:21:54.366+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:21:54.385+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:41382" doAuth() ... 2023-10-21T04:21:54.386+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:41382" doReceive() ... 2023-10-21T04:21:54.408+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.408+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.543+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58064 2023-10-21T04:21:54.565+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 12.338033ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:54.566+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 50.919µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:54.604+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:41416" doAuth() ... 2023-10-21T04:21:54.605+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:41416" doReceive() ... 2023-10-21T04:21:54.608+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.609+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.769+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:21:54.808+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.808+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:54.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8755344447150036, throttleDelayMs (new, old, change): (1, 6, -5) 2023-10-21T04:21:55.009+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.009+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.066+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:21:55.208+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.208+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.408+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.408+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.608+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.608+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.697+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {bada28845870208460ab46b431032b01 88b52e29e23a9bc9a1f18d56ee3fbbd4 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:21:55.700+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {bada28845870208460ab46b431032b01 88b52e29e23a9bc9a1f18d56ee3fbbd4 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:21:55.700+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"bada28845870208460ab46b431032b01","RebalId":"88b52e29e23a9bc9a1f18d56ee3fbbd4","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:21:55.701+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{bada28845870208460ab46b431032b01 88b52e29e23a9bc9a1f18d56ee3fbbd4 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:21:55.701+05:30 [Info] NewShardRebalancer nodeId 4c6e4b64d0c01f9d6431a65ac36fc024 rebalToken &{bada28845870208460ab46b431032b01 88b52e29e23a9bc9a1f18d56ee3fbbd4 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:21:55.702+05:30 [Info] ShardRebalancer::observeRebalance &{bada28845870208460ab46b431032b01 88b52e29e23a9bc9a1f18d56ee3fbbd4 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:21:55.706+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"bada28845870208460ab46b431032b01","RebalId":"88b52e29e23a9bc9a1f18d56ee3fbbd4","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:21:55.706+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:21:55.808+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.808+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:55.853+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{bada28845870208460ab46b431032b01 88b52e29e23a9bc9a1f18d56ee3fbbd4 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:21:55.915+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:42812 2023-10-21T04:21:55.924+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:42812 2023-10-21T04:21:55.948+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:42812 2023-10-21T04:21:55.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6388501859739636, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:21:56.008+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.008+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.149+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.151+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:21:56.151+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:21:56.151+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:21:56.151+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 56.586µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:56.195+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.208+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.208+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.274+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.372+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.380+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.395+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.396+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:21:56.396+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:21:56.396+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:21:56.396+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.233µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:56.407+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.407+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.442+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.447+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.526+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.527+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.559+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.608+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.608+05:30 [Warn] LifecycleMgr::canProcessDDL Disallowing DDL as rebalance is still in planning phase 2023-10-21T04:21:56.629+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.699+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"bada28845870208460ab46b431032b01","RebalId":"88b52e29e23a9bc9a1f18d56ee3fbbd4","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:21:56.699+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:21:56.699+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:21:56.699+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:21:56.699+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:21:56.699+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:21:56.699+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 13.951µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:56.748+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.748+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2864853750198994630:true 12177293282243154215:true] 2023-10-21T04:21:56.749+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.456µs 2023-10-21T04:21:56.749+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [2864853750198994630 12177293282243154215], isLockedForRecovery: false 2023-10-21T04:21:56.749+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [2864853750198994630 12177293282243154215], errMap: map[2864853750198994630: 12177293282243154215:], elapsed: 12.338µs 2023-10-21T04:21:56.749+05:30 [Info] StorageMgr::handleShardTransfer Updated book-keeping for shardIds: [2864853750198994630 12177293282243154215] 2023-10-21T04:21:56.749+05:30 [Info] ShardTransferManager::processShardTransferMessage Initiating command: ShardIds: [2864853750198994630 12177293282243154215] RebalanceId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 TransferTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Task Type: Rebalance IsPeerTransfer: false NewAlternateShardIds: [] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:21:56.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : initiating transfer for shardId 12177293282243154215, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215 2023-10-21T04:21:56.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : initiating transfer for shardId 2864853750198994630, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630 2023-10-21T04:21:56.767+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSS Snapshot SBHead :0 SBTail :7729152 2023-10-21T04:21:56.769+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:21:56.780+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSS Snapshot SBHead :0 SBTail :7176192 2023-10-21T04:21:56.783+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:21:56.803+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.803+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[7659339677706884510:true 10890719122371003479:true] 2023-10-21T04:21:56.803+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.577µs 2023-10-21T04:21:56.804+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [10890719122371003479 7659339677706884510], isLockedForRecovery: false 2023-10-21T04:21:56.804+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [10890719122371003479 7659339677706884510], errMap: map[7659339677706884510: 10890719122371003479:], elapsed: 12.512µs 2023-10-21T04:21:56.804+05:30 [Info] StorageMgr::handleShardTransfer Updated book-keeping for shardIds: [10890719122371003479 7659339677706884510] 2023-10-21T04:21:56.804+05:30 [Info] ShardTransferManager::processShardTransferMessage Initiating command: ShardIds: [10890719122371003479 7659339677706884510] RebalanceId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 TransferTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Task Type: Rebalance IsPeerTransfer: false NewAlternateShardIds: [] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:21:56.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : initiating transfer for shardId 7659339677706884510, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510 2023-10-21T04:21:56.806+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:56.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : initiating transfer for shardId 10890719122371003479, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479 2023-10-21T04:21:56.816+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSS Snapshot SBHead :0 SBTail :8658944 2023-10-21T04:21:56.830+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSS Snapshot SBHead :0 SBTail :8089600 2023-10-21T04:21:56.850+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:21:56.851+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:21:56.993+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:57.210+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery/log.00000000000000.data size:7729152 dur:441.133499ms 2023-10-21T04:21:57.211+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :7729152 current tail :7729152 2023-10-21T04:21:57.243+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery/log.00000000000000.data size:7176192 dur:460.176799ms 2023-10-21T04:21:57.243+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :7176192 current tail :7176192 2023-10-21T04:21:57.257+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery/log.00000000000000.data size:8089600 dur:407.113299ms 2023-10-21T04:21:57.257+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :8089600 current tail :8089600 2023-10-21T04:21:57.261+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery/log.00000000000000.data size:8658944 dur:410.591563ms 2023-10-21T04:21:57.262+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :8658944 current tail :8658944 2023-10-21T04:21:57.267+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSS Snapshot SBHead :0 SBTail :8040448 2023-10-21T04:21:57.287+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSS Snapshot SBHead :0 SBTail :7495680 2023-10-21T04:21:57.288+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSS Snapshot SBHead :0 SBTail :8417280 2023-10-21T04:21:57.311+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/log.00000000000000.data holdOff:0 2023-10-21T04:21:57.324+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSS Snapshot SBHead :0 SBTail :8970240 2023-10-21T04:21:57.345+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/log.00000000000000.data holdOff:0 2023-10-21T04:21:57.348+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/log.00000000000000.data holdOff:0 2023-10-21T04:21:57.393+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/log.00000000000000.data holdOff:0 2023-10-21T04:21:57.673+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/log.00000000000000.data size:8417280 dur:325.324494ms 2023-10-21T04:21:57.674+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data: log copy completed, snapshot head :0 current head :0 snapshot tail :8417280 current tail :8417280 2023-10-21T04:21:57.674+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/log.00000000000000.data size:8040448 dur:363.113852ms 2023-10-21T04:21:57.674+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data: log copy completed, snapshot head :0 current head :0 snapshot tail :8040448 current tail :8040448 2023-10-21T04:21:57.674+05:30 [Info] plasma shard transfer total bandwidth stats (upload), avgBW: 17890237 totalBytes: 16530579 numShards: 4 duration: 924.781521ms 2023-10-21T04:21:57.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : completed shardId 7659339677706884510 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510 took 869.74402ms size :16530579 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data size :8417280 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/log.00000000000000.data size :8417280 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510/data/log.00000000000000.data written :8417280 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery size :8089600 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery/log.00000000000000.data size :8089600 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510/data/recovery/log.00000000000000.data written :8089600 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/meta/shard.json size :7245 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510/meta/shard.json written :7245 retry :0 status :TransferComplete 2023-10-21T04:21:57.674+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 7659339677706884510, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510, err: , elapsed(sec): 0.87009871 2023-10-21T04:21:57.674+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 7659339677706884510 bytesWritten: 16530579, totalBytes: 16530579, transferRate: 1.9022530494821634e+07 2023-10-21T04:21:57.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : completed shardId 2864853750198994630 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630 took 920.280768ms size :15794118 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data size :8040448 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/log.00000000000000.data size :8040448 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630/data/log.00000000000000.data written :8040448 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery size :7729152 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery/log.00000000000000.data size :7729152 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630/data/recovery/log.00000000000000.data written :7729152 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/meta/shard.json size :8064 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630/meta/shard.json written :8064 retry :0 status :TransferComplete 2023-10-21T04:21:57.674+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 2864853750198994630, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630, err: , elapsed(sec): 0.925276021 2023-10-21T04:21:57.674+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 2864853750198994630 bytesWritten: 15794118, totalBytes: 15794118, transferRate: 1.716751956521739e+07 2023-10-21T04:21:57.675+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/log.00000000000000.data size:7495680 dur:330.381485ms 2023-10-21T04:21:57.675+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data: log copy completed, snapshot head :0 current head :0 snapshot tail :7495680 current tail :7495680 2023-10-21T04:21:57.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : completed shardId 12177293282243154215 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215 took 926.22066ms size :14695379 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data size :7495680 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/log.00000000000000.data size :7495680 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215/data/log.00000000000000.data written :7495680 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery size :7176192 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery/log.00000000000000.data size :7176192 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215/data/recovery/log.00000000000000.data written :7176192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/meta/shard.json size :7053 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215/meta/shard.json written :7053 retry :0 status :TransferComplete 2023-10-21T04:21:57.675+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 12177293282243154215, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215, err: , elapsed(sec): 0.926610986 2023-10-21T04:21:57.675+05:30 [Info] ShardTransferManager::processShardTransferMessage All shards processing done. Sending response errMap: map[2864853750198994630: 12177293282243154215:], shardPaths: map[2864853750198994630:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630 12177293282243154215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.92665234 2023-10-21T04:21:57.675+05:30 [Info] StorageMgr::ShardTransferResponse Clearing book-keeping for shardIds: [2864853750198994630 12177293282243154215] 2023-10-21T04:21:57.676+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 12177293282243154215 bytesWritten: 14695379, totalBytes: 14695379, transferRate: 1.5869739740820734e+07 2023-10-21T04:21:57.676+05:30 [Info] ShardRebalancer::startShardTransfer Received response for transfer of shards: [2864853750198994630 12177293282243154215], ttid: ShardTokendc:78:31:73:6:81:ad:f2, elapsed(sec): 0.927353257 2023-10-21T04:21:57.681+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/log.00000000000000.data size:8970240 dur:288.534388ms 2023-10-21T04:21:57.682+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data: log copy completed, snapshot head :0 current head :0 snapshot tail :8970240 current tail :8970240 2023-10-21T04:21:57.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : completed shardId 10890719122371003479 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479 took 875.933074ms size :17653908 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data size :8970240 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/log.00000000000000.data size :8970240 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479/data/log.00000000000000.data written :8970240 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery size :8658944 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery/log.00000000000000.data size :8658944 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479/data/recovery/log.00000000000000.data written :8658944 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/meta/shard.json size :8270 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479/meta/shard.json written :8270 retry :0 status :TransferComplete 2023-10-21T04:21:57.682+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 10890719122371003479, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479, err: , elapsed(sec): 0.878046424 2023-10-21T04:21:57.682+05:30 [Info] ShardTransferManager::processShardTransferMessage All shards processing done. Sending response errMap: map[7659339677706884510: 10890719122371003479:], shardPaths: map[7659339677706884510:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510 10890719122371003479:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.878079805 2023-10-21T04:21:57.682+05:30 [Info] StorageMgr::ShardTransferResponse Clearing book-keeping for shardIds: [10890719122371003479 7659339677706884510] 2023-10-21T04:21:57.684+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:57.715+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 10890719122371003479 bytesWritten: 17653908, totalBytes: 17653908, transferRate: 2.017589485714286e+07 2023-10-21T04:21:57.715+05:30 [Info] ShardRebalancer::startShardTransfer Received response for transfer of shards: [10890719122371003479 7659339677706884510], ttid: ShardToken62:b9:b5:7e:51:46:53:89, elapsed(sec): 0.912029236 2023-10-21T04:21:57.750+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:57.751+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:21:57.753+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:21:57.753+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:21:57.753+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:21:57.754+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:21:57.754+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:21:57.754+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 75.309µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:57.754+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 34.005µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:21:57.789+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:57.878+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:57.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.956161359596601, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:21:58.046+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:58.098+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:58.113+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:58.258+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:58.258+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:58.275+05:30 [Info] switched currmeta from 634 -> 637 force true 2023-10-21T04:21:58.402+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:21:58.444+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:58.445+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:58.539+05:30 [Info] switched currmeta from 637 -> 638 force true 2023-10-21T04:21:58.623+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:58.624+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:58.636+05:30 [Info] switched currmeta from 638 -> 638 force true 2023-10-21T04:21:59.202+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:59.203+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:59.206+05:30 [Info] switched currmeta from 638 -> 639 force true 2023-10-21T04:21:59.231+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:59.231+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:59.242+05:30 [Info] switched currmeta from 639 -> 640 force true 2023-10-21T04:21:59.309+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:59.309+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:59.316+05:30 [Info] switched currmeta from 640 -> 641 force true 2023-10-21T04:21:59.343+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:59.343+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:59.346+05:30 [Info] switched currmeta from 641 -> 642 force true 2023-10-21T04:21:59.604+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:59.604+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:59.609+05:30 [Info] switched currmeta from 642 -> 644 force true 2023-10-21T04:21:59.615+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:21:59.615+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:21:59.617+05:30 [Info] switched currmeta from 644 -> 644 force true 2023-10-21T04:21:59.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9597580620573871, throttleDelayMs (new, old, change): (2, 1, 1) 2023-10-21T04:22:00.057+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.057+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.060+05:30 [Info] switched currmeta from 644 -> 645 force true 2023-10-21T04:22:00.122+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.122+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.124+05:30 [Info] switched currmeta from 645 -> 646 force true 2023-10-21T04:22:00.169+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.169+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.172+05:30 [Info] switched currmeta from 646 -> 648 force true 2023-10-21T04:22:00.178+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.178+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.181+05:30 [Info] switched currmeta from 648 -> 648 force true 2023-10-21T04:22:00.353+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.353+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.356+05:30 [Info] switched currmeta from 648 -> 649 force true 2023-10-21T04:22:00.400+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.400+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.407+05:30 [Info] switched currmeta from 649 -> 650 force true 2023-10-21T04:22:00.521+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.521+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.529+05:30 [Info] switched currmeta from 650 -> 651 force true 2023-10-21T04:22:00.552+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.552+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.558+05:30 [Info] switched currmeta from 651 -> 652 force true 2023-10-21T04:22:00.656+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.656+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.683+05:30 [Info] switched currmeta from 652 -> 653 force true 2023-10-21T04:22:00.784+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.784+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.795+05:30 [Info] switched currmeta from 653 -> 654 force true 2023-10-21T04:22:00.968+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:00.979+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:00.979+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:00.990+05:30 [Info] switched currmeta from 654 -> 654 force true 2023-10-21T04:22:00.996+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9725394225137841, throttleDelayMs (new, old, change): (4, 2, 2) 2023-10-21T04:22:01.045+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.045+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.070+05:30 [Info] switched currmeta from 654 -> 657 force true 2023-10-21T04:22:01.085+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.086+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.101+05:30 [Info] switched currmeta from 657 -> 657 force true 2023-10-21T04:22:01.196+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.196+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.201+05:30 [Info] switched currmeta from 657 -> 658 force true 2023-10-21T04:22:01.225+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.225+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.241+05:30 [Info] switched currmeta from 658 -> 659 force true 2023-10-21T04:22:01.273+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.274+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.299+05:30 [Info] switched currmeta from 659 -> 660 force true 2023-10-21T04:22:01.457+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.457+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.470+05:30 [Info] switched currmeta from 660 -> 661 force true 2023-10-21T04:22:01.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.496+05:30 [Info] switched currmeta from 661 -> 663 force true 2023-10-21T04:22:01.519+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.519+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.522+05:30 [Info] switched currmeta from 663 -> 663 force true 2023-10-21T04:22:01.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:01.613+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:01.619+05:30 [Info] switched currmeta from 663 -> 664 force true 2023-10-21T04:22:01.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.972468997801901, throttleDelayMs (new, old, change): (6, 4, 2) 2023-10-21T04:22:02.186+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.186+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.195+05:30 [Info] switched currmeta from 664 -> 666 force true 2023-10-21T04:22:02.260+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.260+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.276+05:30 [Info] switched currmeta from 666 -> 667 force true 2023-10-21T04:22:02.329+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.329+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.340+05:30 [Info] switched currmeta from 667 -> 667 force true 2023-10-21T04:22:02.356+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.356+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.361+05:30 [Info] switched currmeta from 667 -> 668 force true 2023-10-21T04:22:02.469+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.469+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.473+05:30 [Info] switched currmeta from 668 -> 669 force true 2023-10-21T04:22:02.569+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.570+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.581+05:30 [Info] switched currmeta from 669 -> 672 force true 2023-10-21T04:22:02.603+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.604+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.612+05:30 [Info] switched currmeta from 672 -> 673 force true 2023-10-21T04:22:02.639+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.644+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.648+05:30 [Info] switched currmeta from 673 -> 673 force true 2023-10-21T04:22:02.708+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.708+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.712+05:30 [Info] switched currmeta from 673 -> 674 force true 2023-10-21T04:22:02.735+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:02.736+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:02.743+05:30 [Info] switched currmeta from 674 -> 675 force true 2023-10-21T04:22:02.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9586489458055272, throttleDelayMs (new, old, change): (7, 6, 1) 2023-10-21T04:22:03.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.127+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.146+05:30 [Info] switched currmeta from 675 -> 676 force true 2023-10-21T04:22:03.155+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 220] aggregate storage info: map[bucket_%2:map[kv:5814464 index:68413658] bucket_1:map[kv:6539456 index:61011522]] 2023-10-21T04:22:03.206+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.206+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.216+05:30 [Info] switched currmeta from 676 -> 678 force true 2023-10-21T04:22:03.252+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.252+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.279+05:30 [Info] switched currmeta from 678 -> 679 force true 2023-10-21T04:22:03.319+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.319+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.326+05:30 [Info] switched currmeta from 679 -> 681 force true 2023-10-21T04:22:03.374+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.374+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.390+05:30 [Info] switched currmeta from 681 -> 683 force true 2023-10-21T04:22:03.423+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.423+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.432+05:30 [Info] switched currmeta from 683 -> 683 force true 2023-10-21T04:22:03.458+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.458+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.461+05:30 [Info] switched currmeta from 683 -> 684 force true 2023-10-21T04:22:03.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.495+05:30 [Info] switched currmeta from 684 -> 686 force true 2023-10-21T04:22:03.504+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.504+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.547+05:30 [Info] switched currmeta from 686 -> 687 force true 2023-10-21T04:22:03.574+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.574+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.582+05:30 [Info] switched currmeta from 687 -> 688 force true 2023-10-21T04:22:03.614+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.614+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.617+05:30 [Info] switched currmeta from 688 -> 688 force true 2023-10-21T04:22:03.683+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.683+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.693+05:30 [Info] switched currmeta from 688 -> 690 force true 2023-10-21T04:22:03.706+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.706+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.712+05:30 [Info] switched currmeta from 690 -> 690 force true 2023-10-21T04:22:03.824+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.825+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.828+05:30 [Info] switched currmeta from 690 -> 691 force true 2023-10-21T04:22:03.840+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:03.840+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:03.845+05:30 [Info] switched currmeta from 691 -> 692 force true 2023-10-21T04:22:03.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9673889515434005, throttleDelayMs (new, old, change): (9, 7, 2) 2023-10-21T04:22:04.332+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:04.332+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:04.347+05:30 [Info] switched currmeta from 692 -> 693 force true 2023-10-21T04:22:04.382+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:04.382+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:04.385+05:30 [Info] switched currmeta from 693 -> 694 force true 2023-10-21T04:22:04.495+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:04.495+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:04.510+05:30 [Info] switched currmeta from 694 -> 696 force true 2023-10-21T04:22:04.520+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:04.520+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:04.525+05:30 [Info] switched currmeta from 696 -> 696 force true 2023-10-21T04:22:04.996+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9624338604530435, throttleDelayMs (new, old, change): (10, 9, 1) 2023-10-21T04:22:05.460+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.460+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.463+05:30 [Info] switched currmeta from 696 -> 698 force true 2023-10-21T04:22:05.469+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.469+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.474+05:30 [Info] switched currmeta from 698 -> 698 force true 2023-10-21T04:22:05.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.498+05:30 [Info] switched currmeta from 698 -> 699 force true 2023-10-21T04:22:05.533+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.533+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.547+05:30 [Info] switched currmeta from 699 -> 700 force true 2023-10-21T04:22:05.572+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9912003609070511414:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:05.572+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9912003609070511414:[5 6 7 8 1 2 3 4] 15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:05.622+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.622+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.657+05:30 [Info] switched currmeta from 700 -> 701 force true 2023-10-21T04:22:05.702+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.702+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.708+05:30 [Info] switched currmeta from 701 -> 703 force true 2023-10-21T04:22:05.717+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.717+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.722+05:30 [Info] switched currmeta from 703 -> 703 force true 2023-10-21T04:22:05.810+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.810+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.814+05:30 [Info] switched currmeta from 703 -> 705 force true 2023-10-21T04:22:05.826+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.826+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.834+05:30 [Info] switched currmeta from 705 -> 705 force true 2023-10-21T04:22:05.869+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.870+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.875+05:30 [Info] switched currmeta from 705 -> 706 force true 2023-10-21T04:22:05.937+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.937+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.946+05:30 [Info] switched currmeta from 706 -> 707 force true 2023-10-21T04:22:05.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9535252652174065 2023-10-21T04:22:05.995+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:05.996+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:05.998+05:30 [Info] switched currmeta from 707 -> 709 force true 2023-10-21T04:22:06.010+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.012+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.023+05:30 [Info] switched currmeta from 709 -> 709 force true 2023-10-21T04:22:06.037+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:06.037+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[4 5 6 7 8 1 2 3] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:06.066+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:06.066+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:06.066+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:06.231+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.231+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.241+05:30 [Info] switched currmeta from 709 -> 710 force true 2023-10-21T04:22:06.258+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:06.318+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.320+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.413+05:30 [Info] switched currmeta from 710 -> 713 force true 2023-10-21T04:22:06.436+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.436+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.442+05:30 [Info] switched currmeta from 713 -> 713 force true 2023-10-21T04:22:06.518+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8060737836868569583:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:06.519+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8060737836868569583:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:06.521+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5642987086862451891:[3 4 5 6 7 8 1 2] 8060737836868569583:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:06.700+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.700+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.704+05:30 [Info] switched currmeta from 713 -> 714 force true 2023-10-21T04:22:06.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.899+05:30 [Info] switched currmeta from 714 -> 715 force true 2023-10-21T04:22:06.947+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.947+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.951+05:30 [Info] switched currmeta from 715 -> 717 force true 2023-10-21T04:22:06.980+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.980+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:06.983+05:30 [Info] switched currmeta from 717 -> 717 force true 2023-10-21T04:22:06.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9728651697775604 2023-10-21T04:22:06.998+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:06.998+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:07.002+05:30 [Info] switched currmeta from 717 -> 718 force true 2023-10-21T04:22:07.197+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:07.197+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:07.202+05:30 [Info] switched currmeta from 718 -> 719 force true 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, c2%, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, _default, #primary_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, c1, idx_partitioned) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, _default, #primary_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, _default, idx_partitioned_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c2%, idx_partitioned) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, c1, #primary_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, _default, idx_partitioned) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, _default, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _system, _query, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c1, #primary_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c2%, #primary_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c1, idx_partitioned_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c2%, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, _default, idx_partitioned) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, c1, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c2%, idx_partitioned_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, _default, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, c2%, #primary_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _system, _query, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_%2, _default, _default, idx_partitioned_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, c1, idx_partitioned_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.500+05:30 [Warn] builder: Queuing index (bucket_1, _default, c2%, idx_partitioned) as bucket rebalance is in progress 2023-10-21T04:22:07.501+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c1, idx_partitioned) as bucket rebalance is in progress 2023-10-21T04:22:07.501+05:30 [Warn] builder: Queuing index (bucket_%2, _default, c1, #primary) as bucket rebalance is in progress 2023-10-21T04:22:07.501+05:30 [Warn] builder: Queuing index (bucket_1, _default, c2%, idx_partitioned_defer) as bucket rebalance is in progress 2023-10-21T04:22:07.576+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:07.576+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:07.617+05:30 [Info] switched currmeta from 719 -> 722 force true 2023-10-21T04:22:07.647+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:07.647+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:07.657+05:30 [Info] switched currmeta from 722 -> 722 force true 2023-10-21T04:22:07.722+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:07.722+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:07.748+05:30 [Info] switched currmeta from 722 -> 723 force true 2023-10-21T04:22:07.980+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:07.980+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:07.984+05:30 [Info] switched currmeta from 723 -> 724 force true 2023-10-21T04:22:08.007+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9985727153017631 2023-10-21T04:22:08.189+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:08.191+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:08.201+05:30 [Info] switched currmeta from 724 -> 725 force true 2023-10-21T04:22:08.288+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:08.289+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:08.294+05:30 [Info] switched currmeta from 725 -> 726 force true 2023-10-21T04:22:08.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:08.500+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:08.513+05:30 [Info] switched currmeta from 726 -> 727 force true 2023-10-21T04:22:08.587+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:08.587+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:08.591+05:30 [Info] switched currmeta from 727 -> 728 force true 2023-10-21T04:22:08.990+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:08.990+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:08.992+05:30 [Info] switched currmeta from 728 -> 729 force true 2023-10-21T04:22:08.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9767932046455586 2023-10-21T04:22:09.116+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:09.116+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:09.122+05:30 [Info] switched currmeta from 729 -> 730 force true 2023-10-21T04:22:09.226+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:09.233+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:09.235+05:30 [Info] switched currmeta from 730 -> 731 force true 2023-10-21T04:22:09.271+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:09.271+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:09.276+05:30 [Info] switched currmeta from 731 -> 732 force true 2023-10-21T04:22:09.344+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:09.344+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:09.350+05:30 [Info] switched currmeta from 732 -> 733 force true 2023-10-21T04:22:09.527+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:09.527+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:09.540+05:30 [Info] switched currmeta from 733 -> 734 force true 2023-10-21T04:22:09.606+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:09.606+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:09.609+05:30 [Info] switched currmeta from 734 -> 735 force true 2023-10-21T04:22:09.938+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:09.938+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:09.941+05:30 [Info] switched currmeta from 735 -> 736 force true 2023-10-21T04:22:09.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8233528222801413, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:22:10.039+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.039+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.052+05:30 [Info] switched currmeta from 736 -> 737 force true 2023-10-21T04:22:10.135+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.135+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.137+05:30 [Info] switched currmeta from 737 -> 739 force true 2023-10-21T04:22:10.146+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.146+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.156+05:30 [Info] switched currmeta from 739 -> 739 force true 2023-10-21T04:22:10.317+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.317+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.320+05:30 [Info] switched currmeta from 739 -> 740 force true 2023-10-21T04:22:10.359+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.359+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.371+05:30 [Info] switched currmeta from 740 -> 741 force true 2023-10-21T04:22:10.432+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.432+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.437+05:30 [Info] switched currmeta from 741 -> 742 force true 2023-10-21T04:22:10.461+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.461+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.471+05:30 [Info] switched currmeta from 742 -> 743 force true 2023-10-21T04:22:10.565+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:10.565+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:10.565+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:10.565+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:10.579+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.579+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.584+05:30 [Info] switched currmeta from 743 -> 744 force true 2023-10-21T04:22:10.631+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.631+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.634+05:30 [Info] switched currmeta from 744 -> 745 force true 2023-10-21T04:22:10.653+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.653+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.662+05:30 [Info] switched currmeta from 745 -> 746 force true 2023-10-21T04:22:10.689+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.689+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.700+05:30 [Info] switched currmeta from 746 -> 747 force true 2023-10-21T04:22:10.722+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.723+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.734+05:30 [Info] switched currmeta from 747 -> 749 force true 2023-10-21T04:22:10.743+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:10.744+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:10.747+05:30 [Info] switched currmeta from 749 -> 749 force true 2023-10-21T04:22:10.999+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7776032243252284, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:22:11.002+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:11.002+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:11.011+05:30 [Info] switched currmeta from 749 -> 750 force true 2023-10-21T04:22:11.019+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:11.066+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:11.076+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:11.099+05:30 [Info] switched currmeta from 750 -> 751 force true 2023-10-21T04:22:11.181+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:11.181+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:11.200+05:30 [Info] switched currmeta from 751 -> 752 force true 2023-10-21T04:22:11.324+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:11.334+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:11.334+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:11.337+05:30 [Info] switched currmeta from 752 -> 752 force true 2023-10-21T04:22:11.557+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:11.557+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:11.562+05:30 [Info] switched currmeta from 752 -> 753 force true 2023-10-21T04:22:11.778+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:11.778+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:11.783+05:30 [Info] switched currmeta from 753 -> 754 force true 2023-10-21T04:22:12.026+05:30 [Info] memstats {"Alloc":168640248, "TotalAlloc":7862401592, "Sys":397267288, "Lookups":0, "Mallocs":90990930,"Frees":90487164, "HeapAlloc":168640248, "HeapSys":363233280, "HeapIdle":162799616, "HeapInuse":200433664,"HeapReleased":62357504, "HeapObjects":503766,"MSpanInuse":1563360, "MSpanSys": 3100800, "StackInuse": 10059776,"GCSys":17384720, "LastGC":1697842273169683657,"PauseTotalNs":55574692, "PauseNs":[176543, 128763, 152474, 187821, 107085, 337161, 184761, 155616, 108252], "NumGC":59} 2023-10-21T04:22:12.050+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:22:12.131+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:12.131+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:12.135+05:30 [Info] switched currmeta from 754 -> 755 force true 2023-10-21T04:22:12.180+05:30 [Info] janitor: running cleanup. 2023-10-21T04:22:12.180+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[8669649359784409226 18323460706027642575 14889401051926000176 7392796230534034390 12140458060295645238 15945117906390072636 2048006213723315482 12195715103909237684 6985049432225870291 14741949230702299196 4860549750398470299 14292583894719865678 12027054384338714804 249887442666643345 4535197604682045916 5465835793930010953 15968368593314043432 18046371353728341033 4138047918192025493 8504686614261377811]} 2023-10-21T04:22:12.308+05:30 [Info] connected with 4 indexers 2023-10-21T04:22:12.308+05:30 [Info] client stats current counts: current: 52, not current: 0 2023-10-21T04:22:12.343+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:12.343+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:12.368+05:30 [Info] switched currmeta from 755 -> 758 force true 2023-10-21T04:22:12.436+05:30 [Info] num concurrent scans {0} 2023-10-21T04:22:12.436+05:30 [Info] average scan response {0 ms} 2023-10-21T04:22:12.563+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:12.563+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:12.670+05:30 [Info] switched currmeta from 758 -> 758 force true 2023-10-21T04:22:12.895+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:22:12.924+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:22:13.182+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:13.182+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:13.222+05:30 [Info] switched currmeta from 758 -> 760 force true 2023-10-21T04:22:13.392+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:13.392+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:13.462+05:30 [Info] switched currmeta from 760 -> 765 force true 2023-10-21T04:22:13.563+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:13.563+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:13.576+05:30 [Info] switched currmeta from 765 -> 765 force true 2023-10-21T04:22:13.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9640400435449423, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:22:14.322+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.325+05:30 [Info] switched currmeta from 765 -> 766 force true 2023-10-21T04:22:14.399+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.399+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.406+05:30 [Info] switched currmeta from 766 -> 768 force true 2023-10-21T04:22:14.417+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.417+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.422+05:30 [Info] switched currmeta from 768 -> 768 force true 2023-10-21T04:22:14.437+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.437+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.441+05:30 [Info] switched currmeta from 768 -> 769 force true 2023-10-21T04:22:14.553+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.553+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.558+05:30 [Info] switched currmeta from 769 -> 771 force true 2023-10-21T04:22:14.593+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.593+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.603+05:30 [Info] switched currmeta from 771 -> 771 force true 2023-10-21T04:22:14.904+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.904+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.907+05:30 [Info] switched currmeta from 771 -> 772 force true 2023-10-21T04:22:14.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:14.944+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:14.955+05:30 [Info] switched currmeta from 772 -> 773 force true 2023-10-21T04:22:14.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9867270229589197, throttleDelayMs (new, old, change): (5, 1, 4) 2023-10-21T04:22:15.169+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.169+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.175+05:30 [Info] switched currmeta from 773 -> 774 force true 2023-10-21T04:22:15.194+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.194+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.204+05:30 [Info] switched currmeta from 774 -> 775 force true 2023-10-21T04:22:15.332+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.332+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.348+05:30 [Info] switched currmeta from 775 -> 777 force true 2023-10-21T04:22:15.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.378+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.382+05:30 [Info] switched currmeta from 777 -> 778 force true 2023-10-21T04:22:15.390+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.390+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.395+05:30 [Info] switched currmeta from 778 -> 778 force true 2023-10-21T04:22:15.482+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.490+05:30 [Info] switched currmeta from 778 -> 780 force true 2023-10-21T04:22:15.498+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.498+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.508+05:30 [Info] switched currmeta from 780 -> 780 force true 2023-10-21T04:22:15.872+05:30 [Info] Indexer::ReadMemstats Time Taken 2.73889ms 2023-10-21T04:22:15.873+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.874+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.882+05:30 [Info] switched currmeta from 780 -> 781 force true 2023-10-21T04:22:15.908+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.908+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.916+05:30 [Info] switched currmeta from 781 -> 782 force true 2023-10-21T04:22:15.953+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.954+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.962+05:30 [Info] switched currmeta from 782 -> 783 force true 2023-10-21T04:22:15.988+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:15.988+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:15.991+05:30 [Info] switched currmeta from 783 -> 784 force true 2023-10-21T04:22:15.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9708138680784487, throttleDelayMs (new, old, change): (7, 5, 2) 2023-10-21T04:22:16.308+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.308+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.315+05:30 [Info] switched currmeta from 784 -> 786 force true 2023-10-21T04:22:16.339+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.339+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.341+05:30 [Info] switched currmeta from 786 -> 786 force true 2023-10-21T04:22:16.400+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:16.407+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.407+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.410+05:30 [Info] switched currmeta from 786 -> 786 force true 2023-10-21T04:22:16.487+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.487+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.498+05:30 [Info] switched currmeta from 786 -> 788 force true 2023-10-21T04:22:16.515+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.515+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.520+05:30 [Info] switched currmeta from 788 -> 788 force true 2023-10-21T04:22:16.545+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.545+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.549+05:30 [Info] switched currmeta from 788 -> 789 force true 2023-10-21T04:22:16.578+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5744899830895973442:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:16.578+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5744899830895973442:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:16.579+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5744899830895973442:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:16.580+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5744899830895973442:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:16.580+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5744899830895973442:[6 7 8 1 2 3 4 5] 7582778950442222621:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:16.582+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5744899830895973442:[6 7 8 1 2 3 4 5] 7582778950442222621:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:16.644+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.644+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.647+05:30 [Info] switched currmeta from 789 -> 790 force true 2023-10-21T04:22:16.696+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.696+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.701+05:30 [Info] switched currmeta from 790 -> 791 force true 2023-10-21T04:22:16.798+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.798+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.802+05:30 [Info] switched currmeta from 791 -> 792 force true 2023-10-21T04:22:16.850+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.850+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.853+05:30 [Info] switched currmeta from 792 -> 794 force true 2023-10-21T04:22:16.863+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.865+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.871+05:30 [Info] switched currmeta from 794 -> 794 force true 2023-10-21T04:22:16.928+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:16.928+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:16.938+05:30 [Info] switched currmeta from 794 -> 795 force true 2023-10-21T04:22:16.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9691135798446328, throttleDelayMs (new, old, change): (9, 7, 2) 2023-10-21T04:22:17.017+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.017+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.020+05:30 [Info] switched currmeta from 795 -> 796 force true 2023-10-21T04:22:17.033+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:17.033+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:17.033+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0] 12882995014846270330:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:17.490+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.490+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.493+05:30 [Info] switched currmeta from 796 -> 797 force true 2023-10-21T04:22:17.574+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.574+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.588+05:30 [Info] switched currmeta from 797 -> 798 force true 2023-10-21T04:22:17.611+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.611+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.615+05:30 [Info] switched currmeta from 798 -> 799 force true 2023-10-21T04:22:17.663+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.664+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.694+05:30 [Info] switched currmeta from 799 -> 802 force true 2023-10-21T04:22:17.714+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.714+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.725+05:30 [Info] switched currmeta from 802 -> 803 force true 2023-10-21T04:22:17.754+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.754+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.781+05:30 [Info] switched currmeta from 803 -> 803 force true 2023-10-21T04:22:17.822+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:17.822+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:17.827+05:30 [Info] switched currmeta from 803 -> 804 force true 2023-10-21T04:22:17.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9712976506239108, throttleDelayMs (new, old, change): (10, 9, 1) 2023-10-21T04:22:18.014+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.014+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.022+05:30 [Info] switched currmeta from 804 -> 805 force true 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0] 14579235200590321485:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.067+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13739166377675070800:[0] 14579235200590321485:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:18.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.124+05:30 [Info] switched currmeta from 805 -> 806 force true 2023-10-21T04:22:18.340+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.340+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.353+05:30 [Info] switched currmeta from 806 -> 808 force true 2023-10-21T04:22:18.368+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.368+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.379+05:30 [Info] switched currmeta from 808 -> 808 force true 2023-10-21T04:22:18.592+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.592+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.595+05:30 [Info] switched currmeta from 808 -> 810 force true 2023-10-21T04:22:18.605+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.605+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.613+05:30 [Info] switched currmeta from 810 -> 811 force true 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.613+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8852459821271515836:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:18.635+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.637+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.647+05:30 [Info] switched currmeta from 811 -> 812 force true 2023-10-21T04:22:18.670+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.670+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.679+05:30 [Info] switched currmeta from 812 -> 812 force true 2023-10-21T04:22:18.789+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.789+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.814+05:30 [Info] switched currmeta from 812 -> 815 force true 2023-10-21T04:22:18.826+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.826+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.839+05:30 [Info] switched currmeta from 815 -> 816 force true 2023-10-21T04:22:18.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.870+05:30 [Info] switched currmeta from 816 -> 817 force true 2023-10-21T04:22:18.885+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:18.885+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:18.912+05:30 [Info] switched currmeta from 817 -> 817 force true 2023-10-21T04:22:18.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9807160928489793 2023-10-21T04:22:19.019+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.019+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.019+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.020+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.020+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.020+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.020+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.020+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.020+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12510230312641977282:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:19.263+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.263+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.268+05:30 [Info] switched currmeta from 817 -> 818 force true 2023-10-21T04:22:19.331+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.331+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.334+05:30 [Info] switched currmeta from 818 -> 819 force true 2023-10-21T04:22:19.402+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.402+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.410+05:30 [Info] switched currmeta from 819 -> 820 force true 2023-10-21T04:22:19.431+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.431+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.442+05:30 [Info] switched currmeta from 820 -> 821 force true 2023-10-21T04:22:19.685+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.685+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.691+05:30 [Info] switched currmeta from 821 -> 822 force true 2023-10-21T04:22:19.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.768+05:30 [Info] switched currmeta from 822 -> 823 force true 2023-10-21T04:22:19.801+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.801+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.804+05:30 [Info] switched currmeta from 823 -> 824 force true 2023-10-21T04:22:19.867+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.867+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.872+05:30 [Info] switched currmeta from 824 -> 825 force true 2023-10-21T04:22:19.924+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:19.924+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:19.932+05:30 [Info] switched currmeta from 825 -> 826 force true 2023-10-21T04:22:19.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9541596188528977 2023-10-21T04:22:20.020+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.021+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.026+05:30 [Info] switched currmeta from 826 -> 827 force true 2023-10-21T04:22:20.291+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.291+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.296+05:30 [Info] switched currmeta from 827 -> 828 force true 2023-10-21T04:22:20.338+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.339+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.343+05:30 [Info] switched currmeta from 828 -> 829 force true 2023-10-21T04:22:20.469+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.471+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.483+05:30 [Info] switched currmeta from 829 -> 830 force true 2023-10-21T04:22:20.765+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.765+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.770+05:30 [Info] switched currmeta from 830 -> 832 force true 2023-10-21T04:22:20.777+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.777+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.781+05:30 [Info] switched currmeta from 832 -> 832 force true 2023-10-21T04:22:20.817+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.817+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.833+05:30 [Info] switched currmeta from 832 -> 834 force true 2023-10-21T04:22:20.857+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.858+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.864+05:30 [Info] switched currmeta from 834 -> 835 force true 2023-10-21T04:22:20.972+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:20.972+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:20.977+05:30 [Info] switched currmeta from 835 -> 836 force true 2023-10-21T04:22:20.994+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8834482434689103, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:22:21.016+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:21.016+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:21.059+05:30 [Info] switched currmeta from 836 -> 838 force true 2023-10-21T04:22:21.110+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:21.110+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:21.128+05:30 [Info] switched currmeta from 838 -> 839 force true 2023-10-21T04:22:21.148+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:21.149+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:21.208+05:30 [Info] switched currmeta from 839 -> 840 force true 2023-10-21T04:22:21.225+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:21.225+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:21.246+05:30 [Info] switched currmeta from 840 -> 840 force true 2023-10-21T04:22:21.320+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:21.320+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:21.326+05:30 [Info] switched currmeta from 840 -> 841 force true 2023-10-21T04:22:21.795+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:21.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:21.838+05:30 [Info] switched currmeta from 841 -> 846 force true 2023-10-21T04:22:21.987+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:21.987+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:21.991+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:21.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9203884565593391, throttleDelayMs (new, old, change): (2, 5, -3) 2023-10-21T04:22:22.002+05:30 [Info] switched currmeta from 846 -> 847 force true 2023-10-21T04:22:22.438+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.438+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.455+05:30 [Info] switched currmeta from 847 -> 853 force true 2023-10-21T04:22:22.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.463+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.471+05:30 [Info] switched currmeta from 853 -> 853 force true 2023-10-21T04:22:22.627+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.627+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.660+05:30 [Info] switched currmeta from 853 -> 855 force true 2023-10-21T04:22:22.680+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.689+05:30 [Info] switched currmeta from 855 -> 855 force true 2023-10-21T04:22:22.842+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.842+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.850+05:30 [Info] switched currmeta from 855 -> 856 force true 2023-10-21T04:22:22.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.899+05:30 [Info] switched currmeta from 856 -> 858 force true 2023-10-21T04:22:22.908+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.908+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.911+05:30 [Info] switched currmeta from 858 -> 858 force true 2023-10-21T04:22:22.979+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:22.981+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:22.987+05:30 [Info] switched currmeta from 858 -> 860 force true 2023-10-21T04:22:22.995+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9843001891076965, throttleDelayMs (new, old, change): (5, 2, 3) 2023-10-21T04:22:23.011+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.011+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.024+05:30 [Info] switched currmeta from 860 -> 860 force true 2023-10-21T04:22:23.137+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.137+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.143+05:30 [Info] switched currmeta from 860 -> 861 force true 2023-10-21T04:22:23.173+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.173+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.184+05:30 [Info] switched currmeta from 861 -> 862 force true 2023-10-21T04:22:23.310+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.310+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.316+05:30 [Info] switched currmeta from 862 -> 863 force true 2023-10-21T04:22:23.337+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.337+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.340+05:30 [Info] switched currmeta from 863 -> 864 force true 2023-10-21T04:22:23.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.664+05:30 [Info] switched currmeta from 864 -> 865 force true 2023-10-21T04:22:23.672+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.672+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.679+05:30 [Info] switched currmeta from 865 -> 866 force true 2023-10-21T04:22:23.806+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.806+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.812+05:30 [Info] switched currmeta from 866 -> 867 force true 2023-10-21T04:22:23.842+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:23.842+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:23.852+05:30 [Info] switched currmeta from 867 -> 868 force true 2023-10-21T04:22:23.858+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:22:23.858+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[bada28845870208460ab46b431032b01 f22df9bc00cfefa0906def8153acae8a] IsBalanced:true Messages:[]} 2023-10-21T04:22:23.861+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:22:23.863+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. err: operation_canceled 2023-10-21T04:22:23.863+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:22:23.863+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/88b52e29e23a9bc9a1f18d56ee3fbbd4 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:88b52e29e23a9bc9a1f18d56ee3fbbd4]}]} 2023-10-21T04:22:23.866+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:22:24.067+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.068+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.071+05:30 [Info] switched currmeta from 868 -> 870 force true 2023-10-21T04:22:24.076+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.076+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.080+05:30 [Info] switched currmeta from 870 -> 870 force true 2023-10-21T04:22:24.108+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.108+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.112+05:30 [Info] switched currmeta from 870 -> 871 force true 2023-10-21T04:22:24.154+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.154+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.157+05:30 [Info] switched currmeta from 871 -> 872 force true 2023-10-21T04:22:24.315+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.319+05:30 [Info] switched currmeta from 872 -> 873 force true 2023-10-21T04:22:24.409+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.409+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.426+05:30 [Info] switched currmeta from 873 -> 876 force true 2023-10-21T04:22:24.435+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:24.447+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.448+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.450+05:30 [Info] switched currmeta from 876 -> 876 force true 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[2 3 4 5 6 7 8 1] 7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[2 3 4 5 6 7 8 1] 7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[2 3 4 5 6 7 8 1] 7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[2 3 4 5 6 7 8 1] 7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[2 3 4 5 6 7 8 1] 7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[2 3 4 5 6 7 8 1] 7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.567+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4440544141961093904:[2 3 4 5 6 7 8 1] 7494249569853685720:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:24.845+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.845+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.850+05:30 [Info] switched currmeta from 876 -> 877 force true 2023-10-21T04:22:24.861+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.861+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.864+05:30 [Info] switched currmeta from 877 -> 878 force true 2023-10-21T04:22:24.993+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:24.993+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:24.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9006590689103254, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:22:24.999+05:30 [Info] switched currmeta from 878 -> 880 force true 2023-10-21T04:22:25.021+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:25.022+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:25.025+05:30 [Info] switched currmeta from 880 -> 880 force true 2023-10-21T04:22:25.062+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:25.069+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.069+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.069+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.069+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.069+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1265729768899070816:[0] 17357232428819395534:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:22:25.191+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:25.191+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:25.194+05:30 [Info] switched currmeta from 880 -> 882 force true 2023-10-21T04:22:25.199+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:25.199+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:25.202+05:30 [Info] switched currmeta from 882 -> 882 force true 2023-10-21T04:22:25.538+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:25.538+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:25.544+05:30 [Info] switched currmeta from 882 -> 883 force true 2023-10-21T04:22:25.553+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:25.554+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:25.566+05:30 [Info] switched currmeta from 883 -> 884 force true 2023-10-21T04:22:25.619+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:25.619+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:25.625+05:30 [Info] switched currmeta from 884 -> 885 force true 2023-10-21T04:22:25.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:25.686+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:25.692+05:30 [Info] switched currmeta from 885 -> 886 force true 2023-10-21T04:22:26.017+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:26.017+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:26.017+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:26.017+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 {Index inst : [partitions]}: map[2311200026499605920:[0] 6919182200795062103:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:22:26.018+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2311200026499605920:[0] 6919182200795062103:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 {Index inst : [partitions]}: map[11304328130226754712:[0] 11863658526331737681:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:26.660+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:26.661+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:26.665+05:30 [Info] switched currmeta from 886 -> 887 force true 2023-10-21T04:22:26.711+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:26.711+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:26.713+05:30 [Info] switched currmeta from 887 -> 888 force true 2023-10-21T04:22:26.844+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:26.849+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:26.849+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:26.852+05:30 [Info] switched currmeta from 888 -> 888 force true 2023-10-21T04:22:27.593+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:27.593+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:27.598+05:30 [Info] switched currmeta from 888 -> 889 force true 2023-10-21T04:22:28.100+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:28.100+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:28.102+05:30 [Info] switched currmeta from 889 -> 890 force true 2023-10-21T04:22:28.600+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:28.600+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:28.605+05:30 [Info] switched currmeta from 890 -> 891 force true 2023-10-21T04:22:29.114+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:29.114+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:29.118+05:30 [Info] switched currmeta from 891 -> 892 force true 2023-10-21T04:22:29.206+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:29.206+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:29.209+05:30 [Info] switched currmeta from 892 -> 893 force true 2023-10-21T04:22:29.238+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:29.290+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken51:b0:f9:2b:92:f4:ae:dc MasterId: SourceId: DestId: RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokendc:78:31:73:6:81:ad:f2 SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:29.290+05:30 [Info] ShardRebalancer::checkAndQueueTokenForDrop Queuing token: ShardTokendc:78:31:73:6:81:ad:f2 for drop 2023-10-21T04:22:29.290+05:30 [Info] ShardRebalancer::queueDropShardRequests: Queued ttid: ShardTokendc:78:31:73:6:81:ad:f2 for drop 2023-10-21T04:22:29.290+05:30 [Info] ShardRebalancer::initiateShardTransferCleanup Initiating clean-up for ttid: ShardTokendc:78:31:73:6:81:ad:f2, destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: 2023-10-21T04:22:29.290+05:30 [Info] ShardRebalancer::processDropShards processing drop shards request for ttid: ShardTokendc:78:31:73:6:81:ad:f2 2023-10-21T04:22:29.290+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 TransferTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:22:29.290+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000148818 2023-10-21T04:22:29.290+05:30 [Info] ShardRebalancer::initiateShardTransferCleanup Done clean-up for ttid: ShardTokendc:78:31:73:6:81:ad:f2, shard paths: map[2864853750198994630:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630 12177293282243154215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.000285665 2023-10-21T04:22:29.291+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2, shardId :0 2023-10-21T04:22:29.291+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2, shardId :0 2023-10-21T04:22:29.574+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.576+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:22:29.747+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:29.747+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:29.750+05:30 [Info] switched currmeta from 893 -> 894 force true 2023-10-21T04:22:29.806+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken30:b4:a5:29:32:8b:e6:df MasterId: SourceId: DestId: RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken80:25:58:aa:f5:3d:42:3a SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:29.806+05:30 [Info] ShardRebalancer::checkAndQueueTokenForDrop Queuing token: ShardToken62:b9:b5:7e:51:46:53:89 for drop 2023-10-21T04:22:29.806+05:30 [Info] ShardRebalancer::queueDropShardRequests: Queued ttid: ShardToken62:b9:b5:7e:51:46:53:89 for drop 2023-10-21T04:22:29.806+05:30 [Info] ShardRebalancer::initiateShardTransferCleanup Initiating clean-up for ttid: ShardToken62:b9:b5:7e:51:46:53:89, destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: 2023-10-21T04:22:29.806+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 TransferTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:22:29.806+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000126669 2023-10-21T04:22:29.806+05:30 [Info] ShardRebalancer::initiateShardTransferCleanup Done clean-up for ttid: ShardToken62:b9:b5:7e:51:46:53:89, shard paths: map[7659339677706884510:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510 10890719122371003479:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.000246318 2023-10-21T04:22:29.808+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89, shardId :0 2023-10-21T04:22:29.808+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89, shardId :0 2023-10-21T04:22:29.809+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:22:30.290+05:30 [Info] ShardRebalancer::processDropShards processing drop shards request for ttid: ShardToken62:b9:b5:7e:51:46:53:89 2023-10-21T04:22:30.296+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 17656144409930763141 instance id 12652336003508392720 real instance id 7315818257769233877 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:30.296+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 17656144409930763141 instance 7315818257769233877 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:30.296+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 17656144409930763141 instance id 7315818257769233877 2023-10-21T04:22:30.296+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 17656144409930763141 2023-10-21T04:22:30.359+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 7315818257769233877 &{1} 2023-10-21T04:22:30.359+05:30 [Info] Indexer::handleDropIndex - IndexInstId 7315818257769233877 2023-10-21T04:22:30.359+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17656144409930763141, instId: 7315818257769233877 2023-10-21T04:22:30.359+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [7315818257769233877] 2023-10-21T04:22:30.359+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [7315818257769233877] 2023-10-21T04:22:30.360+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:30.360+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [7315818257769233877] 2023-10-21T04:22:30.360+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 7, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.363+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.364+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 8, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:22:30.364+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:22:30.364+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:22:30.364+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:22:30.364+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:22:30.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex stopped 2023-10-21T04:22:30.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:22:30.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex closed 2023-10-21T04:22:30.375+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 1, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.375+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.375+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.375+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:22:30.380+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:22:30.375+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 2, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.376+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 3, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:22:30.376+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 4, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.376+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:22:30.377+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 5, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.377+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 6, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.375+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.380+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:22:30.382+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.382+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.382+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.385+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 26.039823ms 2023-10-21T04:22:30.385+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 26.079966ms 2023-10-21T04:22:30.385+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 7315818257769233877 2023-10-21T04:22:30.394+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.433+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.433+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:22:30.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.434+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.434+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.438+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:22:30.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:22:30.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:22:30.440+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.440+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:22:30.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:22:30.440+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:22:30.442+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:22:30.442+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:22:30.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex stopped 2023-10-21T04:22:30.442+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.442+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.442+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.442+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.447+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:22:30.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex closed 2023-10-21T04:22:30.449+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.449+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:22:30.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 146 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:22:30.451+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:22:30.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex stopped 2023-10-21T04:22:30.451+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.451+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.451+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.451+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.451+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:22:30.451+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:22:30.451+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:22:30.451+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:22:30.452+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:22:30.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex stopped 2023-10-21T04:22:30.452+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.452+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.452+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.452+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.452+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:22:30.452+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:22:30.452+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:22:30.452+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:22:30.452+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:22:30.452+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:22:30.452+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:22:30.452+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:22:30.453+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:22:30.453+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:22:30.453+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:22:30.454+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:22:30.454+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:22:30.454+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:22:30.454+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:22:30.454+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:22:30.454+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:22:30.454+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:22:30.454+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:22:30.454+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:22:30.454+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:22:30.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex stopped 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex stopped 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex stopped 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex stopped 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex stopped 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.471+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:30.471+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:30.482+05:30 [Info] switched currmeta from 894 -> 897 force true 2023-10-21T04:22:30.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:30.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... 2023-10-21T04:22:30.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 162 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... 2023-10-21T04:22:30.493+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 197.508734ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:30.507+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 15991242037348127852 instance id 16489953270406880971 real instance id 14579235200590321485 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:30.507+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 15991242037348127852 instance 14579235200590321485 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:30.507+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 15991242037348127852 instance id 14579235200590321485 2023-10-21T04:22:30.508+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 15991242037348127852 2023-10-21T04:22:30.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index destroyed 2023-10-21T04:22:30.529+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 8, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.529+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 8 Destroy Done 2023-10-21T04:22:30.570+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 14579235200590321485 &{1} 2023-10-21T04:22:30.570+05:30 [Info] Indexer::handleDropIndex - IndexInstId 14579235200590321485 2023-10-21T04:22:30.570+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 15991242037348127852, instId: 14579235200590321485 2023-10-21T04:22:30.571+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:30.571+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:30.573+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14579235200590321485 Name: idx_partitioned Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:30.599+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:22:30.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex closed 2023-10-21T04:22:30.599+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.599+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.601+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [14579235200590321485] 2023-10-21T04:22:30.605+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:22:30.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex closed 2023-10-21T04:22:30.605+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [14579235200590321485] 2023-10-21T04:22:30.607+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:30.613+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.614+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.601+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.615+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:22:30.616+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:22:30.617+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:30.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:30.617+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [14579235200590321485] 2023-10-21T04:22:30.625+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.617+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:22:30.615+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.627+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:22:30.627+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.632+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.632+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex] 2023-10-21T04:22:30.627+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:22:30.627+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.627+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.627+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.628+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.628+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.628+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.629+05:30 [Info] switched currmeta from 897 -> 900 force true 2023-10-21T04:22:30.632+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.628+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:30.635+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4, IndexDefnId 15991242037348127852. 2023-10-21T04:22:30.638+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex 2023-10-21T04:22:30.638+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:22:30.638+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.638+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.641+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:22:30.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex closed 2023-10-21T04:22:30.642+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14579235200590321485 Defn: DefnId: 15991242037348127852 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:30.644+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 73.939456ms 2023-10-21T04:22:30.644+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.644+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.644+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex] 2023-10-21T04:22:30.645+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex] 2023-10-21T04:22:30.645+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex] 2023-10-21T04:22:30.645+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex 2023-10-21T04:22:30.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex stopped 2023-10-21T04:22:30.645+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.645+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.645+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.645+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex stopped 2023-10-21T04:22:30.646+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.646+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.646+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.647+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.647+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:22:30.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex closed 2023-10-21T04:22:30.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex] 2023-10-21T04:22:30.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex] 2023-10-21T04:22:30.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex] 2023-10-21T04:22:30.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex] 2023-10-21T04:22:30.647+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex] 2023-10-21T04:22:30.647+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 14579235200590321485 2023-10-21T04:22:30.647+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:22:30.651+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:22:30.651+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:22:30.657+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:22:30.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex stopped 2023-10-21T04:22:30.657+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.652+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [14579235200590321485] 2023-10-21T04:22:30.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.642+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.660+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.660+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.661+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:22:30.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex closed 2023-10-21T04:22:30.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex] 2023-10-21T04:22:30.657+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex 2023-10-21T04:22:30.673+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex] 2023-10-21T04:22:30.673+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex] 2023-10-21T04:22:30.673+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex 2023-10-21T04:22:30.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex stopped 2023-10-21T04:22:30.673+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.673+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.673+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.673+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.665+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:30.676+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:22:30.679+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:22:30.679+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:22:30.679+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:22:30.679+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:22:30.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex stopped 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.679+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.680+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex 2023-10-21T04:22:30.680+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex] 2023-10-21T04:22:30.680+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex] 2023-10-21T04:22:30.668+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:22:30.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex closed 2023-10-21T04:22:30.686+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex 2023-10-21T04:22:30.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex stopped 2023-10-21T04:22:30.691+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.691+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.691+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.691+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.692+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:22:30.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex closed 2023-10-21T04:22:30.688+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.693+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.693+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.694+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:22:30.688+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.689+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:30.690+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex 2023-10-21T04:22:30.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.695+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:22:30.699+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 54.622512ms 2023-10-21T04:22:30.695+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex] 2023-10-21T04:22:30.699+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.699+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:22:30.702+05:30 [Info] switched currmeta from 900 -> 901 force true 2023-10-21T04:22:30.704+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.704+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 133.736386ms 2023-10-21T04:22:30.704+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.704+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.712+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 205.365595ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:30.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:22:30.740+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 1 Close Done 2023-10-21T04:22:30.740+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:22:30.740+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex] 2023-10-21T04:22:30.743+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.743+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.744+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 573521267164877540 instance id 17196107967903578507 real instance id 9912003609070511414 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:30.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex closed 2023-10-21T04:22:30.748+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.748+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.748+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.748+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:22:30.748+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.748+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:22:30.749+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:22:30.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex stopped 2023-10-21T04:22:30.749+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.752+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.752+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.752+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.748+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 573521267164877540 instance 9912003609070511414 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:30.753+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 573521267164877540 instance id 9912003609070511414 2023-10-21T04:22:30.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:22:30.748+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.748+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex 2023-10-21T04:22:30.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:22:30.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex closed 2023-10-21T04:22:30.769+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:22:30.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex closed 2023-10-21T04:22:30.757+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 573521267164877540 2023-10-21T04:22:30.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex stopped 2023-10-21T04:22:30.757+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex 2023-10-21T04:22:30.768+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.775+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:22:30.775+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:22:30.775+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:22:30.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex stopped 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:22:30.775+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:22:30.775+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:22:30.775+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:22:30.775+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:22:30.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex stopped 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.779+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex] 2023-10-21T04:22:30.779+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex] 2023-10-21T04:22:30.779+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex 2023-10-21T04:22:30.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex stopped 2023-10-21T04:22:30.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.780+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex] 2023-10-21T04:22:30.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex closed 2023-10-21T04:22:30.780+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex 2023-10-21T04:22:30.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:22:30.780+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex] 2023-10-21T04:22:30.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 140 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:22:30.780+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex] 2023-10-21T04:22:30.780+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex 2023-10-21T04:22:30.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex stopped 2023-10-21T04:22:30.780+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.780+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.780+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.780+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.783+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.783+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.783+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex 2023-10-21T04:22:30.783+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex] 2023-10-21T04:22:30.783+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex] 2023-10-21T04:22:30.783+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex 2023-10-21T04:22:30.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex stopped 2023-10-21T04:22:30.783+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.783+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.783+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.783+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.784+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.784+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.784+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.784+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:22:30.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex] 2023-10-21T04:22:30.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex closed 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex] 2023-10-21T04:22:30.784+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex] 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex] 2023-10-21T04:22:30.784+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex 2023-10-21T04:22:30.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex stopped 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.784+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.785+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex 2023-10-21T04:22:30.785+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex] 2023-10-21T04:22:30.785+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex] 2023-10-21T04:22:30.785+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex 2023-10-21T04:22:30.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex stopped 2023-10-21T04:22:30.785+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.785+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.785+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.785+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex] 2023-10-21T04:22:30.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex closed 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex] 2023-10-21T04:22:30.786+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex] 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex] 2023-10-21T04:22:30.786+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex 2023-10-21T04:22:30.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex stopped 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex] 2023-10-21T04:22:30.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex closed 2023-10-21T04:22:30.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex] 2023-10-21T04:22:30.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex closed 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex] 2023-10-21T04:22:30.787+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex] 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex] 2023-10-21T04:22:30.787+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex 2023-10-21T04:22:30.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex stopped 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.787+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.788+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.788+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.786+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.788+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex] 2023-10-21T04:22:30.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex] 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex] 2023-10-21T04:22:30.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex 2023-10-21T04:22:30.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex stopped 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex] 2023-10-21T04:22:30.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex] 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex] 2023-10-21T04:22:30.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex 2023-10-21T04:22:30.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex stopped 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.794+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.794+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex] 2023-10-21T04:22:30.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex closed 2023-10-21T04:22:30.797+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.797+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex] 2023-10-21T04:22:30.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex closed 2023-10-21T04:22:30.798+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.798+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.798+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.798+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex] 2023-10-21T04:22:30.798+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex 2023-10-21T04:22:30.798+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex] 2023-10-21T04:22:30.798+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex] 2023-10-21T04:22:30.799+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex 2023-10-21T04:22:30.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex stopped 2023-10-21T04:22:30.799+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.799+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.799+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.799+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.806+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.806+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.807+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.807+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex] 2023-10-21T04:22:30.810+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex 2023-10-21T04:22:30.810+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex] 2023-10-21T04:22:30.810+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex] 2023-10-21T04:22:30.810+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex 2023-10-21T04:22:30.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex stopped 2023-10-21T04:22:30.810+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.810+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.810+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.810+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:30.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:22:30.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex closed 2023-10-21T04:22:30.815+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:22:30.815+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:22:30.815+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:22:30.815+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:22:30.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:22:30.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex closed 2023-10-21T04:22:30.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:22:30.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex closed 2023-10-21T04:22:30.816+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.816+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:22:30.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 145 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:22:30.814+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:30.817+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:30.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:22:30.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 156 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:22:30.818+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.819+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex stopped 2023-10-21T04:22:30.819+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.820+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.820+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.820+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.821+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.847+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex] 2023-10-21T04:22:30.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex closed 2023-10-21T04:22:30.867+05:30 [Info] switched currmeta from 901 -> 903 force true 2023-10-21T04:22:30.869+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 9912003609070511414 &{1} 2023-10-21T04:22:30.869+05:30 [Info] Indexer::handleDropIndex - IndexInstId 9912003609070511414 2023-10-21T04:22:30.870+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 573521267164877540, instId: 9912003609070511414 2023-10-21T04:22:30.870+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:30.870+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:30.870+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:30.871+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [9912003609070511414] 2023-10-21T04:22:30.871+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [9912003609070511414] 2023-10-21T04:22:30.871+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:30.871+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [9912003609070511414] 2023-10-21T04:22:30.872+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex] 2023-10-21T04:22:30.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex closed 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex] 2023-10-21T04:22:30.875+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex] 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex] 2023-10-21T04:22:30.875+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex 2023-10-21T04:22:30.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex stopped 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.875+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:30.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:22:30.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 139 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:22:30.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index destroyed 2023-10-21T04:22:30.890+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.890+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.890+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.890+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:22:30.890+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:22:30.890+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:22:30.890+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:22:30.890+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:22:30.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex stopped 2023-10-21T04:22:30.890+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.890+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.890+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.890+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.891+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:22:30.891+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:22:30.891+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:22:30.891+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:22:30.891+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:22:30.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex stopped 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.891+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:22:30.891+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:22:30.891+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:22:30.891+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:22:30.891+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:22:30.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex stopped 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.891+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.892+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:22:30.892+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:22:30.892+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:22:30.892+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:22:30.892+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:22:30.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex stopped 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.892+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.892+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:30.892+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:30.893+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 23.062285ms 2023-10-21T04:22:30.893+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:22:30.893+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:22:30.893+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:22:30.893+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:22:30.893+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:22:30.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex stopped 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.893+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:22:30.893+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:22:30.893+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:22:30.893+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:22:30.893+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:22:30.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex stopped 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.893+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.893+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 9912003609070511414 2023-10-21T04:22:30.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:22:30.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 161 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:22:30.901+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 2, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.901+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 2 Destroy Done 2023-10-21T04:22:30.901+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex] 2023-10-21T04:22:30.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex closed 2023-10-21T04:22:30.902+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.902+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.915+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.915+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index destroyed 2023-10-21T04:22:30.922+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 7, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.922+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 7 Destroy Done 2023-10-21T04:22:30.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:22:30.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex closed 2023-10-21T04:22:30.923+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.923+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.923+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.923+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:22:30.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:22:30.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex closed 2023-10-21T04:22:30.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.925+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.925+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.925+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:22:30.890+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.928+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.928+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.890+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1, IndexDefnId 573521267164877540. 2023-10-21T04:22:30.889+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.932+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.932+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:22:30.933+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:22:30.933+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:22:30.933+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:22:30.933+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:22:30.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:30.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... 2023-10-21T04:22:30.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 155 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... 2023-10-21T04:22:30.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex stopped 2023-10-21T04:22:30.934+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.944+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:30.956+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:30.944+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex] 2023-10-21T04:22:30.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex closed 2023-10-21T04:22:30.961+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.961+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.962+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex] 2023-10-21T04:22:30.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex closed 2023-10-21T04:22:30.963+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.963+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.963+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [9912003609070511414] 2023-10-21T04:22:30.965+05:30 [Info] switched currmeta from 903 -> 904 force true 2023-10-21T04:22:30.969+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex] 2023-10-21T04:22:30.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex closed 2023-10-21T04:22:30.969+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.970+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.971+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.973+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:30.973+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:22:30.973+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 80.463404ms 2023-10-21T04:22:30.973+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 103.757337ms 2023-10-21T04:22:30.974+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex] 2023-10-21T04:22:30.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex closed 2023-10-21T04:22:30.975+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.976+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:22:30.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 142 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:22:30.934+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.980+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.980+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:22:30.980+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex] 2023-10-21T04:22:30.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex closed 2023-10-21T04:22:30.980+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.981+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:30.952+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:30.981+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:30.981+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:30.981+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:30.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:30.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index destroyed 2023-10-21T04:22:30.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:22:30.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex closed 2023-10-21T04:22:30.984+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.985+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.986+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.987+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:22:30.986+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:22:30.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex closed 2023-10-21T04:22:30.988+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.988+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.986+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 1, IndexDefnId 17656144409930763141. 2023-10-21T04:22:30.988+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 1 Destroy Done 2023-10-21T04:22:30.986+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:30.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:22:30.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex closed 2023-10-21T04:22:30.989+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:30.989+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:30.990+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.990+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:30.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:22:30.991+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:22:30.992+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:22:30.993+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:22:31.002+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:22:30.998+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:30.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex ... 2023-10-21T04:22:31.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.000+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.002+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.004+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex stopped 2023-10-21T04:22:31.009+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:22:31.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex ... 2023-10-21T04:22:31.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.010+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:22:31.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 143 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:22:31.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:22:31.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex closed 2023-10-21T04:22:31.012+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.012+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.013+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.013+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:22:31.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:22:31.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 158 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:22:31.010+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.014+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.014+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.015+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.018+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:22:31.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex closed 2023-10-21T04:22:31.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex ... 2023-10-21T04:22:31.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex ... 2023-10-21T04:22:31.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.019+05:30 [Info] switched currmeta from 904 -> 905 force true 2023-10-21T04:22:31.020+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:22:31.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex ... 2023-10-21T04:22:31.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex ... 2023-10-21T04:22:31.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:22:31.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex closed 2023-10-21T04:22:31.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:22:31.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex closed 2023-10-21T04:22:31.023+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:22:31.023+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:22:31.023+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:22:31.024+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:22:31.024+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:22:31.024+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:22:31.024+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:22:31.024+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:22:31.024+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:22:31.025+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:22:31.025+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:22:31.025+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:22:31.026+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.026+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.026+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.026+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.026+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.026+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:22:31.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex stopped 2023-10-21T04:22:31.027+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.027+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.027+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.027+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.028+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.028+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.028+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.029+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] 2023-10-21T04:22:31.029+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:22:31.029+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:22:31.029+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:22:31.029+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:22:31.029+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:22:31.030+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:22:31.030+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:22:31.030+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:22:31.030+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:22:31.030+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:22:31.031+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:22:31.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:22:31.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 141 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:22:31.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:22:31.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 159 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:22:31.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex stopped 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex stopped 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.033+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_6.index destroyed 2023-10-21T04:22:31.033+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 6, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.033+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 6 Destroy Done 2023-10-21T04:22:31.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex stopped 2023-10-21T04:22:31.034+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.034+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.034+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.034+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex stopped 2023-10-21T04:22:31.035+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.035+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.035+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.035+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex stopped 2023-10-21T04:22:31.035+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.035+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.035+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.036+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex] 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex closed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index destroyed 2023-10-21T04:22:31.037+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 5, IndexDefnId 17656144409930763141. 2023-10-21T04:22:31.037+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 5 Destroy Done 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex ... 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex ... 2023-10-21T04:22:31.040+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.041+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:22:31.041+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:22:31.041+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:22:31.041+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:22:31.041+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:22:31.041+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] 2023-10-21T04:22:31.041+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] 2023-10-21T04:22:31.041+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:22:31.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:22:31.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex closed 2023-10-21T04:22:31.043+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.043+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex ... 2023-10-21T04:22:31.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex ... 2023-10-21T04:22:31.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex ... 2023-10-21T04:22:31.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex ... 2023-10-21T04:22:31.048+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.048+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:22:31.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 157 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:22:31.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex stopped 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex stopped 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.049+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex ... 2023-10-21T04:22:31.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex ... 2023-10-21T04:22:31.051+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_2.index destroyed 2023-10-21T04:22:31.056+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 2, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.057+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index destroyed 2023-10-21T04:22:31.057+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 4, IndexDefnId 17656144409930763141. 2023-10-21T04:22:31.057+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 4 Destroy Done 2023-10-21T04:22:31.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex ... 2023-10-21T04:22:31.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex ... 2023-10-21T04:22:31.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_7.index destroyed 2023-10-21T04:22:31.061+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 7, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.062+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 7 Destroy Done 2023-10-21T04:22:31.062+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 2 Destroy Done 2023-10-21T04:22:31.069+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex ... 2023-10-21T04:22:31.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex ... 2023-10-21T04:22:31.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.072+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 328.861659ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:31.073+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 15649171661394925140 instance id 7256866756526982695 real instance id 10477796393418268202 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex ... 2023-10-21T04:22:31.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex ... 2023-10-21T04:22:31.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:22:31.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 144 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:22:31.074+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex ... 2023-10-21T04:22:31.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex ... 2023-10-21T04:22:31.075+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 15649171661394925140 instance 10477796393418268202 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.075+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 15649171661394925140 instance id 10477796393418268202 2023-10-21T04:22:31.075+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 15649171661394925140 2023-10-21T04:22:31.083+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.088+05:30 [Info] switched currmeta from 905 -> 906 force true 2023-10-21T04:22:31.088+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_8.index destroyed 2023-10-21T04:22:31.089+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 8, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.089+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 8 Destroy Done 2023-10-21T04:22:31.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex ... 2023-10-21T04:22:31.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex ... 2023-10-21T04:22:31.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex ... 2023-10-21T04:22:31.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex ... 2023-10-21T04:22:31.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex ... 2023-10-21T04:22:31.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex ... 2023-10-21T04:22:31.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex ... 2023-10-21T04:22:31.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex ... 2023-10-21T04:22:31.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.107+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10477796393418268202 &{1} 2023-10-21T04:22:31.107+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10477796393418268202 2023-10-21T04:22:31.107+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 15649171661394925140, instId: 10477796393418268202 2023-10-21T04:22:31.108+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10477796393418268202] 2023-10-21T04:22:31.108+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10477796393418268202] 2023-10-21T04:22:31.108+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:31.109+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10477796393418268202] 2023-10-21T04:22:31.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:22:31.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 160 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:22:31.109+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:22:31.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex closed 2023-10-21T04:22:31.112+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.112+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex ... 2023-10-21T04:22:31.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex ... 2023-10-21T04:22:31.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_1.index destroyed 2023-10-21T04:22:31.120+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 1, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.121+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 1 Destroy Done 2023-10-21T04:22:31.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex ... 2023-10-21T04:22:31.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex ... 2023-10-21T04:22:31.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex closed 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex closed 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex closed 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex closed 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex closed 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex closed 2023-10-21T04:22:31.123+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.123+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex ... 2023-10-21T04:22:31.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex ... 2023-10-21T04:22:31.124+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.124+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.125+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.125+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.125+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.125+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex ... 2023-10-21T04:22:31.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index destroyed 2023-10-21T04:22:31.127+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.127+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 3, IndexDefnId 17656144409930763141. 2023-10-21T04:22:31.128+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 3 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_3.index destroyed 2023-10-21T04:22:31.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex ... 2023-10-21T04:22:31.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex ... 2023-10-21T04:22:31.128+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 3, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.128+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 3 Destroy Done 2023-10-21T04:22:31.128+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.128+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index in /opt/build/ns_server/data/n_3/data/@2i ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index destroyed 2023-10-21T04:22:31.129+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.129+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 2 Destroy Done 2023-10-21T04:22:31.128+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_5.index destroyed 2023-10-21T04:22:31.132+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 5, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.132+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 5 Destroy Done 2023-10-21T04:22:31.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex ... 2023-10-21T04:22:31.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex ... 2023-10-21T04:22:31.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex ... 2023-10-21T04:22:31.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex ... 2023-10-21T04:22:31.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_14579235200590321485_4.index destroyed 2023-10-21T04:22:31.135+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14579235200590321485, PartitionId 4, IndexDefnId 15991242037348127852. 2023-10-21T04:22:31.135+05:30 [Info] Indexer::cleanupIndexData IndexInst 14579235200590321485 Partition 4 Destroy Done 2023-10-21T04:22:31.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index destroyed 2023-10-21T04:22:31.139+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 6, IndexDefnId 17656144409930763141. 2023-10-21T04:22:31.139+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 6 Destroy Done 2023-10-21T04:22:31.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex ... 2023-10-21T04:22:31.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex ... 2023-10-21T04:22:31.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex ... 2023-10-21T04:22:31.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex ... 2023-10-21T04:22:31.149+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.152+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index destroyed 2023-10-21T04:22:31.158+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.158+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 3 Destroy Done 2023-10-21T04:22:31.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex ... 2023-10-21T04:22:31.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex ... 2023-10-21T04:22:31.170+05:30 [Info] switched currmeta from 906 -> 908 force true 2023-10-21T04:22:31.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex ... 2023-10-21T04:22:31.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex ... 2023-10-21T04:22:31.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex ... 2023-10-21T04:22:31.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index destroyed 2023-10-21T04:22:31.185+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.185+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 6 Destroy Done 2023-10-21T04:22:31.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex ... 2023-10-21T04:22:31.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex ... 2023-10-21T04:22:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index destroyed 2023-10-21T04:22:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex ... 2023-10-21T04:22:31.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex ... 2023-10-21T04:22:31.197+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.197+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 1 Destroy Done 2023-10-21T04:22:31.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index destroyed 2023-10-21T04:22:31.212+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.212+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 7 Destroy Done 2023-10-21T04:22:31.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex ... 2023-10-21T04:22:31.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex ... 2023-10-21T04:22:31.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex ... 2023-10-21T04:22:31.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index destroyed 2023-10-21T04:22:31.219+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.219+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 4 Destroy Done 2023-10-21T04:22:31.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index destroyed 2023-10-21T04:22:31.221+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.221+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 8 Destroy Done 2023-10-21T04:22:31.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex ... 2023-10-21T04:22:31.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex ... 2023-10-21T04:22:31.223+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.223+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index destroyed 2023-10-21T04:22:31.224+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5, IndexDefnId 573521267164877540. 2023-10-21T04:22:31.224+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 5 Destroy Done 2023-10-21T04:22:31.224+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 1, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:22:31.225+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:22:31.225+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:22:31.225+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:22:31.225+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex 2023-10-21T04:22:31.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex stopped 2023-10-21T04:22:31.225+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.225+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.225+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.225+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.226+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 2, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:22:31.226+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:22:31.226+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex 2023-10-21T04:22:31.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex stopped 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.227+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 3, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex 2023-10-21T04:22:31.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex stopped 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.227+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 4, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex 2023-10-21T04:22:31.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex stopped 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.227+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 5, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:22:31.227+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:22:31.228+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex 2023-10-21T04:22:31.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex stopped 2023-10-21T04:22:31.228+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.228+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.228+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.228+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.228+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 10477796393418268202 Defn: DefnId: 15649171661394925140 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:22:31.228+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 120.951924ms 2023-10-21T04:22:31.229+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 6, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:22:31.229+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:22:31.229+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex 2023-10-21T04:22:31.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex stopped 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.229+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 7, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:22:31.229+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:22:31.231+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:22:31.231+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:22:31.231+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex 2023-10-21T04:22:31.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex stopped 2023-10-21T04:22:31.231+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.238+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.232+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10477796393418268202, PartitionId 8, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.269+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.269+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.269+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:22:31.269+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:22:31.269+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:22:31.269+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:22:31.270+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex 2023-10-21T04:22:31.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex stopped 2023-10-21T04:22:31.270+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.270+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.270+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.274+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.238+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.275+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.232+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10477796393418268202 2023-10-21T04:22:31.232+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 124.599863ms 2023-10-21T04:22:31.231+05:30 [Info] switched currmeta from 908 -> 908 force true 2023-10-21T04:22:31.341+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.341+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.352+05:30 [Info] switched currmeta from 908 -> 908 force true 2023-10-21T04:22:31.358+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex] 2023-10-21T04:22:31.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex closed 2023-10-21T04:22:31.363+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 289.85554ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:31.363+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 14026476026078401783 instance id 15512663964869546873 real instance id 5744899830895973442 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.363+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 14026476026078401783 instance 5744899830895973442 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.363+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 14026476026078401783 instance id 5744899830895973442 2023-10-21T04:22:31.363+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 14026476026078401783 2023-10-21T04:22:31.364+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:22:31.386+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:22:31.386+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex 2023-10-21T04:22:31.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex stopped 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.386+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.399+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex] 2023-10-21T04:22:31.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex closed 2023-10-21T04:22:31.400+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.401+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.401+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.401+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:22:31.402+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:22:31.402+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:22:31.402+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:22:31.402+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex 2023-10-21T04:22:31.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex stopped 2023-10-21T04:22:31.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex] 2023-10-21T04:22:31.414+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex] 2023-10-21T04:22:31.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex closed 2023-10-21T04:22:31.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.448+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex ... 2023-10-21T04:22:31.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 161 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex ... 2023-10-21T04:22:31.447+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex closed 2023-10-21T04:22:31.461+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.463+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.464+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.465+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:22:31.453+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.477+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.478+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.479+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 5744899830895973442 &{1} 2023-10-21T04:22:31.481+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.482+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.489+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex] 2023-10-21T04:22:31.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex closed 2023-10-21T04:22:31.491+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.491+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.492+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.508+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:22:31.492+05:30 [Info] Indexer::handleDropIndex - IndexInstId 5744899830895973442 2023-10-21T04:22:31.512+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 14026476026078401783, instId: 5744899830895973442 2023-10-21T04:22:31.496+05:30 [Info] switched currmeta from 908 -> 910 force true 2023-10-21T04:22:31.496+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex] 2023-10-21T04:22:31.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex closed 2023-10-21T04:22:31.514+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.514+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.514+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.514+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:31.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:22:31.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex] 2023-10-21T04:22:31.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex closed 2023-10-21T04:22:31.515+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:31.515+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5744899830895973442 Name: idx_partitioned Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:31.516+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [5744899830895973442] 2023-10-21T04:22:31.516+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [5744899830895973442] 2023-10-21T04:22:31.517+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:31.517+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [5744899830895973442] 2023-10-21T04:22:31.515+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.518+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.514+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.515+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:22:31.519+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:22:31.519+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:22:31.519+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.519+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.519+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.519+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex] 2023-10-21T04:22:31.519+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex 2023-10-21T04:22:31.520+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex] 2023-10-21T04:22:31.520+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex] 2023-10-21T04:22:31.520+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex 2023-10-21T04:22:31.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex stopped 2023-10-21T04:22:31.520+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.520+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.520+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.520+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.519+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex 2023-10-21T04:22:31.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex stopped 2023-10-21T04:22:31.520+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.521+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.521+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.521+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.521+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.520+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex 2023-10-21T04:22:31.522+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex 2023-10-21T04:22:31.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex stopped 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.521+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex 2023-10-21T04:22:31.522+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex 2023-10-21T04:22:31.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex stopped 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.519+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex 2023-10-21T04:22:31.522+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex] 2023-10-21T04:22:31.522+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex 2023-10-21T04:22:31.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex stopped 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.522+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.520+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:31.523+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 5744899830895973442 Defn: DefnId: 14026476026078401783 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:31.523+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 30.770968ms 2023-10-21T04:22:31.519+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.523+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.523+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.523+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex] 2023-10-21T04:22:31.523+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex 2023-10-21T04:22:31.523+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex] 2023-10-21T04:22:31.523+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex] 2023-10-21T04:22:31.523+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex 2023-10-21T04:22:31.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex stopped 2023-10-21T04:22:31.523+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.523+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.523+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.523+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.523+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 5744899830895973442 2023-10-21T04:22:31.520+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.524+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.524+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.524+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex] 2023-10-21T04:22:31.525+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex 2023-10-21T04:22:31.525+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex] 2023-10-21T04:22:31.525+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex] 2023-10-21T04:22:31.525+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex 2023-10-21T04:22:31.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex stopped 2023-10-21T04:22:31.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.526+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.527+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [5744899830895973442] 2023-10-21T04:22:31.521+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.532+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.533+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.533+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.521+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex] 2023-10-21T04:22:31.521+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:22:31.534+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:22:31.534+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:22:31.534+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex 2023-10-21T04:22:31.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex stopped 2023-10-21T04:22:31.534+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.535+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.535+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.535+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex] 2023-10-21T04:22:31.536+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex 2023-10-21T04:22:31.536+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex] 2023-10-21T04:22:31.536+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex] 2023-10-21T04:22:31.536+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex 2023-10-21T04:22:31.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex stopped 2023-10-21T04:22:31.537+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.537+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.537+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.537+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.539+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.539+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.539+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex] 2023-10-21T04:22:31.540+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex 2023-10-21T04:22:31.540+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex] 2023-10-21T04:22:31.540+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex] 2023-10-21T04:22:31.541+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex 2023-10-21T04:22:31.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex stopped 2023-10-21T04:22:31.541+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.541+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.542+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.542+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex closed 2023-10-21T04:22:31.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex] 2023-10-21T04:22:31.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex closed 2023-10-21T04:22:31.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex] 2023-10-21T04:22:31.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex closed 2023-10-21T04:22:31.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex ... 2023-10-21T04:22:31.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 179 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex ... 2023-10-21T04:22:31.548+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:22:31.558+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:22:31.558+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:22:31.558+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex 2023-10-21T04:22:31.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex stopped 2023-10-21T04:22:31.558+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.558+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.558+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.558+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.552+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex] 2023-10-21T04:22:31.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex closed 2023-10-21T04:22:31.540+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:31.559+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:22:31.559+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 36.362094ms 2023-10-21T04:22:31.559+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 67.38151ms 2023-10-21T04:22:31.560+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.560+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.561+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.564+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex ... 2023-10-21T04:22:31.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 162 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex ... 2023-10-21T04:22:31.562+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.565+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.565+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.565+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:22:31.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:22:31.562+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.566+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.562+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#10477796393418268202:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.566+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.566+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.566+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:22:31.563+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex] 2023-10-21T04:22:31.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex closed 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex] 2023-10-21T04:22:31.566+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex 2023-10-21T04:22:31.566+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex] 2023-10-21T04:22:31.566+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex] 2023-10-21T04:22:31.566+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex 2023-10-21T04:22:31.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex stopped 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.566+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.569+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 206.614051ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:31.570+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16162012140118277065 instance id 5701304422421248176 real instance id 11460826864060385936 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.570+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16162012140118277065 instance 11460826864060385936 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.570+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16162012140118277065 instance id 11460826864060385936 2023-10-21T04:22:31.570+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16162012140118277065 2023-10-21T04:22:31.613+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.614+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:22:31.614+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:22:31.615+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:22:31.615+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex 2023-10-21T04:22:31.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex stopped 2023-10-21T04:22:31.615+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.615+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.615+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.616+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.572+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex] 2023-10-21T04:22:31.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex closed 2023-10-21T04:22:31.617+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.617+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.617+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.617+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex] 2023-10-21T04:22:31.618+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex 2023-10-21T04:22:31.618+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex] 2023-10-21T04:22:31.618+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex] 2023-10-21T04:22:31.618+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex 2023-10-21T04:22:31.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex stopped 2023-10-21T04:22:31.620+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.620+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.620+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.620+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.621+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:22:31.621+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:22:31.621+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:22:31.622+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex 2023-10-21T04:22:31.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex stopped 2023-10-21T04:22:31.622+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.622+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.622+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.622+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex ... 2023-10-21T04:22:31.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 158 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex ... 2023-10-21T04:22:31.580+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.580+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.604+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.631+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex] 2023-10-21T04:22:31.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex closed 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex] 2023-10-21T04:22:31.635+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex 2023-10-21T04:22:31.635+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex] 2023-10-21T04:22:31.635+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex] 2023-10-21T04:22:31.635+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex 2023-10-21T04:22:31.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex stopped 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.635+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.635+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 11460826864060385936 &{1} 2023-10-21T04:22:31.642+05:30 [Info] Indexer::handleDropIndex - IndexInstId 11460826864060385936 2023-10-21T04:22:31.642+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16162012140118277065, instId: 11460826864060385936 2023-10-21T04:22:31.642+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:31.642+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:31.642+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:31.643+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [11460826864060385936] 2023-10-21T04:22:31.643+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [11460826864060385936] 2023-10-21T04:22:31.643+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:31.643+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [11460826864060385936] 2023-10-21T04:22:31.642+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex] 2023-10-21T04:22:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex closed 2023-10-21T04:22:31.644+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.644+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex ... 2023-10-21T04:22:31.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex ... 2023-10-21T04:22:31.642+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex] 2023-10-21T04:22:31.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex closed 2023-10-21T04:22:31.646+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.646+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.646+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.646+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex] 2023-10-21T04:22:31.647+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.647+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.647+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.649+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:22:31.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex ... 2023-10-21T04:22:31.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex ... 2023-10-21T04:22:31.652+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.652+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex 2023-10-21T04:22:31.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex] 2023-10-21T04:22:31.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex] 2023-10-21T04:22:31.653+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex 2023-10-21T04:22:31.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex stopped 2023-10-21T04:22:31.653+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.653+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.653+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.653+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.647+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:22:31.647+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:22:31.647+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:22:31.647+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:22:31.647+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.653+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:22:31.647+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:31.653+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:31.653+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 11.709825ms 2023-10-21T04:22:31.648+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.654+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:22:31.654+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 11460826864060385936 2023-10-21T04:22:31.648+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1, IndexDefnId 16162012140118277065. 2023-10-21T04:22:31.657+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.657+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.657+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:22:31.657+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [11460826864060385936] 2023-10-21T04:22:31.649+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex] 2023-10-21T04:22:31.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex closed 2023-10-21T04:22:31.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.665+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex ... 2023-10-21T04:22:31.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex ... 2023-10-21T04:22:31.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.667+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:22:31.667+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:22:31.667+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:22:31.667+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex 2023-10-21T04:22:31.668+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex] 2023-10-21T04:22:31.668+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex closed 2023-10-21T04:22:31.668+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex] 2023-10-21T04:22:31.668+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex closed 2023-10-21T04:22:31.669+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.670+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex] 2023-10-21T04:22:31.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex closed 2023-10-21T04:22:31.673+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.674+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.674+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.674+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex] 2023-10-21T04:22:31.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex stopped 2023-10-21T04:22:31.674+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.674+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.675+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.675+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.675+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.675+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_1.index destroyed 2023-10-21T04:22:31.677+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex] 2023-10-21T04:22:31.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex closed 2023-10-21T04:22:31.681+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.681+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.678+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex] 2023-10-21T04:22:31.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex closed 2023-10-21T04:22:31.681+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.681+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.681+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.681+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex] 2023-10-21T04:22:31.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex ... 2023-10-21T04:22:31.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 180 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex ... 2023-10-21T04:22:31.683+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:31.684+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:22:31.684+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 30.648098ms 2023-10-21T04:22:31.684+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 42.554083ms 2023-10-21T04:22:31.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex] 2023-10-21T04:22:31.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex closed 2023-10-21T04:22:31.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.698+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:22:31.698+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:22:31.698+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:22:31.698+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_2.index destroyed 2023-10-21T04:22:31.699+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 2, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.699+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 2 Destroy Done 2023-10-21T04:22:31.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex stopped 2023-10-21T04:22:31.699+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.699+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.699+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.699+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.700+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:22:31.700+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:22:31.705+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:22:31.705+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:22:31.705+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:22:31.702+05:30 [Info] switched currmeta from 910 -> 914 force true 2023-10-21T04:22:31.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.681+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.681+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 1, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.676+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.703+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex stopped 2023-10-21T04:22:31.713+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:22:31.714+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:22:31.714+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:22:31.714+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:22:31.714+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:22:31.714+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:22:31.714+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:22:31.715+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:22:31.715+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:22:31.715+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:22:31.715+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:22:31.715+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:22:31.715+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:22:31.715+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:22:31.716+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:22:31.716+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:22:31.716+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:22:31.716+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:22:31.716+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:22:31.716+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:22:31.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.716+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex 2023-10-21T04:22:31.716+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:22:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex stopped 2023-10-21T04:22:31.717+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.717+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.717+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.717+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.717+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:22:31.717+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:22:31.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex ... 2023-10-21T04:22:31.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 176 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex ... 2023-10-21T04:22:31.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex] 2023-10-21T04:22:31.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex closed 2023-10-21T04:22:31.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.727+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex stopped 2023-10-21T04:22:31.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.727+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex stopped 2023-10-21T04:22:31.728+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.728+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.728+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.728+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.717+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 1 Destroy Done 2023-10-21T04:22:31.717+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.728+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.728+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex] 2023-10-21T04:22:31.717+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.728+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex] 2023-10-21T04:22:31.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex closed 2023-10-21T04:22:31.728+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.728+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.729+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.729+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex stopped 2023-10-21T04:22:31.729+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.729+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.729+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.729+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex ... 2023-10-21T04:22:31.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_6.index destroyed 2023-10-21T04:22:31.729+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 6, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.729+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 6 Destroy Done 2023-10-21T04:22:31.729+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.729+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex] 2023-10-21T04:22:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex stopped 2023-10-21T04:22:31.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex stopped 2023-10-21T04:22:31.716+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex] 2023-10-21T04:22:31.724+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.730+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 160.203821ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:31.735+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.736+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.737+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.737+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.737+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex] 2023-10-21T04:22:31.737+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex 2023-10-21T04:22:31.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex stopped 2023-10-21T04:22:31.737+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.737+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.738+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.738+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex] 2023-10-21T04:22:31.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex closed 2023-10-21T04:22:31.739+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#10477796393418268202:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.739+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.739+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 2872726210871813325 instance id 1882546579949915688 real instance id 11375521620401506748 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.739+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 2872726210871813325 instance 11375521620401506748 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.739+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 2872726210871813325 instance id 11375521620401506748 2023-10-21T04:22:31.739+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 2872726210871813325 2023-10-21T04:22:31.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex ... 2023-10-21T04:22:31.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 163 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex ... 2023-10-21T04:22:31.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex] 2023-10-21T04:22:31.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex closed 2023-10-21T04:22:31.754+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.785+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.755+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 11375521620401506748 &{1} 2023-10-21T04:22:31.788+05:30 [Info] Indexer::handleDropIndex - IndexInstId 11375521620401506748 2023-10-21T04:22:31.788+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 2872726210871813325, instId: 11375521620401506748 2023-10-21T04:22:31.788+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [11375521620401506748] 2023-10-21T04:22:31.788+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [11375521620401506748] 2023-10-21T04:22:31.789+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:31.789+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [11375521620401506748] 2023-10-21T04:22:31.754+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.756+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex 2023-10-21T04:22:31.789+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex] 2023-10-21T04:22:31.789+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex] 2023-10-21T04:22:31.789+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex 2023-10-21T04:22:31.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex stopped 2023-10-21T04:22:31.789+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.789+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.789+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.789+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.795+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.796+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 3, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.796+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.796+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.796+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:22:31.796+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:22:31.797+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:22:31.797+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:22:31.797+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex 2023-10-21T04:22:31.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex stopped 2023-10-21T04:22:31.797+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.797+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.797+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.797+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.797+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.798+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 4, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.798+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.798+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.798+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:22:31.757+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex] 2023-10-21T04:22:31.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex closed 2023-10-21T04:22:31.757+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.798+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.798+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 5, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.770+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex] 2023-10-21T04:22:31.799+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 2, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.799+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.799+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.799+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:22:31.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex ... 2023-10-21T04:22:31.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 164 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex ... 2023-10-21T04:22:31.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex closed 2023-10-21T04:22:31.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_6.index destroyed 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:22:31.798+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 7, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:22:31.798+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 6, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:22:31.799+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 8, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:22:31.799+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 11375521620401506748 Defn: DefnId: 2872726210871813325 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:22:31.800+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 12.352499ms 2023-10-21T04:22:31.799+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11375521620401506748, PartitionId 1, IndexDefnId 2872726210871813325. 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:22:31.800+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 11375521620401506748 2023-10-21T04:22:31.800+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 6, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.801+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 6 Destroy Done 2023-10-21T04:22:31.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:22:31.802+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.803+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 14.806214ms 2023-10-21T04:22:31.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex closed 2023-10-21T04:22:31.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex ... 2023-10-21T04:22:31.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 160 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex ... 2023-10-21T04:22:31.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:22:31.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex closed 2023-10-21T04:22:31.805+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.798+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:22:31.806+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:22:31.807+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:22:31.807+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex 2023-10-21T04:22:31.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:22:31.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex closed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_5.index destroyed 2023-10-21T04:22:31.808+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 5, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.808+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 5 Destroy Done 2023-10-21T04:22:31.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:22:31.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex closed 2023-10-21T04:22:31.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:22:31.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex closed 2023-10-21T04:22:31.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:22:31.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex closed 2023-10-21T04:22:31.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex ... 2023-10-21T04:22:31.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 181 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex ... 2023-10-21T04:22:31.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex stopped 2023-10-21T04:22:31.811+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.817+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.813+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.813+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.814+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.814+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.814+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.814+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.814+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.814+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:22:31.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.817+05:30 [Info] switched currmeta from 914 -> 916 force true 2023-10-21T04:22:31.817+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.819+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.820+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.820+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:22:31.821+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.822+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.822+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.822+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:22:31.822+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.823+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.823+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.823+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.823+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.824+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.824+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 85.168299ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:31.824+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 2374548411389959576 instance id 18286344351415703073 real instance id 12762948908485355776 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.824+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 2374548411389959576 instance 12762948908485355776 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:31.824+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 2374548411389959576 instance id 12762948908485355776 2023-10-21T04:22:31.824+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 2374548411389959576 2023-10-21T04:22:31.824+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:22:31.825+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.826+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.826+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.826+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex 2023-10-21T04:22:31.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:22:31.826+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:22:31.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.834+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:22:31.837+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.837+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:22:31.837+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.837+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex ... 2023-10-21T04:22:31.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 159 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex ... 2023-10-21T04:22:31.837+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex] 2023-10-21T04:22:31.840+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex] 2023-10-21T04:22:31.841+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex 2023-10-21T04:22:31.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex stopped 2023-10-21T04:22:31.841+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.841+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.841+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.841+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:22:31.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex closed 2023-10-21T04:22:31.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:22:31.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex closed 2023-10-21T04:22:31.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex ... 2023-10-21T04:22:31.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 178 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex ... 2023-10-21T04:22:31.837+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:22:31.842+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex 2023-10-21T04:22:31.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex stopped 2023-10-21T04:22:31.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_3.index destroyed 2023-10-21T04:22:31.843+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 3, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.843+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 3 Destroy Done 2023-10-21T04:22:31.843+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.843+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.843+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.843+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:22:31.848+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.849+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.851+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:22:31.851+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:22:31.851+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:22:31.852+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex 2023-10-21T04:22:31.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex stopped 2023-10-21T04:22:31.852+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.852+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.852+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.852+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.852+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex 2023-10-21T04:22:31.852+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex] 2023-10-21T04:22:31.852+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex] 2023-10-21T04:22:31.852+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex 2023-10-21T04:22:31.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex stopped 2023-10-21T04:22:31.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex ... 2023-10-21T04:22:31.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex ... 2023-10-21T04:22:31.853+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.874+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:22:31.854+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12762948908485355776 &{1} 2023-10-21T04:22:31.875+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12762948908485355776 2023-10-21T04:22:31.875+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 2374548411389959576, instId: 12762948908485355776 2023-10-21T04:22:31.875+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12762948908485355776] 2023-10-21T04:22:31.875+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12762948908485355776] 2023-10-21T04:22:31.875+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:31.875+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12762948908485355776] 2023-10-21T04:22:31.854+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:22:31.876+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:22:31.876+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:22:31.876+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex 2023-10-21T04:22:31.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex stopped 2023-10-21T04:22:31.876+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.876+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.876+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.876+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.876+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 3, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.876+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.876+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.876+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:22:31.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_8.index destroyed 2023-10-21T04:22:31.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex ... 2023-10-21T04:22:31.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 182 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex ... 2023-10-21T04:22:31.881+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:22:31.881+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:22:31.881+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:22:31.881+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex 2023-10-21T04:22:31.881+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex stopped 2023-10-21T04:22:31.881+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.881+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.881+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.881+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.881+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 4, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.881+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:22:31.882+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 5, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:22:31.862+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.871+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:31.882+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:31.882+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:22:31.882+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:22:31.882+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:22:31.882+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex 2023-10-21T04:22:31.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex stopped 2023-10-21T04:22:31.882+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.882+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.882+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.882+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.882+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 6, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:22:31.882+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 7, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.882+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:22:31.872+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex] 2023-10-21T04:22:31.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex closed 2023-10-21T04:22:31.883+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.883+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.883+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.883+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:22:31.883+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:22:31.883+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:22:31.883+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:22:31.883+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex 2023-10-21T04:22:31.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex stopped 2023-10-21T04:22:31.883+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.883+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.884+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.884+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.884+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 8, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.884+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.884+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.885+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:22:31.885+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 12762948908485355776 Defn: DefnId: 2374548411389959576 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:22:31.912+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 37.802383ms 2023-10-21T04:22:31.913+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 38.012836ms 2023-10-21T04:22:31.886+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 1, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.913+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.913+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.913+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:22:31.913+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12762948908485355776 2023-10-21T04:22:31.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex] 2023-10-21T04:22:31.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex closed 2023-10-21T04:22:31.916+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.916+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.916+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.916+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:22:31.886+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12762948908485355776, PartitionId 2, IndexDefnId 2374548411389959576. 2023-10-21T04:22:31.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:22:31.888+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.898+05:30 [Info] switched currmeta from 916 -> 918 force true 2023-10-21T04:22:31.898+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 8, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.917+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 8 Destroy Done 2023-10-21T04:22:31.899+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex] 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex closed 2023-10-21T04:22:31.917+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.917+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:31.917+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:31.917+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:22:31.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex closed 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex closed 2023-10-21T04:22:31.917+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:22:31.917+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex 2023-10-21T04:22:31.917+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:22:31.917+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex closed 2023-10-21T04:22:31.917+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:22:31.917+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:22:31.917+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex 2023-10-21T04:22:31.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex stopped 2023-10-21T04:22:31.918+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.918+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.918+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.918+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.918+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:22:31.918+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:22:31.918+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:22:31.918+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:22:31.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex stopped 2023-10-21T04:22:31.918+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.918+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.918+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.918+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex ... 2023-10-21T04:22:31.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex ... 2023-10-21T04:22:31.920+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.920+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex ... 2023-10-21T04:22:31.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex ... 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.921+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex stopped 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex stopped 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.921+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.921+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:31.921+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:22:31.921+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:22:31.921+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:22:31.921+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:22:31.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex stopped 2023-10-21T04:22:31.921+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.921+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.921+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.921+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex stopped 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex stopped 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex stopped 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex stopped 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.922+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:22:31.922+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:22:31.923+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:22:31.923+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:22:31.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex stopped 2023-10-21T04:22:31.923+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.923+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.923+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.923+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex ... 2023-10-21T04:22:31.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 157 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex ... 2023-10-21T04:22:31.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.963+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_4.index destroyed 2023-10-21T04:22:31.964+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 4, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.965+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 4 Destroy Done 2023-10-21T04:22:31.965+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:22:31.967+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:22:31.968+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:22:31.968+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex 2023-10-21T04:22:31.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex stopped 2023-10-21T04:22:31.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.972+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.973+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:22:31.973+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:22:31.973+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:22:31.973+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:22:31.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:22:31.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex closed 2023-10-21T04:22:31.974+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.974+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 4 Close Done 2023-10-21T04:22:31.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex ... 2023-10-21T04:22:31.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 175 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex ... 2023-10-21T04:22:31.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:31.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex ... 2023-10-21T04:22:31.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex ... 2023-10-21T04:22:31.977+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:22:31.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex closed 2023-10-21T04:22:31.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.978+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 1 Close Done 2023-10-21T04:22:31.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex stopped 2023-10-21T04:22:31.978+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.979+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.979+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.979+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex ... 2023-10-21T04:22:31.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex ... 2023-10-21T04:22:31.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_5.index destroyed 2023-10-21T04:22:31.980+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 5, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.980+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 5 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_1.index destroyed 2023-10-21T04:22:31.980+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:22:31.988+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:22:31.989+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex 2023-10-21T04:22:31.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex stopped 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.989+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:22:31.989+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex 2023-10-21T04:22:31.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex stopped 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.989+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.990+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:22:31.990+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex 2023-10-21T04:22:31.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex stopped 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.990+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 1, IndexDefnId 14026476026078401783. 2023-10-21T04:22:31.990+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 1 Destroy Done 2023-10-21T04:22:31.990+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:22:31.990+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:22:31.991+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex 2023-10-21T04:22:31.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex stopped 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.991+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:22:31.991+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex 2023-10-21T04:22:31.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex stopped 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.991+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:22:31.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex closed 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.992+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 3 Close Done 2023-10-21T04:22:31.992+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:22:31.992+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex 2023-10-21T04:22:31.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex stopped 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:31.992+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:31.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex ... 2023-10-21T04:22:31.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 177 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex ... 2023-10-21T04:22:31.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index/mainIndex sucessfully destroyed 2023-10-21T04:22:31.993+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:31.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_10477796393418268202_7.index destroyed 2023-10-21T04:22:31.997+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10477796393418268202, PartitionId 7, IndexDefnId 15649171661394925140. 2023-10-21T04:22:31.997+05:30 [Info] Indexer::cleanupIndexData IndexInst 10477796393418268202 Partition 7 Destroy Done 2023-10-21T04:22:31.997+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:31.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:22:31.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex closed 2023-10-21T04:22:31.998+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:31.998+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 8 Close Done 2023-10-21T04:22:32.004+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:22:32.023+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:22:32.023+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:22:32.023+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex 2023-10-21T04:22:32.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex stopped 2023-10-21T04:22:32.023+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.023+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.024+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.024+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.029+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 204.725763ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:32.029+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 10585832761158485697 instance id 15074233131683912490 real instance id 759049595436785640 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:32.031+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 10585832761158485697 instance 759049595436785640 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:32.032+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 10585832761158485697 instance id 759049595436785640 2023-10-21T04:22:32.032+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 10585832761158485697 2023-10-21T04:22:32.042+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:32.042+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:32.048+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex] 2023-10-21T04:22:32.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex closed 2023-10-21T04:22:32.048+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.049+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.049+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.049+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:22:32.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex ... 2023-10-21T04:22:32.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex ... 2023-10-21T04:22:32.054+05:30 [Info] switched currmeta from 918 -> 920 force true 2023-10-21T04:22:32.054+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_3.index destroyed 2023-10-21T04:22:32.055+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex] 2023-10-21T04:22:32.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex closed 2023-10-21T04:22:32.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex ... 2023-10-21T04:22:32.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex ... 2023-10-21T04:22:32.059+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:22:32.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex closed 2023-10-21T04:22:32.059+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.056+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.056+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 3, IndexDefnId 14026476026078401783. 2023-10-21T04:22:32.059+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 3 Destroy Done 2023-10-21T04:22:32.059+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.059+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 3 Close Done 2023-10-21T04:22:32.059+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 5 Close Done 2023-10-21T04:22:32.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_8.index destroyed 2023-10-21T04:22:32.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex] 2023-10-21T04:22:32.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex closed 2023-10-21T04:22:32.069+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:22:32.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex] 2023-10-21T04:22:32.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex closed 2023-10-21T04:22:32.071+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:22:32.070+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 8, IndexDefnId 14026476026078401783. 2023-10-21T04:22:32.071+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 8 Destroy Done 2023-10-21T04:22:32.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex ... 2023-10-21T04:22:32.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex ... 2023-10-21T04:22:32.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex ... 2023-10-21T04:22:32.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex ... 2023-10-21T04:22:32.075+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:22:32.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex closed 2023-10-21T04:22:32.075+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.075+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 7 Close Done 2023-10-21T04:22:32.087+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex] 2023-10-21T04:22:32.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex closed 2023-10-21T04:22:32.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex] 2023-10-21T04:22:32.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex closed 2023-10-21T04:22:32.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:22:32.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:22:32.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:22:32.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex 2023-10-21T04:22:32.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:22:32.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:22:32.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:22:32.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex 2023-10-21T04:22:32.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:22:32.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:22:32.102+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:22:32.102+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex 2023-10-21T04:22:32.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex ... 2023-10-21T04:22:32.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex ... 2023-10-21T04:22:32.104+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.099+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 759049595436785640 &{1} 2023-10-21T04:22:32.104+05:30 [Info] Indexer::handleDropIndex - IndexInstId 759049595436785640 2023-10-21T04:22:32.105+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 10585832761158485697, instId: 759049595436785640 2023-10-21T04:22:32.105+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [759049595436785640] 2023-10-21T04:22:32.106+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [759049595436785640] 2023-10-21T04:22:32.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_7.index destroyed 2023-10-21T04:22:32.107+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 7, IndexDefnId 14026476026078401783. 2023-10-21T04:22:32.107+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 7 Destroy Done 2023-10-21T04:22:32.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex stopped 2023-10-21T04:22:32.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.107+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 2 Close Done 2023-10-21T04:22:32.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.108+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 4 Close Done 2023-10-21T04:22:32.106+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:32.108+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [759049595436785640] 2023-10-21T04:22:32.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex] 2023-10-21T04:22:32.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex closed 2023-10-21T04:22:32.110+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.110+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.110+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.110+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:22:32.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex ... 2023-10-21T04:22:32.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex ... 2023-10-21T04:22:32.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex stopped 2023-10-21T04:22:32.111+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.111+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex stopped 2023-10-21T04:22:32.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.127+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:22:32.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex closed 2023-10-21T04:22:32.128+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.128+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 6 Close Done 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex] 2023-10-21T04:22:32.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex closed 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:22:32.129+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:22:32.129+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex 2023-10-21T04:22:32.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex stopped 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.130+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:22:32.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex closed 2023-10-21T04:22:32.131+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.131+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 2 Close Done 2023-10-21T04:22:32.138+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.165+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:22:32.165+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:22:32.165+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:22:32.165+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex 2023-10-21T04:22:32.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex stopped 2023-10-21T04:22:32.166+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.166+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.166+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.166+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.167+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex] 2023-10-21T04:22:32.167+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex closed 2023-10-21T04:22:32.167+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.167+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 7 Close Done 2023-10-21T04:22:32.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex] 2023-10-21T04:22:32.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex closed 2023-10-21T04:22:32.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:22:32.173+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:22:32.176+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:22:32.177+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:22:32.177+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex 2023-10-21T04:22:32.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex stopped 2023-10-21T04:22:32.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_4.index destroyed 2023-10-21T04:22:32.178+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 4, IndexDefnId 14026476026078401783. 2023-10-21T04:22:32.181+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 4 Destroy Done 2023-10-21T04:22:32.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex] 2023-10-21T04:22:32.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex closed 2023-10-21T04:22:32.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex ... 2023-10-21T04:22:32.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex ... 2023-10-21T04:22:32.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_5744899830895973442_2.index destroyed 2023-10-21T04:22:32.190+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5744899830895973442, PartitionId 2, IndexDefnId 14026476026078401783. 2023-10-21T04:22:32.191+05:30 [Info] Indexer::cleanupIndexData IndexInst 5744899830895973442 Partition 2 Destroy Done 2023-10-21T04:22:32.191+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#11375521620401506748:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.193+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.195+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.195+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:22:32.196+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:22:32.196+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:22:32.196+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:22:32.197+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex 2023-10-21T04:22:32.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex stopped 2023-10-21T04:22:32.197+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.197+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex ... 2023-10-21T04:22:32.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 155 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex ... 2023-10-21T04:22:32.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex ... 2023-10-21T04:22:32.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 172 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex ... 2023-10-21T04:22:32.195+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex] 2023-10-21T04:22:32.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex closed 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:22:32.214+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:22:32.214+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex 2023-10-21T04:22:32.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex stopped 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.215+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.215+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.199+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex] 2023-10-21T04:22:32.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex closed 2023-10-21T04:22:32.215+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.215+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex ... 2023-10-21T04:22:32.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 151 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.216+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:32.217+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:32.217+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex] 2023-10-21T04:22:32.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex closed Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.217+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:22:32.218+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:22:32.218+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex 2023-10-21T04:22:32.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex stopped 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.212+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex] 2023-10-21T04:22:32.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex closed 2023-10-21T04:22:32.218+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.218+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 4 Close Done 2023-10-21T04:22:32.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_7.index destroyed 2023-10-21T04:22:32.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex ... 2023-10-21T04:22:32.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 147 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex ... 2023-10-21T04:22:32.239+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 7, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.242+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 7 Destroy Done 2023-10-21T04:22:32.239+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.240+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex] 2023-10-21T04:22:32.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex closed 2023-10-21T04:22:32.242+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.242+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.242+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.242+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:22:32.250+05:30 [Info] switched currmeta from 920 -> 922 force true 2023-10-21T04:22:32.253+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex] 2023-10-21T04:22:32.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex closed 2023-10-21T04:22:32.254+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.254+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.254+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.254+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:22:32.255+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.261+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex ... 2023-10-21T04:22:32.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 168 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex ... 2023-10-21T04:22:32.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex ... 2023-10-21T04:22:32.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 150 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex ... 2023-10-21T04:22:32.263+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex] 2023-10-21T04:22:32.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex closed 2023-10-21T04:22:32.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:22:32.267+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.268+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex] 2023-10-21T04:22:32.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex closed 2023-10-21T04:22:32.268+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12762948908485355776:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.268+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.268+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.268+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:22:32.270+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.272+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:32.272+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:32.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex] 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex closed 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex] 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex closed 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex] 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex closed 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex ... 2023-10-21T04:22:32.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 152 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex ... 2023-10-21T04:22:32.277+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.277+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 8 Close Done 2023-10-21T04:22:32.277+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.277+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 5 Close Done 2023-10-21T04:22:32.277+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.277+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 6 Close Done 2023-10-21T04:22:32.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex ... 2023-10-21T04:22:32.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex ... 2023-10-21T04:22:32.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex ... 2023-10-21T04:22:32.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 163 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex ... 2023-10-21T04:22:32.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:22:32.288+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:22:32.288+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:22:32.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex 2023-10-21T04:22:32.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:22:32.288+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:22:32.288+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:22:32.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex 2023-10-21T04:22:32.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex] 2023-10-21T04:22:32.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex closed 2023-10-21T04:22:32.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:22:32.288+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:22:32.288+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:22:32.288+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex 2023-10-21T04:22:32.289+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:22:32.289+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:22:32.289+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:22:32.289+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex 2023-10-21T04:22:32.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex] 2023-10-21T04:22:32.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex closed 2023-10-21T04:22:32.289+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.289+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 2 Close Done 2023-10-21T04:22:32.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex ... 2023-10-21T04:22:32.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex ... 2023-10-21T04:22:32.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex ... 2023-10-21T04:22:32.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex ... 2023-10-21T04:22:32.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex stopped 2023-10-21T04:22:32.295+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.295+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.295+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.295+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.294+05:30 [Info] switched currmeta from 922 -> 922 force true 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.296+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 8 Close Done 2023-10-21T04:22:32.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex stopped 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex stopped 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex stopped 2023-10-21T04:22:32.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.299+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex] 2023-10-21T04:22:32.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex closed 2023-10-21T04:22:32.300+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#11375521620401506748:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.300+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 1 Close Done 2023-10-21T04:22:32.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex ... 2023-10-21T04:22:32.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex ... 2023-10-21T04:22:32.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_3.index destroyed 2023-10-21T04:22:32.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex ... 2023-10-21T04:22:32.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex ... 2023-10-21T04:22:32.332+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 3, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.332+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 3 Destroy Done 2023-10-21T04:22:32.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex ... 2023-10-21T04:22:32.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index destroyed 2023-10-21T04:22:32.338+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4, IndexDefnId 16162012140118277065. 2023-10-21T04:22:32.338+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 4 Destroy Done 2023-10-21T04:22:32.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex ... 2023-10-21T04:22:32.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex ... 2023-10-21T04:22:32.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex ... 2023-10-21T04:22:32.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 167 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex ... 2023-10-21T04:22:32.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index destroyed 2023-10-21T04:22:32.346+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8, IndexDefnId 16162012140118277065. 2023-10-21T04:22:32.347+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 8 Destroy Done 2023-10-21T04:22:32.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex ... 2023-10-21T04:22:32.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex ... 2023-10-21T04:22:32.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex ... 2023-10-21T04:22:32.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 169 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex ... 2023-10-21T04:22:32.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex ... 2023-10-21T04:22:32.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex ... 2023-10-21T04:22:32.362+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:32.363+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:32.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index destroyed 2023-10-21T04:22:32.385+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1, IndexDefnId 16162012140118277065. 2023-10-21T04:22:32.385+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 1 Destroy Done 2023-10-21T04:22:32.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex ... 2023-10-21T04:22:32.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex ... 2023-10-21T04:22:32.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.390+05:30 [Info] switched currmeta from 922 -> 923 force true 2023-10-21T04:22:32.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index destroyed 2023-10-21T04:22:32.404+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3, IndexDefnId 16162012140118277065. 2023-10-21T04:22:32.404+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 3 Destroy Done 2023-10-21T04:22:32.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_3.index destroyed 2023-10-21T04:22:32.412+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 3, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.412+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 3 Destroy Done 2023-10-21T04:22:32.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex ... 2023-10-21T04:22:32.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex ... 2023-10-21T04:22:32.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex ... 2023-10-21T04:22:32.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex ... 2023-10-21T04:22:32.406+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index destroyed 2023-10-21T04:22:32.420+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5, IndexDefnId 16162012140118277065. 2023-10-21T04:22:32.420+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 5 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_4.index destroyed 2023-10-21T04:22:32.420+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 4, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.421+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 4 Destroy Done 2023-10-21T04:22:32.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex ... 2023-10-21T04:22:32.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex ... 2023-10-21T04:22:32.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.425+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.434+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex] 2023-10-21T04:22:32.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex closed 2023-10-21T04:22:32.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.435+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 1 Close Done 2023-10-21T04:22:32.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex] 2023-10-21T04:22:32.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex closed 2023-10-21T04:22:32.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.436+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 7 Close Done 2023-10-21T04:22:32.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex] 2023-10-21T04:22:32.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex closed 2023-10-21T04:22:32.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.436+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 5 Close Done 2023-10-21T04:22:32.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex] 2023-10-21T04:22:32.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex closed 2023-10-21T04:22:32.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12762948908485355776:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.436+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 6 Close Done 2023-10-21T04:22:32.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex ... 2023-10-21T04:22:32.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_2.index destroyed 2023-10-21T04:22:32.444+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 2, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.445+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 2 Destroy Done 2023-10-21T04:22:32.444+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7, IndexDefnId 16162012140118277065. 2023-10-21T04:22:32.445+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 7 Destroy Done 2023-10-21T04:22:32.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index destroyed 2023-10-21T04:22:32.448+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6, IndexDefnId 16162012140118277065. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.451+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 6 Destroy Done 2023-10-21T04:22:32.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex ... 2023-10-21T04:22:32.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex ... 2023-10-21T04:22:32.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 142 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex ... 2023-10-21T04:22:32.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 156 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex ... 2023-10-21T04:22:32.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex ... 2023-10-21T04:22:32.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 158 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex ... 2023-10-21T04:22:32.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex ... 2023-10-21T04:22:32.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 173 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex ... 2023-10-21T04:22:32.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_6.index destroyed 2023-10-21T04:22:32.467+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 6, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.468+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 6 Destroy Done 2023-10-21T04:22:32.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex ... 2023-10-21T04:22:32.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex ... 2023-10-21T04:22:32.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex ... 2023-10-21T04:22:32.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 153 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex ... 2023-10-21T04:22:32.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_8.index destroyed 2023-10-21T04:22:32.486+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 8, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.486+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 8 Destroy Done 2023-10-21T04:22:32.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex ... 2023-10-21T04:22:32.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex ... 2023-10-21T04:22:32.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex ... 2023-10-21T04:22:32.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 170 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex ... 2023-10-21T04:22:32.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex ... 2023-10-21T04:22:32.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 140 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex ... 2023-10-21T04:22:32.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex ... 2023-10-21T04:22:32.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 149 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex ... 2023-10-21T04:22:32.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index destroyed 2023-10-21T04:22:32.496+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2, IndexDefnId 16162012140118277065. 2023-10-21T04:22:32.496+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 2 Destroy Done 2023-10-21T04:22:32.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex ... 2023-10-21T04:22:32.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 166 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex ... 2023-10-21T04:22:32.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex ... 2023-10-21T04:22:32.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 154 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex ... 2023-10-21T04:22:32.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex ... 2023-10-21T04:22:32.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 156 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex ... 2023-10-21T04:22:32.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex ... 2023-10-21T04:22:32.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 141 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex ... 2023-10-21T04:22:32.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_4.index destroyed 2023-10-21T04:22:32.509+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 4, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.509+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 4 Destroy Done 2023-10-21T04:22:32.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex ... 2023-10-21T04:22:32.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 144 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex ... 2023-10-21T04:22:32.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex ... 2023-10-21T04:22:32.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 157 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex ... 2023-10-21T04:22:32.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex ... 2023-10-21T04:22:32.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 171 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex ... 2023-10-21T04:22:32.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex ... 2023-10-21T04:22:32.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 143 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex ... 2023-10-21T04:22:32.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index/mainIndex sucessfully destroyed 2023-10-21T04:22:32.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_5.index destroyed 2023-10-21T04:22:32.519+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 5, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.519+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 5 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_5.index destroyed 2023-10-21T04:22:32.519+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 5, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.519+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 5 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_1.index destroyed 2023-10-21T04:22:32.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex ... 2023-10-21T04:22:32.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 160 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex ... 2023-10-21T04:22:32.520+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 1, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.520+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 1 Destroy Done 2023-10-21T04:22:32.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex ... 2023-10-21T04:22:32.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 145 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex ... 2023-10-21T04:22:32.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex ... 2023-10-21T04:22:32.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 146 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex ... 2023-10-21T04:22:32.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_8.index destroyed 2023-10-21T04:22:32.538+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 8, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.538+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 8 Destroy Done 2023-10-21T04:22:32.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex ... 2023-10-21T04:22:32.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 159 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex ... 2023-10-21T04:22:32.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_11375521620401506748_6.index destroyed 2023-10-21T04:22:32.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex ... 2023-10-21T04:22:32.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 161 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_7.index destroyed 2023-10-21T04:22:32.548+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 7, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.548+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 7 Destroy Done 2023-10-21T04:22:32.547+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11375521620401506748, PartitionId 6, IndexDefnId 2872726210871813325. 2023-10-21T04:22:32.548+05:30 [Info] Indexer::cleanupIndexData IndexInst 11375521620401506748 Partition 6 Destroy Done 2023-10-21T04:22:32.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_1.index destroyed 2023-10-21T04:22:32.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex ... 2023-10-21T04:22:32.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 162 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex ... 2023-10-21T04:22:32.555+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 1, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.555+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 1 Destroy Done 2023-10-21T04:22:32.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12762948908485355776_2.index destroyed 2023-10-21T04:22:32.557+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12762948908485355776, PartitionId 2, IndexDefnId 2374548411389959576. 2023-10-21T04:22:32.557+05:30 [Info] Indexer::cleanupIndexData IndexInst 12762948908485355776 Partition 2 Destroy Done 2023-10-21T04:22:32.560+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 5, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:22:32.560+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:22:32.560+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:22:32.560+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:22:32.560+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex 2023-10-21T04:22:32.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex stopped 2023-10-21T04:22:32.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.560+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 7, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:22:32.561+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:22:32.561+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:22:32.561+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:22:32.561+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex 2023-10-21T04:22:32.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex stopped 2023-10-21T04:22:32.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.562+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 8, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.562+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.562+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.562+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:22:32.562+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:22:32.562+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:22:32.562+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:22:32.562+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex 2023-10-21T04:22:32.562+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex stopped 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.563+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 1, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:22:32.563+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:22:32.563+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:22:32.563+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:22:32.563+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex 2023-10-21T04:22:32.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex stopped 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.563+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 759049595436785640 Defn: DefnId: 10585832761158485697 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:22:32.564+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 459.207525ms 2023-10-21T04:22:32.564+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 459.251849ms 2023-10-21T04:22:32.564+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 2, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:22:32.564+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:22:32.564+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:22:32.564+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:22:32.564+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex 2023-10-21T04:22:32.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex stopped 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.564+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 3, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.565+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:22:32.565+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:22:32.565+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:22:32.565+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:22:32.565+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex 2023-10-21T04:22:32.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex stopped 2023-10-21T04:22:32.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.566+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 4, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.566+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:22:32.566+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:22:32.566+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:22:32.567+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:22:32.567+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex 2023-10-21T04:22:32.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex stopped 2023-10-21T04:22:32.567+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.567+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.567+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.567+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.567+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 759049595436785640 2023-10-21T04:22:32.568+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 759049595436785640, PartitionId 6, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:22:32.568+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:22:32.568+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:22:32.568+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:22:32.568+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex 2023-10-21T04:22:32.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex stopped 2023-10-21T04:22:32.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.709+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex] 2023-10-21T04:22:32.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex closed 2023-10-21T04:22:32.710+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.710+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.710+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.710+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:22:32.710+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:22:32.710+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:22:32.711+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:22:32.711+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex 2023-10-21T04:22:32.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex stopped 2023-10-21T04:22:32.711+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.711+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.711+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.711+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.735+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 706.420744ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:32.736+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 14340086159258068923 instance id 4086959971053976291 real instance id 17357232428819395534 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:32.737+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 14340086159258068923 instance 17357232428819395534 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:32.737+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 14340086159258068923 instance id 17357232428819395534 2023-10-21T04:22:32.737+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 14340086159258068923 2023-10-21T04:22:32.743+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 17357232428819395534 &{1} 2023-10-21T04:22:32.743+05:30 [Info] Indexer::handleDropIndex - IndexInstId 17357232428819395534 2023-10-21T04:22:32.743+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 14340086159258068923, instId: 17357232428819395534 2023-10-21T04:22:32.744+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:32.744+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:32.744+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17357232428819395534 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:32.744+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [17357232428819395534] 2023-10-21T04:22:32.744+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [17357232428819395534] 2023-10-21T04:22:32.745+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:32.745+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [17357232428819395534] 2023-10-21T04:22:32.745+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex] 2023-10-21T04:22:32.745+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex] 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex] 2023-10-21T04:22:32.745+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex 2023-10-21T04:22:32.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex stopped 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.745+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.748+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex] 2023-10-21T04:22:32.748+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex] 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex] 2023-10-21T04:22:32.748+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex 2023-10-21T04:22:32.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex stopped 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.748+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.748+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex] 2023-10-21T04:22:32.749+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex] 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex] 2023-10-21T04:22:32.749+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex 2023-10-21T04:22:32.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex stopped 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.749+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex] 2023-10-21T04:22:32.749+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex] 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex] 2023-10-21T04:22:32.749+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex 2023-10-21T04:22:32.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex stopped 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.749+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.750+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex] 2023-10-21T04:22:32.750+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex] 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex] 2023-10-21T04:22:32.750+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex 2023-10-21T04:22:32.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex stopped 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.750+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.751+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex] 2023-10-21T04:22:32.755+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex] 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex] 2023-10-21T04:22:32.755+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex 2023-10-21T04:22:32.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex stopped 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.755+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.751+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:32.755+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 17357232428819395534 Defn: DefnId: 14340086159258068923 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:32.756+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 12.309511ms 2023-10-21T04:22:32.751+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex] 2023-10-21T04:22:32.756+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex] 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex] 2023-10-21T04:22:32.756+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex 2023-10-21T04:22:32.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex stopped 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.756+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 17357232428819395534 2023-10-21T04:22:32.751+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6, IndexDefnId 14340086159258068923. 2023-10-21T04:22:32.758+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.758+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.760+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex] 2023-10-21T04:22:32.760+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex 2023-10-21T04:22:32.760+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex] 2023-10-21T04:22:32.760+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex] 2023-10-21T04:22:32.760+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex 2023-10-21T04:22:32.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex stopped 2023-10-21T04:22:32.760+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.760+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.760+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.760+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.759+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [17357232428819395534] 2023-10-21T04:22:32.753+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex] 2023-10-21T04:22:32.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex closed 2023-10-21T04:22:32.764+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex] 2023-10-21T04:22:32.765+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex] 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex] 2023-10-21T04:22:32.765+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex 2023-10-21T04:22:32.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex stopped 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.765+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.766+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.766+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex] 2023-10-21T04:22:32.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex closed 2023-10-21T04:22:32.752+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex] 2023-10-21T04:22:32.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex closed 2023-10-21T04:22:32.768+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.769+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex ... 2023-10-21T04:22:32.768+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:32.770+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:22:32.770+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 14.765543ms 2023-10-21T04:22:32.771+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 27.354613ms 2023-10-21T04:22:32.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.777+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.777+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.777+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex] 2023-10-21T04:22:32.778+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex 2023-10-21T04:22:32.778+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex] 2023-10-21T04:22:32.778+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex] 2023-10-21T04:22:32.778+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex 2023-10-21T04:22:32.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex stopped 2023-10-21T04:22:32.778+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.778+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.778+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.778+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex] 2023-10-21T04:22:32.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex closed 2023-10-21T04:22:32.779+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 158 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex ... 2023-10-21T04:22:32.772+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.772+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.788+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.788+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex] 2023-10-21T04:22:32.788+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex closed 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex] 2023-10-21T04:22:32.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex] 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex] 2023-10-21T04:22:32.789+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex 2023-10-21T04:22:32.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex stopped 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.789+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex] 2023-10-21T04:22:32.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex closed 2023-10-21T04:22:32.790+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.790+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex] 2023-10-21T04:22:32.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex closed 2023-10-21T04:22:32.790+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.790+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.790+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.790+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex] 2023-10-21T04:22:32.790+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex 2023-10-21T04:22:32.790+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex] 2023-10-21T04:22:32.790+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex] 2023-10-21T04:22:32.790+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex 2023-10-21T04:22:32.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex stopped 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex] 2023-10-21T04:22:32.791+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex] 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex] 2023-10-21T04:22:32.791+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex 2023-10-21T04:22:32.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex stopped 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.791+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.797+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.797+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex] 2023-10-21T04:22:32.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex] 2023-10-21T04:22:32.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex closed 2023-10-21T04:22:32.830+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.801+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.808+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex closed 2023-10-21T04:22:32.830+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.832+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:32.833+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.841+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex] 2023-10-21T04:22:32.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex closed 2023-10-21T04:22:32.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.842+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.842+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.842+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:22:32.842+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.843+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.843+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex] 2023-10-21T04:22:32.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex ... 2023-10-21T04:22:32.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 176 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex ... 2023-10-21T04:22:32.849+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.849+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.849+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex 2023-10-21T04:22:32.849+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex] 2023-10-21T04:22:32.849+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex] 2023-10-21T04:22:32.849+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex 2023-10-21T04:22:32.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex stopped 2023-10-21T04:22:32.849+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.849+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.849+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.849+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex] 2023-10-21T04:22:32.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex closed 2023-10-21T04:22:32.851+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.851+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.851+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.851+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex] 2023-10-21T04:22:32.851+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex 2023-10-21T04:22:32.851+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex] 2023-10-21T04:22:32.852+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex] 2023-10-21T04:22:32.852+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex 2023-10-21T04:22:32.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex stopped 2023-10-21T04:22:32.852+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.852+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.852+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.852+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.852+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:32.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex] 2023-10-21T04:22:32.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex closed 2023-10-21T04:22:32.853+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.853+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.841+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex] 2023-10-21T04:22:32.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex closed 2023-10-21T04:22:32.853+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 117.79234ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:32.853+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.853+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex] 2023-10-21T04:22:32.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex 2023-10-21T04:22:32.854+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex] 2023-10-21T04:22:32.854+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex] 2023-10-21T04:22:32.854+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex 2023-10-21T04:22:32.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex stopped 2023-10-21T04:22:32.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.854+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.855+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.855+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:22:32.855+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:22:32.855+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:22:32.855+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex 2023-10-21T04:22:32.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex stopped 2023-10-21T04:22:32.855+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.855+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.856+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.856+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex] 2023-10-21T04:22:32.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex closed 2023-10-21T04:22:32.857+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.857+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex] 2023-10-21T04:22:32.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex closed 2023-10-21T04:22:32.857+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.857+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:32.857+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16013578430151540990 instance id 13452430413863876848 real instance id 4440544141961093904 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:32.866+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.869+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16013578430151540990 instance 4440544141961093904 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:32.869+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16013578430151540990 instance id 4440544141961093904 2023-10-21T04:22:32.869+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16013578430151540990 2023-10-21T04:22:32.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_5.index destroyed 2023-10-21T04:22:32.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex ... 2023-10-21T04:22:32.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex ... 2023-10-21T04:22:32.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex] 2023-10-21T04:22:32.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex closed 2023-10-21T04:22:32.876+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.876+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.876+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.876+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:22:32.877+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:22:32.877+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:22:32.877+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:22:32.877+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex 2023-10-21T04:22:32.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex stopped 2023-10-21T04:22:32.877+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.877+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.877+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.877+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.877+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.883+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 5, IndexDefnId 10585832761158485697. 2023-10-21T04:22:32.884+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 5 Destroy Done 2023-10-21T04:22:32.884+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex] 2023-10-21T04:22:32.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex closed 2023-10-21T04:22:32.886+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.897+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.897+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.898+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:22:32.898+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:22:32.902+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:22:32.898+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.888+05:30 [Info] switched currmeta from 923 -> 929 force true 2023-10-21T04:22:32.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:32.897+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex] 2023-10-21T04:22:32.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex closed 2023-10-21T04:22:32.918+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.918+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.919+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.920+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4440544141961093904 &{1} 2023-10-21T04:22:32.922+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4440544141961093904 2023-10-21T04:22:32.922+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16013578430151540990, instId: 4440544141961093904 2023-10-21T04:22:32.922+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:32.922+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:32.922+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4440544141961093904 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:32.923+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4440544141961093904] 2023-10-21T04:22:32.923+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4440544141961093904] 2023-10-21T04:22:32.923+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:32.923+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4440544141961093904] 2023-10-21T04:22:32.924+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex] 2023-10-21T04:22:32.924+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex 2023-10-21T04:22:32.924+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex] 2023-10-21T04:22:32.924+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex] 2023-10-21T04:22:32.924+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex 2023-10-21T04:22:32.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex stopped 2023-10-21T04:22:32.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.924+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.934+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.934+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.934+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.934+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex] 2023-10-21T04:22:32.934+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex 2023-10-21T04:22:32.934+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex] 2023-10-21T04:22:32.934+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex] 2023-10-21T04:22:32.934+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex 2023-10-21T04:22:32.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex stopped 2023-10-21T04:22:32.934+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.935+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex] 2023-10-21T04:22:32.935+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex 2023-10-21T04:22:32.935+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex] 2023-10-21T04:22:32.935+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex] 2023-10-21T04:22:32.935+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex 2023-10-21T04:22:32.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex stopped 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.935+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.935+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex] 2023-10-21T04:22:32.936+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex 2023-10-21T04:22:32.936+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex] 2023-10-21T04:22:32.936+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex] 2023-10-21T04:22:32.936+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex 2023-10-21T04:22:32.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex stopped 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.936+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex] 2023-10-21T04:22:32.936+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex 2023-10-21T04:22:32.936+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex] 2023-10-21T04:22:32.936+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex] 2023-10-21T04:22:32.936+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex 2023-10-21T04:22:32.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex stopped 2023-10-21T04:22:32.936+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.937+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex] 2023-10-21T04:22:32.937+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex 2023-10-21T04:22:32.937+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex] 2023-10-21T04:22:32.937+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex] 2023-10-21T04:22:32.937+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex 2023-10-21T04:22:32.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex stopped 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.937+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.938+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:32.938+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4440544141961093904 Defn: DefnId: 16013578430151540990 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:32.939+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 17.02441ms 2023-10-21T04:22:32.939+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.939+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.939+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.939+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.940+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex] 2023-10-21T04:22:32.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex closed 2023-10-21T04:22:32.977+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.977+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:22:32.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex] 2023-10-21T04:22:32.978+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex 2023-10-21T04:22:32.978+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex] 2023-10-21T04:22:32.978+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex] 2023-10-21T04:22:32.978+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex 2023-10-21T04:22:32.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex stopped 2023-10-21T04:22:32.978+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.978+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:32.979+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:32.979+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:32.941+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6, IndexDefnId 16013578430151540990. 2023-10-21T04:22:32.979+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.979+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:32.979+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:32.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex] 2023-10-21T04:22:32.979+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex 2023-10-21T04:22:32.979+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex] 2023-10-21T04:22:32.980+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex] 2023-10-21T04:22:32.980+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex 2023-10-21T04:22:32.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex stopped 2023-10-21T04:22:32.941+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4440544141961093904 2023-10-21T04:22:32.941+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [4440544141961093904] 2023-10-21T04:22:32.949+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:22:32.956+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.957+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:32.966+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.968+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:32.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:32.989+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:32.993+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 1, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:22:32.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex] 2023-10-21T04:22:32.998+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex 2023-10-21T04:22:32.998+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:32.999+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:33.042+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:22:33.043+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 103.909629ms 2023-10-21T04:22:33.043+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 121.074975ms 2023-10-21T04:22:33.047+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex ... 2023-10-21T04:22:33.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex ... 2023-10-21T04:22:33.058+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex ... 2023-10-21T04:22:33.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex ... 2023-10-21T04:22:33.065+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.065+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.065+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.065+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex closed 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:22:33.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex stopped 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.080+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.081+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:22:33.081+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:22:33.081+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:22:33.081+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:22:33.081+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex 2023-10-21T04:22:33.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex stopped 2023-10-21T04:22:33.081+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.081+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.082+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.082+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex] 2023-10-21T04:22:33.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex closed 2023-10-21T04:22:33.099+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#759049595436785640:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.099+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.099+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.099+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.099+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:22:33.099+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:22:33.099+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:22:33.099+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:22:33.099+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex 2023-10-21T04:22:33.099+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex stopped 2023-10-21T04:22:33.099+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.099+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.099+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.106+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex] 2023-10-21T04:22:33.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex closed 2023-10-21T04:22:33.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.107+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex ... 2023-10-21T04:22:33.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 162 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex ... 2023-10-21T04:22:33.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex] 2023-10-21T04:22:33.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex closed 2023-10-21T04:22:33.108+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.108+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.108+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.108+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex] 2023-10-21T04:22:33.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_8.index destroyed 2023-10-21T04:22:33.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex] 2023-10-21T04:22:33.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex closed 2023-10-21T04:22:33.121+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.121+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.121+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.121+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex] 2023-10-21T04:22:33.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex closed 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex] 2023-10-21T04:22:33.122+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex 2023-10-21T04:22:33.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex stopped 2023-10-21T04:22:33.123+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.123+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.123+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.123+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex] 2023-10-21T04:22:33.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex closed 2023-10-21T04:22:33.124+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.124+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.124+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.124+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex] 2023-10-21T04:22:33.124+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex 2023-10-21T04:22:33.124+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex] 2023-10-21T04:22:33.124+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex] 2023-10-21T04:22:33.125+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex 2023-10-21T04:22:33.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex stopped 2023-10-21T04:22:33.125+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.125+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.125+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.125+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.110+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex] 2023-10-21T04:22:33.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex closed 2023-10-21T04:22:33.125+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.126+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.126+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.126+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex] 2023-10-21T04:22:33.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex 2023-10-21T04:22:33.126+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex] 2023-10-21T04:22:33.126+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex] 2023-10-21T04:22:33.126+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex 2023-10-21T04:22:33.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex stopped 2023-10-21T04:22:33.126+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.126+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.126+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.126+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex ... 2023-10-21T04:22:33.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 180 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex ... 2023-10-21T04:22:33.129+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.130+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex ... 2023-10-21T04:22:33.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 160 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex ... 2023-10-21T04:22:33.130+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex stopped 2023-10-21T04:22:33.131+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex stopped 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex stopped 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.132+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.113+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.113+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 8, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.439+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 8 Destroy Done 2023-10-21T04:22:33.115+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex ... 2023-10-21T04:22:33.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex ... 2023-10-21T04:22:33.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex] 2023-10-21T04:22:33.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex closed 2023-10-21T04:22:33.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex] 2023-10-21T04:22:33.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex closed 2023-10-21T04:22:33.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex] 2023-10-21T04:22:33.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex closed 2023-10-21T04:22:33.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex ... 2023-10-21T04:22:33.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex ... 2023-10-21T04:22:33.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_6.index destroyed 2023-10-21T04:22:33.443+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 6, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.443+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 6 Destroy Done 2023-10-21T04:22:33.443+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.443+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.133+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.133+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.134+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:33.443+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:33.139+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 285.767376ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:33.443+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 15431181238515168303 instance id 13554123636232028903 real instance id 13987057765505463258 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:33.443+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 15431181238515168303 instance 13987057765505463258 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:22:33.443+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 15431181238515168303 instance id 13987057765505463258 2023-10-21T04:22:33.443+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 15431181238515168303 2023-10-21T04:22:33.446+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.446+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex] 2023-10-21T04:22:33.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex closed 2023-10-21T04:22:33.447+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.448+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex ... 2023-10-21T04:22:33.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 161 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex ... 2023-10-21T04:22:33.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex ... 2023-10-21T04:22:33.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex ... 2023-10-21T04:22:33.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex] 2023-10-21T04:22:33.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex closed 2023-10-21T04:22:33.450+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.450+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.450+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.451+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex] 2023-10-21T04:22:33.189+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.190+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.197+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.245+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.247+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.249+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.250+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.322+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:33.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex] 2023-10-21T04:22:33.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex closed 2023-10-21T04:22:33.457+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.457+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.457+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.457+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex] 2023-10-21T04:22:33.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex ... 2023-10-21T04:22:33.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex ... 2023-10-21T04:22:33.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex] 2023-10-21T04:22:33.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex closed 2023-10-21T04:22:33.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.457+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.451+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 13987057765505463258 &{1} 2023-10-21T04:22:33.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.458+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex 2023-10-21T04:22:33.458+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex] 2023-10-21T04:22:33.458+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex] 2023-10-21T04:22:33.458+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex 2023-10-21T04:22:33.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex stopped 2023-10-21T04:22:33.458+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.458+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.458+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.458+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.447+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.458+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex ... 2023-10-21T04:22:33.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex ... 2023-10-21T04:22:33.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex] 2023-10-21T04:22:33.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex closed 2023-10-21T04:22:33.459+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.459+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex ... 2023-10-21T04:22:33.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 157 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex ... 2023-10-21T04:22:33.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_7.index destroyed 2023-10-21T04:22:33.461+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 7, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.461+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 7 Destroy Done 2023-10-21T04:22:33.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex] 2023-10-21T04:22:33.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex closed 2023-10-21T04:22:33.462+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.457+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.462+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.462+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex ... 2023-10-21T04:22:33.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex] 2023-10-21T04:22:33.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex ... 2023-10-21T04:22:33.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.458+05:30 [Info] Indexer::handleDropIndex - IndexInstId 13987057765505463258 2023-10-21T04:22:33.462+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 15431181238515168303, instId: 13987057765505463258 2023-10-21T04:22:33.462+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [13987057765505463258] 2023-10-21T04:22:33.462+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [13987057765505463258] 2023-10-21T04:22:33.462+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.462+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.463+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex] 2023-10-21T04:22:33.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex closed 2023-10-21T04:22:33.463+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.463+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex ... 2023-10-21T04:22:33.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex ... 2023-10-21T04:22:33.463+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex] 2023-10-21T04:22:33.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex ... 2023-10-21T04:22:33.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 178 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex ... 2023-10-21T04:22:33.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_4.index destroyed 2023-10-21T04:22:33.465+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 4, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.465+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 4 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_1.index destroyed 2023-10-21T04:22:33.465+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 1, IndexDefnId 10585832761158485697. 2023-10-21T04:22:33.465+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 1 Destroy Done 2023-10-21T04:22:33.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex ... 2023-10-21T04:22:33.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex ... 2023-10-21T04:22:33.462+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:33.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.466+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex 2023-10-21T04:22:33.466+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex] 2023-10-21T04:22:33.466+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex] 2023-10-21T04:22:33.466+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex 2023-10-21T04:22:33.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex stopped 2023-10-21T04:22:33.466+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.466+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.466+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.466+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.466+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [13987057765505463258] 2023-10-21T04:22:33.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex] 2023-10-21T04:22:33.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex closed 2023-10-21T04:22:33.466+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.466+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex ... 2023-10-21T04:22:33.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex ... 2023-10-21T04:22:33.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex ... 2023-10-21T04:22:33.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex ... 2023-10-21T04:22:33.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_1.index destroyed 2023-10-21T04:22:33.470+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 1, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.470+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 1 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_7.index destroyed 2023-10-21T04:22:33.470+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 7, IndexDefnId 10585832761158485697. 2023-10-21T04:22:33.470+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 7 Destroy Done 2023-10-21T04:22:33.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex ... 2023-10-21T04:22:33.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex ... 2023-10-21T04:22:33.470+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex] 2023-10-21T04:22:33.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex closed 2023-10-21T04:22:33.472+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.472+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex ... 2023-10-21T04:22:33.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex ... 2023-10-21T04:22:33.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex ... 2023-10-21T04:22:33.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex ... 2023-10-21T04:22:33.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex ... 2023-10-21T04:22:33.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 179 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex ... 2023-10-21T04:22:33.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex] 2023-10-21T04:22:33.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex closed 2023-10-21T04:22:33.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex] 2023-10-21T04:22:33.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex closed 2023-10-21T04:22:33.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex] 2023-10-21T04:22:33.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex closed 2023-10-21T04:22:33.485+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex] 2023-10-21T04:22:33.485+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex closed 2023-10-21T04:22:33.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.485+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.485+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex 2023-10-21T04:22:33.485+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex] 2023-10-21T04:22:33.485+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex] 2023-10-21T04:22:33.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex 2023-10-21T04:22:33.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex 2023-10-21T04:22:33.485+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex] 2023-10-21T04:22:33.486+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex] 2023-10-21T04:22:33.486+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex] 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex closed 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex] 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex closed 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex ... 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex ... 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_5.index destroyed 2023-10-21T04:22:33.486+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 5, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.486+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 5 Destroy Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_6.index destroyed 2023-10-21T04:22:33.486+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 6, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.486+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 6 Destroy Done 2023-10-21T04:22:33.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex stopped 2023-10-21T04:22:33.486+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.486+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.486+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 1 Close Done 2023-10-21T04:22:33.486+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.486+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.486+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.486+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#759049595436785640:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.487+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.487+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex stopped 2023-10-21T04:22:33.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_8.index destroyed 2023-10-21T04:22:33.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex ... 2023-10-21T04:22:33.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 175 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex ... 2023-10-21T04:22:33.489+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 8, IndexDefnId 10585832761158485697. 2023-10-21T04:22:33.489+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 8 Destroy Done 2023-10-21T04:22:33.491+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex] 2023-10-21T04:22:33.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex closed 2023-10-21T04:22:33.492+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.492+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex ... 2023-10-21T04:22:33.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex ... 2023-10-21T04:22:33.493+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_4.index destroyed 2023-10-21T04:22:33.494+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 4, IndexDefnId 10585832761158485697. 2023-10-21T04:22:33.494+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 4 Destroy Done 2023-10-21T04:22:33.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex ... 2023-10-21T04:22:33.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex ... 2023-10-21T04:22:33.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex] 2023-10-21T04:22:33.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex closed 2023-10-21T04:22:33.503+05:30 [Info] bucket_1/idx_partitioned/Backstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.503+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex ... 2023-10-21T04:22:33.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex ... 2023-10-21T04:22:33.504+05:30 [Info] switched currmeta from 929 -> 935 force true 2023-10-21T04:22:33.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex ... 2023-10-21T04:22:33.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_8.index destroyed 2023-10-21T04:22:33.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex ... 2023-10-21T04:22:33.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex ... 2023-10-21T04:22:33.508+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 8, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.508+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 8 Destroy Done 2023-10-21T04:22:33.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex ... 2023-10-21T04:22:33.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex ... 2023-10-21T04:22:33.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex ... 2023-10-21T04:22:33.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 164 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex ... 2023-10-21T04:22:33.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex ... 2023-10-21T04:22:33.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex ... 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex ... 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex ... 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex ... 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex ... 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex ... 2023-10-21T04:22:33.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 159 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex ... 2023-10-21T04:22:33.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex ... 2023-10-21T04:22:33.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex ... 2023-10-21T04:22:33.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex ... 2023-10-21T04:22:33.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex ... 2023-10-21T04:22:33.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_2.index destroyed 2023-10-21T04:22:33.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex ... 2023-10-21T04:22:33.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex ... 2023-10-21T04:22:33.523+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 2, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.523+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 2 Destroy Done 2023-10-21T04:22:33.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex ... 2023-10-21T04:22:33.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 163 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex ... 2023-10-21T04:22:33.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex ... 2023-10-21T04:22:33.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex ... 2023-10-21T04:22:33.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_3.index destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_4.index destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_17357232428819395534_3.index destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_5.index destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_7.index destroyed 2023-10-21T04:22:33.547+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 3, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.547+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 3 Destroy Done 2023-10-21T04:22:33.549+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 4, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.549+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 4 Destroy Done 2023-10-21T04:22:33.551+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17357232428819395534, PartitionId 3, IndexDefnId 14340086159258068923. 2023-10-21T04:22:33.551+05:30 [Info] Indexer::cleanupIndexData IndexInst 17357232428819395534 Partition 3 Destroy Done 2023-10-21T04:22:33.551+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 7, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.551+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 7 Destroy Done 2023-10-21T04:22:33.551+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 5, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.551+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 5 Destroy Done 2023-10-21T04:22:33.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex ... 2023-10-21T04:22:33.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex ... 2023-10-21T04:22:33.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_1.index destroyed 2023-10-21T04:22:33.552+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 1, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.552+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 1 Destroy Done 2023-10-21T04:22:33.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex ... 2023-10-21T04:22:33.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 177 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex ... 2023-10-21T04:22:33.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_6.index destroyed 2023-10-21T04:22:33.558+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 6, IndexDefnId 10585832761158485697. 2023-10-21T04:22:33.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex ... 2023-10-21T04:22:33.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 182 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex ... 2023-10-21T04:22:33.559+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 6 Destroy Done 2023-10-21T04:22:33.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_3.index destroyed 2023-10-21T04:22:33.566+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 3, IndexDefnId 10585832761158485697. 2023-10-21T04:22:33.566+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 3 Destroy Done 2023-10-21T04:22:33.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex ... 2023-10-21T04:22:33.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 181 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex ... 2023-10-21T04:22:33.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_759049595436785640_2.index destroyed 2023-10-21T04:22:33.568+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 759049595436785640, PartitionId 2, IndexDefnId 10585832761158485697. 2023-10-21T04:22:33.568+05:30 [Info] Indexer::cleanupIndexData IndexInst 759049595436785640 Partition 2 Destroy Done 2023-10-21T04:22:33.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex ... 2023-10-21T04:22:33.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex ... 2023-10-21T04:22:33.575+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.575+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_4440544141961093904_2.index destroyed 2023-10-21T04:22:33.575+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4440544141961093904, PartitionId 2, IndexDefnId 16013578430151540990. 2023-10-21T04:22:33.575+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 6, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.575+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.575+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:22:33.576+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:22:33.576+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:22:33.576+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:22:33.576+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:22:33.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex stopped 2023-10-21T04:22:33.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.578+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 8, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:22:33.578+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:22:33.578+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:22:33.578+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:22:33.578+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:22:33.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex stopped 2023-10-21T04:22:33.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.579+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 7, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:22:33.579+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:22:33.579+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:22:33.579+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:22:33.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex stopped 2023-10-21T04:22:33.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.581+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 1, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:22:33.581+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:22:33.575+05:30 [Info] Indexer::cleanupIndexData IndexInst 4440544141961093904 Partition 2 Destroy Done 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:22:33.581+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:22:33.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex stopped 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.581+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 2, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:22:33.581+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:22:33.581+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:22:33.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex stopped 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.582+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 4 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[1:{1 4 [:9123] []} 2:{2 4 [:9123] []} 3:{3 4 [:9123] []} 4:{4 4 [:9123] []} 5:{5 4 [:9123] []} 6:{6 4 [:9123] []} 7:{7 4 [:9123] []} 8:{8 4 [:9123] []}] 8 KEY 0} 2023-10-21T04:22:33.582+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 124.272423ms 2023-10-21T04:22:33.582+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 124.312864ms 2023-10-21T04:22:33.582+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 3, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:22:33.582+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:22:33.582+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:22:33.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex stopped 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.582+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 4, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:22:33.582+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:22:33.582+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:22:33.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex stopped 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.583+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.583+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 5, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.583+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.583+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.583+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.583+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 13987057765505463258 2023-10-21T04:22:33.583+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:22:33.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex closed 2023-10-21T04:22:33.583+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.583+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:22:33.584+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:22:33.584+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:22:33.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex stopped 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.584+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:22:33.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex closed 2023-10-21T04:22:33.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.589+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:22:33.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 152 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:22:33.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:22:33.592+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:22:33.592+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:22:33.592+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:22:33.592+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:22:33.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex stopped 2023-10-21T04:22:33.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:22:33.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 169 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:22:33.600+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 156.696999ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:33.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index destroyed 2023-10-21T04:22:33.614+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 6, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.614+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 6 Destroy Done 2023-10-21T04:22:33.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:22:33.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex closed 2023-10-21T04:22:33.644+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.644+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.644+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:33.644+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.644+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:22:33.644+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:22:33.644+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:22:33.644+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:22:33.644+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:22:33.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex stopped 2023-10-21T04:22:33.645+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.645+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.645+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.645+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.644+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:33.647+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 5004576769623010783 &{1} 2023-10-21T04:22:33.647+05:30 [Info] Indexer::handleDropIndex - IndexInstId 5004576769623010783 2023-10-21T04:22:33.647+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 12833670934079048728, instId: 5004576769623010783 2023-10-21T04:22:33.648+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:33.648+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:33.648+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:33.649+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [5004576769623010783] 2023-10-21T04:22:33.649+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [5004576769623010783] 2023-10-21T04:22:33.649+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:33.649+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [5004576769623010783] 2023-10-21T04:22:33.650+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:33.650+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:33.650+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.237531ms 2023-10-21T04:22:33.652+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0, IndexDefnId 12833670934079048728. 2023-10-21T04:22:33.652+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.652+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.653+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:22:33.653+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:22:33.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:22:33.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:22:33.653+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:22:33.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex stopped 2023-10-21T04:22:33.653+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.654+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.654+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.654+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.654+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 5004576769623010783 2023-10-21T04:22:33.656+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [5004576769623010783] 2023-10-21T04:22:33.661+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:33.661+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:22:33.661+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 10.698659ms 2023-10-21T04:22:33.661+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 14.141988ms 2023-10-21T04:22:33.664+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:22:33.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex closed 2023-10-21T04:22:33.665+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.665+05:30 [Info] Indexer::cleanupIndexData IndexInst 5004576769623010783 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex ... 2023-10-21T04:22:33.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex ... 2023-10-21T04:22:33.666+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:22:33.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex closed 2023-10-21T04:22:33.668+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.668+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index destroyed 2023-10-21T04:22:33.680+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0, IndexDefnId 12833670934079048728. 2023-10-21T04:22:33.680+05:30 [Info] Indexer::cleanupIndexData IndexInst 5004576769623010783 Partition 0 Destroy Done 2023-10-21T04:22:33.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:22:33.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 154 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:22:33.691+05:30 [Info] switched currmeta from 935 -> 938 force true 2023-10-21T04:22:33.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:22:33.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 171 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:22:33.713+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index destroyed 2023-10-21T04:22:33.719+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 8, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.719+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 8 Destroy Done 2023-10-21T04:22:33.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:22:33.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex closed 2023-10-21T04:22:33.720+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.720+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.720+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.720+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:22:33.720+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:22:33.720+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:22:33.745+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:22:33.745+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:22:33.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex stopped 2023-10-21T04:22:33.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.768+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:22:33.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex closed 2023-10-21T04:22:33.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:22:33.775+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:22:33.775+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:22:33.775+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:22:33.775+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:22:33.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex stopped 2023-10-21T04:22:33.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.769+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:33.776+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:33.770+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:22:33.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex closed 2023-10-21T04:22:33.777+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.777+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:22:33.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 153 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:22:33.777+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:22:33.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex closed 2023-10-21T04:22:33.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:22:33.749+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:22:33.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex closed 2023-10-21T04:22:33.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.771+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:22:33.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex closed 2023-10-21T04:22:33.779+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.779+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.779+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.779+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:22:33.780+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.780+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:22:33.789+05:30 [Info] switched currmeta from 938 -> 940 force true 2023-10-21T04:22:33.797+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.800+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 199.647729ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:33.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:22:33.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 170 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:22:33.810+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:22:33.810+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:22:33.810+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:22:33.810+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:22:33.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex stopped 2023-10-21T04:22:33.810+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.810+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.810+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.810+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.810+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:22:33.811+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:22:33.811+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:22:33.811+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:22:33.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex stopped 2023-10-21T04:22:33.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.812+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:22:33.812+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:22:33.812+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:22:33.812+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:22:33.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex stopped 2023-10-21T04:22:33.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:22:33.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex closed 2023-10-21T04:22:33.813+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.813+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:22:33.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 155 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:22:33.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.826+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.826+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:22:33.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex closed 2023-10-21T04:22:33.826+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.826+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:22:33.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 156 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:22:33.827+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:22:33.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex closed 2023-10-21T04:22:33.829+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.829+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:22:33.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 150 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:22:33.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index destroyed 2023-10-21T04:22:33.833+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 7, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.833+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 7 Destroy Done 2023-10-21T04:22:33.835+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:22:33.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 172 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:22:33.841+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:33.847+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:33.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:22:33.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex closed 2023-10-21T04:22:33.848+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.848+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:22:33.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 149 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:22:33.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex sucessfully destroyed 2023-10-21T04:22:33.859+05:30 [Info] switched currmeta from 940 -> 941 force true 2023-10-21T04:22:33.863+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 18415077564616412601 &{1} 2023-10-21T04:22:33.863+05:30 [Info] Indexer::handleDropIndex - IndexInstId 18415077564616412601 2023-10-21T04:22:33.863+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17878371269610391239, instId: 18415077564616412601 2023-10-21T04:22:33.863+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [18415077564616412601] 2023-10-21T04:22:33.863+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [18415077564616412601] 2023-10-21T04:22:33.863+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:33.864+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [18415077564616412601] 2023-10-21T04:22:33.864+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 18415077564616412601 Defn: DefnId: 17878371269610391239 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:22:33.871+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 8.390383ms 2023-10-21T04:22:33.873+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 18415077564616412601, PartitionId 0, IndexDefnId 17878371269610391239. 2023-10-21T04:22:33.873+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.873+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.873+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:22:33.873+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 18415077564616412601 2023-10-21T04:22:33.875+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 11.820571ms 2023-10-21T04:22:33.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex sucessfully destroyed 2023-10-21T04:22:33.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:22:33.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex closed 2023-10-21T04:22:33.878+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:22:33.878+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:22:33.878+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:22:33.878+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex 2023-10-21T04:22:33.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex stopped 2023-10-21T04:22:33.878+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.878+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.878+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.878+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:33.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:22:33.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 173 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:22:33.879+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:33.880+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:33.880+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:33.880+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:33.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:22:33.880+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:22:33.881+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:22:33.881+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:22:33.881+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:22:33.881+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex stopped 2023-10-21T04:22:33.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:33.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:33.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:33.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index destroyed 2023-10-21T04:22:33.882+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 1, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.882+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 1 Destroy Done 2023-10-21T04:22:33.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index destroyed 2023-10-21T04:22:33.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:22:33.884+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 2, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.884+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 2 Destroy Done 2023-10-21T04:22:33.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 167 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:22:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index destroyed 2023-10-21T04:22:33.888+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 4, IndexDefnId 15431181238515168303. 2023-10-21T04:22:33.888+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 4 Destroy Done 2023-10-21T04:22:33.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:22:33.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 166 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:22:33.891+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:33.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:22:34.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex closed 2023-10-21T04:22:34.195+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.195+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:22:34.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 151 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:22:33.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index destroyed 2023-10-21T04:22:33.897+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex] 2023-10-21T04:22:34.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex closed 2023-10-21T04:22:34.196+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 3, IndexDefnId 15431181238515168303. 2023-10-21T04:22:34.196+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 3 Destroy Done 2023-10-21T04:22:34.196+05:30 [Info] bucket_%2/#primary_defer/Mainstore#18415077564616412601:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.196+05:30 [Info] Indexer::cleanupIndexData IndexInst 18415077564616412601 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:33.916+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 116.363708ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:33.946+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.196+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:33.995+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9999920047851361, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:22:34.200+05:30 [Info] switched currmeta from 941 -> 948 force true 2023-10-21T04:22:34.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex sucessfully destroyed 2023-10-21T04:22:34.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:22:34.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 168 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:22:34.202+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 11790381330881939912 &{1} 2023-10-21T04:22:34.202+05:30 [Info] Indexer::handleDropIndex - IndexInstId 11790381330881939912 2023-10-21T04:22:34.202+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17489537288579938718, instId: 11790381330881939912 2023-10-21T04:22:34.203+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [11790381330881939912] 2023-10-21T04:22:34.203+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [11790381330881939912] 2023-10-21T04:22:34.203+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.203+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [11790381330881939912] 2023-10-21T04:22:34.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index destroyed 2023-10-21T04:22:34.204+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 5, IndexDefnId 15431181238515168303. 2023-10-21T04:22:34.204+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 5 Destroy Done 2023-10-21T04:22:34.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex ... 2023-10-21T04:22:34.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 184 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex ... 2023-10-21T04:22:34.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_18415077564616412601_0.index destroyed 2023-10-21T04:22:34.206+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 18415077564616412601, PartitionId 0, IndexDefnId 17878371269610391239. 2023-10-21T04:22:34.206+05:30 [Info] Indexer::cleanupIndexData IndexInst 18415077564616412601 Partition 0 Destroy Done 2023-10-21T04:22:34.207+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 11790381330881939912 Defn: DefnId: 17489537288579938718 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:22:34.207+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.298979ms 2023-10-21T04:22:34.207+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11790381330881939912, PartitionId 0, IndexDefnId 17489537288579938718. 2023-10-21T04:22:34.207+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.207+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.207+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.207+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 11790381330881939912 2023-10-21T04:22:34.207+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.207+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.208+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 5.360812ms 2023-10-21T04:22:34.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:22:34.208+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:22:34.208+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:22:34.208+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:22:34.208+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex 2023-10-21T04:22:34.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex stopped 2023-10-21T04:22:34.208+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.208+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.208+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.208+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.210+05:30 [Info] switched currmeta from 948 -> 949 force true 2023-10-21T04:22:34.215+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.215+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.216+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 19.500245ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.219+05:30 [Info] switched currmeta from 949 -> 950 force true 2023-10-21T04:22:34.227+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex] 2023-10-21T04:22:34.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex closed 2023-10-21T04:22:34.227+05:30 [Info] bucket_1/#primary_defer/Mainstore#11790381330881939912:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.227+05:30 [Info] Indexer::cleanupIndexData IndexInst 11790381330881939912 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex ... 2023-10-21T04:22:34.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 174 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex ... 2023-10-21T04:22:34.230+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4611895227247053723 &{1} 2023-10-21T04:22:34.230+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4611895227247053723 2023-10-21T04:22:34.230+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 1844709904168202817, instId: 4611895227247053723 2023-10-21T04:22:34.230+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4611895227247053723] 2023-10-21T04:22:34.230+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4611895227247053723] 2023-10-21T04:22:34.230+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.230+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4611895227247053723] 2023-10-21T04:22:34.231+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 4611895227247053723 Defn: DefnId: 1844709904168202817 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:22:34.231+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 925.229µs 2023-10-21T04:22:34.231+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 1.008017ms 2023-10-21T04:22:34.231+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4611895227247053723, PartitionId 0, IndexDefnId 1844709904168202817. 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:22:34.231+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:22:34.231+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex 2023-10-21T04:22:34.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex stopped 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.231+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.232+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4611895227247053723 2023-10-21T04:22:34.238+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex] 2023-10-21T04:22:34.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex closed 2023-10-21T04:22:34.238+05:30 [Info] bucket_%2/#primary_defer/Mainstore#4611895227247053723:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.239+05:30 [Info] Indexer::cleanupIndexData IndexInst 4611895227247053723 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_11790381330881939912_0.index destroyed 2023-10-21T04:22:34.239+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11790381330881939912, PartitionId 0, IndexDefnId 17489537288579938718. 2023-10-21T04:22:34.239+05:30 [Info] Indexer::cleanupIndexData IndexInst 11790381330881939912 Partition 0 Destroy Done 2023-10-21T04:22:34.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex ... 2023-10-21T04:22:34.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 164 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex ... 2023-10-21T04:22:34.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_4611895227247053723_0.index destroyed 2023-10-21T04:22:34.248+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4611895227247053723, PartitionId 0, IndexDefnId 1844709904168202817. 2023-10-21T04:22:34.248+05:30 [Info] Indexer::cleanupIndexData IndexInst 4611895227247053723 Partition 0 Destroy Done 2023-10-21T04:22:34.248+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 32.365445ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.262+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15667952562388529486 &{1} 2023-10-21T04:22:34.262+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15667952562388529486 2023-10-21T04:22:34.262+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 12338036531800130039, instId: 15667952562388529486 2023-10-21T04:22:34.262+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:34.263+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.263+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:34.263+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15667952562388529486] 2023-10-21T04:22:34.263+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15667952562388529486] 2023-10-21T04:22:34.263+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.263+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15667952562388529486] 2023-10-21T04:22:34.263+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.263+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.264+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.363506ms 2023-10-21T04:22:34.264+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0, IndexDefnId 12338036531800130039. 2023-10-21T04:22:34.264+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.264+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.264+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:22:34.264+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:22:34.278+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:22:34.278+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:22:34.267+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [15667952562388529486] 2023-10-21T04:22:34.267+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15667952562388529486 2023-10-21T04:22:34.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:22:34.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex stopped 2023-10-21T04:22:34.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.281+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.288+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:22:34.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex closed 2023-10-21T04:22:34.289+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.289+05:30 [Info] Indexer::cleanupIndexData IndexInst 15667952562388529486 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex ... 2023-10-21T04:22:34.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex ... 2023-10-21T04:22:34.289+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:34.290+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:22:34.290+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 26.150708ms 2023-10-21T04:22:34.290+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 27.672507ms 2023-10-21T04:22:34.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index destroyed 2023-10-21T04:22:34.294+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0, IndexDefnId 12338036531800130039. 2023-10-21T04:22:34.294+05:30 [Info] Indexer::cleanupIndexData IndexInst 15667952562388529486 Partition 0 Destroy Done 2023-10-21T04:22:34.299+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 51.336349ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.308+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 13739166377675070800 &{1} 2023-10-21T04:22:34.309+05:30 [Info] Indexer::handleDropIndex - IndexInstId 13739166377675070800 2023-10-21T04:22:34.309+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 7874378322029374196, instId: 13739166377675070800 2023-10-21T04:22:34.309+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:34.309+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.310+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13739166377675070800 Name: #primary Keyspace: bucket_%2:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:34.310+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [13739166377675070800] 2023-10-21T04:22:34.310+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [13739166377675070800] 2023-10-21T04:22:34.310+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.310+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [13739166377675070800] 2023-10-21T04:22:34.310+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.311+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 13739166377675070800 Defn: DefnId: 7874378322029374196 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.311+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.105553ms 2023-10-21T04:22:34.311+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0, IndexDefnId 7874378322029374196. 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex] 2023-10-21T04:22:34.311+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex] 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex] 2023-10-21T04:22:34.311+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex 2023-10-21T04:22:34.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex stopped 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.311+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.312+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.311+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 13739166377675070800 2023-10-21T04:22:34.312+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [13739166377675070800] 2023-10-21T04:22:34.316+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:34.316+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:22:34.316+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.903074ms 2023-10-21T04:22:34.316+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 7.123149ms 2023-10-21T04:22:34.316+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex] 2023-10-21T04:22:34.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex closed 2023-10-21T04:22:34.316+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.316+05:30 [Info] Indexer::cleanupIndexData IndexInst 13739166377675070800 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex ... 2023-10-21T04:22:34.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex ... 2023-10-21T04:22:34.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_13739166377675070800_0.index destroyed 2023-10-21T04:22:34.322+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13739166377675070800, PartitionId 0, IndexDefnId 7874378322029374196. 2023-10-21T04:22:34.323+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.323+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.324+05:30 [Info] Indexer::cleanupIndexData IndexInst 13739166377675070800 Partition 0 Destroy Done 2023-10-21T04:22:34.348+05:30 [Info] switched currmeta from 950 -> 955 force true 2023-10-21T04:22:34.360+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.360+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.366+05:30 [Info] switched currmeta from 955 -> 955 force true 2023-10-21T04:22:34.368+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 69.00647ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.386+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 7494249569853685720 &{1} 2023-10-21T04:22:34.386+05:30 [Info] Indexer::handleDropIndex - IndexInstId 7494249569853685720 2023-10-21T04:22:34.386+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 1571318351652255745, instId: 7494249569853685720 2023-10-21T04:22:34.386+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:34.386+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.386+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7494249569853685720 Name: #primary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:34.387+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [7494249569853685720] 2023-10-21T04:22:34.387+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [7494249569853685720] 2023-10-21T04:22:34.387+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.387+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [7494249569853685720] 2023-10-21T04:22:34.387+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.387+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0, IndexDefnId 1571318351652255745. 2023-10-21T04:22:34.388+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.388+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.388+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.387+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 7494249569853685720 Defn: DefnId: 1571318351652255745 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.388+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.330006ms 2023-10-21T04:22:34.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex] 2023-10-21T04:22:34.389+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex 2023-10-21T04:22:34.389+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex] 2023-10-21T04:22:34.389+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex] 2023-10-21T04:22:34.389+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex 2023-10-21T04:22:34.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex stopped 2023-10-21T04:22:34.390+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.390+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.389+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [7494249569853685720] 2023-10-21T04:22:34.390+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.392+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.389+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 7494249569853685720 2023-10-21T04:22:34.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.393+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.395+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:34.395+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:22:34.395+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 7.118749ms 2023-10-21T04:22:34.395+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 9.828759ms 2023-10-21T04:22:34.396+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex] 2023-10-21T04:22:34.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex closed 2023-10-21T04:22:34.397+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.397+05:30 [Info] Indexer::cleanupIndexData IndexInst 7494249569853685720 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex ... 2023-10-21T04:22:34.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex ... 2023-10-21T04:22:34.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7494249569853685720_0.index destroyed 2023-10-21T04:22:34.406+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7494249569853685720, PartitionId 0, IndexDefnId 1571318351652255745. 2023-10-21T04:22:34.406+05:30 [Info] Indexer::cleanupIndexData IndexInst 7494249569853685720 Partition 0 Destroy Done 2023-10-21T04:22:34.410+05:30 [Info] switched currmeta from 955 -> 957 force true 2023-10-21T04:22:34.445+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.446+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.453+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 84.002168ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.462+05:30 [Info] switched currmeta from 957 -> 957 force true 2023-10-21T04:22:34.477+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 2296526085582506469 &{1} 2023-10-21T04:22:34.477+05:30 [Info] Indexer::handleDropIndex - IndexInstId 2296526085582506469 2023-10-21T04:22:34.477+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 8857269605176042330, instId: 2296526085582506469 2023-10-21T04:22:34.477+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.477+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.479+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:34.479+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.479+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:34.480+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [2296526085582506469] 2023-10-21T04:22:34.480+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [2296526085582506469] 2023-10-21T04:22:34.480+05:30 [Info] switched currmeta from 957 -> 959 force true 2023-10-21T04:22:34.490+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.490+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.493+05:30 [Info] switched currmeta from 959 -> 959 force true 2023-10-21T04:22:34.480+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.498+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [2296526085582506469] 2023-10-21T04:22:34.498+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.498+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.498+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 21.604909ms 2023-10-21T04:22:34.498+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0, IndexDefnId 8857269605176042330. 2023-10-21T04:22:34.498+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.498+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.498+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:22:34.498+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:22:34.498+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:22:34.498+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:22:34.499+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:22:34.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex stopped 2023-10-21T04:22:34.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.499+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.499+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [2296526085582506469] 2023-10-21T04:22:34.499+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 2296526085582506469 2023-10-21T04:22:34.507+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:22:34.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex closed 2023-10-21T04:22:34.516+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.516+05:30 [Info] Indexer::cleanupIndexData IndexInst 2296526085582506469 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.508+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:34.517+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:22:34.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex ... 2023-10-21T04:22:34.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex ... 2023-10-21T04:22:34.517+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 18.403574ms 2023-10-21T04:22:34.810+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 333.792705ms 2023-10-21T04:22:34.537+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.810+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.545+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 92.398543ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index destroyed 2023-10-21T04:22:34.815+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0, IndexDefnId 8857269605176042330. 2023-10-21T04:22:34.816+05:30 [Info] Indexer::cleanupIndexData IndexInst 2296526085582506469 Partition 0 Destroy Done 2023-10-21T04:22:34.822+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12265503634969032942 &{1} 2023-10-21T04:22:34.822+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12265503634969032942 2023-10-21T04:22:34.822+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 9619308266130016313, instId: 12265503634969032942 2023-10-21T04:22:34.823+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12265503634969032942] 2023-10-21T04:22:34.823+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12265503634969032942] 2023-10-21T04:22:34.823+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.823+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12265503634969032942] 2023-10-21T04:22:34.823+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 12265503634969032942 Defn: DefnId: 9619308266130016313 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c2%/d IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:22:34.823+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 634.886µs 2023-10-21T04:22:34.823+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 677.856µs 2023-10-21T04:22:34.823+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12265503634969032942, PartitionId 0, IndexDefnId 9619308266130016313. 2023-10-21T04:22:34.823+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.823+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.823+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:22:34.823+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:22:34.823+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:22:34.823+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:22:34.823+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex 2023-10-21T04:22:34.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex stopped 2023-10-21T04:22:34.823+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12265503634969032942 2023-10-21T04:22:34.823+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.823+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.823+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.823+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.833+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex] 2023-10-21T04:22:34.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex closed 2023-10-21T04:22:34.833+05:30 [Info] bucket_1/#primary_defer/Mainstore#12265503634969032942:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.833+05:30 [Info] Indexer::cleanupIndexData IndexInst 12265503634969032942 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex ... 2023-10-21T04:22:34.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 184 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex ... 2023-10-21T04:22:34.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_12265503634969032942_0.index destroyed 2023-10-21T04:22:34.838+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12265503634969032942, PartitionId 0, IndexDefnId 9619308266130016313. 2023-10-21T04:22:34.838+05:30 [Info] Indexer::cleanupIndexData IndexInst 12265503634969032942 Partition 0 Destroy Done 2023-10-21T04:22:34.847+05:30 [Info] switched currmeta from 959 -> 964 force true 2023-10-21T04:22:34.854+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.854+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.859+05:30 [Info] switched currmeta from 964 -> 964 force true 2023-10-21T04:22:34.862+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 51.101602ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.864+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.864+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.867+05:30 [Info] switched currmeta from 964 -> 965 force true 2023-10-21T04:22:34.873+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4992762551574016313 &{1} 2023-10-21T04:22:34.874+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4992762551574016313 2023-10-21T04:22:34.874+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 3674078063353481907, instId: 4992762551574016313 2023-10-21T04:22:34.874+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:34.874+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.874+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 1 2023-10-21T04:22:34.874+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4992762551574016313] 2023-10-21T04:22:34.874+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4992762551574016313] 2023-10-21T04:22:34.875+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.875+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4992762551574016313] 2023-10-21T04:22:34.875+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.875+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.875+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0, IndexDefnId 3674078063353481907. 2023-10-21T04:22:34.875+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4992762551574016313 2023-10-21T04:22:34.875+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.876+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.876+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:22:34.876+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:22:34.876+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:22:34.876+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:22:34.876+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:22:34.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex stopped 2023-10-21T04:22:34.876+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.876+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.876+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.877+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.877+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.877+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.875+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.717648ms 2023-10-21T04:22:34.877+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId Instances [4992762551574016313] 2023-10-21T04:22:34.880+05:30 [Info] switched currmeta from 965 -> 966 force true 2023-10-21T04:22:34.880+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceId 2023-10-21T04:22:34.881+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:22:34.881+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 5.362481ms 2023-10-21T04:22:34.881+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 7.180091ms 2023-10-21T04:22:34.883+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:22:34.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex closed 2023-10-21T04:22:34.884+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.884+05:30 [Info] Indexer::cleanupIndexData IndexInst 4992762551574016313 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex ... 2023-10-21T04:22:34.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex ... 2023-10-21T04:22:34.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index destroyed 2023-10-21T04:22:34.891+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0, IndexDefnId 3674078063353481907. 2023-10-21T04:22:34.891+05:30 [Info] Indexer::cleanupIndexData IndexInst 4992762551574016313 Partition 0 Destroy Done 2023-10-21T04:22:34.891+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.891+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.895+05:30 [Info] switched currmeta from 966 -> 967 force true 2023-10-21T04:22:34.913+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.913+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.931+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 68.943718ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:34.931+05:30 [Info] switched currmeta from 967 -> 967 force true 2023-10-21T04:22:34.947+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 7582778950442222621 &{1} 2023-10-21T04:22:34.948+05:30 [Info] Indexer::handleDropIndex - IndexInstId 7582778950442222621 2023-10-21T04:22:34.948+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 5103684059183126577, instId: 7582778950442222621 2023-10-21T04:22:34.948+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:34.948+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.948+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7582778950442222621 Name: #primary Keyspace: bucket_1:_default:c2% State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:34.948+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [7582778950442222621] 2023-10-21T04:22:34.949+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [7582778950442222621] 2023-10-21T04:22:34.949+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:34.949+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [7582778950442222621] 2023-10-21T04:22:34.949+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:22:34.949+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:22:34.949+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.950+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc00e1af380 }] 2023-10-21T04:22:34.950+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 28469 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:34.950+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:22:34.950+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:22:34.950+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 1 New 0 2023-10-21T04:22:34.950+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:22:34.950+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:22:34.951+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.587319ms 2023-10-21T04:22:34.951+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0, IndexDefnId 5103684059183126577. 2023-10-21T04:22:34.951+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:34.951+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:34.952+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:34.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex] 2023-10-21T04:22:34.952+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex 2023-10-21T04:22:34.952+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex] 2023-10-21T04:22:34.952+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex] 2023-10-21T04:22:34.952+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex 2023-10-21T04:22:34.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex stopped 2023-10-21T04:22:34.952+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:34.952+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:34.953+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:34.953+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:34.953+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1] 2023-10-21T04:22:34.953+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 7582778950442222621 2023-10-21T04:22:34.956+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 0 186517626268899 55 2023-10-21T04:22:34.956+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.956+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 1 178840478062605 59 2023-10-21T04:22:34.956+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.956+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 2 148075163102199 54 2023-10-21T04:22:34.957+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.958+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 6 242327631330054 62 2023-10-21T04:22:34.958+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.958+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 7 191903564840858 63 2023-10-21T04:22:34.958+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.958+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 8 50961953886801 45 2023-10-21T04:22:34.958+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.959+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 3 215374859564337 35 2023-10-21T04:22:34.959+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.959+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 4 28605411869539 58 2023-10-21T04:22:34.959+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.959+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 5 39552632604610 52 2023-10-21T04:22:34.959+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.960+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 12 4429827396609 42 2023-10-21T04:22:34.960+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.960+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 13 183148377680315 47 2023-10-21T04:22:34.960+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.960+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 14 226814108379280 73 2023-10-21T04:22:34.960+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.960+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 17 55290066231400 51 2023-10-21T04:22:34.960+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.960+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 10 274410200699189 55 2023-10-21T04:22:34.962+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.962+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 11 66881023717063 51 2023-10-21T04:22:34.962+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.963+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 15 127185710837163 50 2023-10-21T04:22:34.963+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.963+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 16 257031483891250 59 2023-10-21T04:22:34.963+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.963+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 21 209771427122473 48 2023-10-21T04:22:34.964+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.964+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:34.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex] 2023-10-21T04:22:34.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex closed 2023-10-21T04:22:34.965+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is disabled 2023-10-21T04:22:34.965+05:30 [Info] bucket_1/idx_partitioned/Mainstore#9912003609070511414:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:34.965+05:30 [Info] Indexer::cleanupIndexData IndexInst 7582778950442222621 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:34.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex ... 2023-10-21T04:22:34.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex ... 2023-10-21T04:22:34.968+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_1] 2023-10-21T04:22:34.968+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 28469 2023-10-21T04:22:34.968+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 17.020196ms 2023-10-21T04:22:34.968+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 20.769433ms 2023-10-21T04:22:34.965+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 22 260036488774725 51 2023-10-21T04:22:34.969+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.969+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 23 216462437971699 58 2023-10-21T04:22:34.969+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.969+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 24 160961818008256 59 2023-10-21T04:22:34.970+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.970+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 25 280874278672777 71 2023-10-21T04:22:34.970+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.970+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 18 242578104468179 63 2023-10-21T04:22:34.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:34.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_7582778950442222621_0.index destroyed 2023-10-21T04:22:34.974+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7582778950442222621, PartitionId 0, IndexDefnId 5103684059183126577. 2023-10-21T04:22:34.974+05:30 [Info] Indexer::cleanupIndexData IndexInst 7582778950442222621 Partition 0 Destroy Done 2023-10-21T04:22:34.970+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.977+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 19 210306088158507 48 2023-10-21T04:22:34.977+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.977+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 20 230239359684242 50 2023-10-21T04:22:34.977+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.978+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 30 99433130332778 58 2023-10-21T04:22:34.978+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.979+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 31 217381278987664 67 2023-10-21T04:22:34.980+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.980+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 32 82469662608686 60 2023-10-21T04:22:34.981+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.985+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 33 136958029391999 66 2023-10-21T04:22:34.985+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.986+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 26 219132834127088 52 2023-10-21T04:22:34.986+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.986+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 27 184056281482149 50 2023-10-21T04:22:34.986+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.986+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 28 102578907139247 58 2023-10-21T04:22:34.986+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.986+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 29 45957426777107 57 2023-10-21T04:22:34.986+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.986+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 39 222555662236724 68 2023-10-21T04:22:34.987+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.987+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 34 28417669987133 59 2023-10-21T04:22:34.987+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.987+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 36 253845164956592 58 2023-10-21T04:22:34.987+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.987+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 37 253494845579900 54 2023-10-21T04:22:34.988+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.988+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 41 259381390638966 58 2023-10-21T04:22:34.988+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.988+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 40 5915864305121 61 2023-10-21T04:22:34.994+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:34.988+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:34.994+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:34.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9999792515146394 2023-10-21T04:22:35.007+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 76.084695ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:35.021+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 1265729768899070816 &{1} 2023-10-21T04:22:35.063+05:30 [Info] Indexer::handleDropIndex - IndexInstId 1265729768899070816 2023-10-21T04:22:35.063+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 5197198800322775808, instId: 1265729768899070816 2023-10-21T04:22:35.063+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:35.063+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:35.064+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1265729768899070816 Name: #primary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 4 ReplicaId: 0 2023-10-21T04:22:35.066+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [1265729768899070816] 2023-10-21T04:22:35.066+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [1265729768899070816] 2023-10-21T04:22:35.066+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:35.067+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [1265729768899070816] 2023-10-21T04:22:35.067+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:22:35.067+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:22:35.067+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:35.067+05:30 [Info] MutationStreamReader:Shutdown StreamReader MAINT_STREAM 2023-10-21T04:22:35.068+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 total received key-versions: map[0:1 1:2 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:1 10:0 11:0 12:0 13:1 14:1 15:0 16:1 17:1 18:0 19:0 20:1 21:0 22:0 23:1 24:0 25:0 26:0 27:1 28:0 29:0 30:0 31:0 32:0 33:1 34:0 35:0 36:0 37:0 38:0 39:1 40:1 41:0 42:0 43:0 44:0 45:2 46:1 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:22:35.068+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 latest sequence numbers: map[0:53 1:63 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:63 10:0 11:0 12:0 13:54 14:61 15:0 16:55 17:46 18:0 19:0 20:51 21:0 22:0 23:52 24:0 25:0 26:0 27:55 28:0 29:0 30:0 31:0 32:0 33:55 34:0 35:0 36:0 37:0 38:0 39:50 40:69 41:0 42:0 43:0 44:0 45:63 46:55 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:22:35.068+05:30 [Error] DATP[->dataport ":9123"] Accept() Error: accept tcp4 0.0.0.0:9123: use of closed network connection 2023-10-21T04:22:35.063+05:30 [Info] switched currmeta from 967 -> 971 force true 2023-10-21T04:22:35.068+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0, IndexDefnId 5197198800322775808. 2023-10-21T04:22:35.069+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9123->127.0.0.1:43508: use of closed network connection 2023-10-21T04:22:35.069+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:43508" closed ! 2023-10-21T04:22:35.093+05:30 [Info] DATP[->dataport ":9123"] ... stopped 2023-10-21T04:22:35.093+05:30 [Error] DATP[->dataport ":9123"] worker "127.0.0.1:43508" exit: read tcp4 127.0.0.1:9123->127.0.0.1:43508: use of closed network connection 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:35.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex] 2023-10-21T04:22:35.093+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex] 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex] 2023-10-21T04:22:35.093+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex 2023-10-21T04:22:35.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex stopped 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:35.093+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:35.094+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 32478 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:22:35.094+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:22:35.094+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:22:35.094+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:22:35.095+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 31.918932ms 2023-10-21T04:22:35.095+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 42 252931432434966 46 2023-10-21T04:22:35.095+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.095+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 35 70213628416850 47 2023-10-21T04:22:35.095+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.095+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 38 188107826286720 60 2023-10-21T04:22:35.096+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.096+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:35.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex] 2023-10-21T04:22:35.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex closed 2023-10-21T04:22:35.096+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is disabled 2023-10-21T04:22:35.096+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#15425360657090425347:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:35.096+05:30 [Info] Indexer::cleanupIndexData IndexInst 1265729768899070816 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:35.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex ... 2023-10-21T04:22:35.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex ... 2023-10-21T04:22:35.097+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 44 158545118983293 67 2023-10-21T04:22:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.097+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 45 12301831552869 68 2023-10-21T04:22:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.097+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 46 242994859770444 54 2023-10-21T04:22:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.097+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 47 271074273345339 53 2023-10-21T04:22:35.097+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.097+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 48 227378777358533 59 2023-10-21T04:22:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.098+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 49 215467668600630 34 2023-10-21T04:22:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.098+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 50 207367350302991 64 2023-10-21T04:22:35.098+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.098+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 43 146267938201125 55 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 56 264835102027079 54 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 59 125009072873642 53 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 9 225280996447309 55 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 53 204867593700407 56 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 58 16044440367455 53 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 63 133748675726909 56 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 51 200436537417141 54 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 62 10527297481171 57 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.099+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 61 106932309485550 53 2023-10-21T04:22:35.099+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.100+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 52 188718367400378 56 2023-10-21T04:22:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.100+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 55 170056388266420 71 2023-10-21T04:22:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.100+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 57 129411666769864 55 2023-10-21T04:22:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.100+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 60 75566139929862 57 2023-10-21T04:22:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.100+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 54 279362078165491 55 2023-10-21T04:22:35.100+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:22:35.098+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 1265729768899070816 2023-10-21T04:22:35.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:35.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_1265729768899070816_0.index destroyed 2023-10-21T04:22:35.101+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 1265729768899070816, PartitionId 0, IndexDefnId 5197198800322775808. 2023-10-21T04:22:35.101+05:30 [Info] Indexer::cleanupIndexData IndexInst 1265729768899070816 Partition 0 Destroy Done 2023-10-21T04:22:35.099+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2] 2023-10-21T04:22:35.104+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.104+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.110+05:30 [Info] switched currmeta from 971 -> 971 force true 2023-10-21T04:22:35.118+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.118+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.122+05:30 [Info] switched currmeta from 971 -> 971 force true 2023-10-21T04:22:35.122+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4c6e4b64d0c01f9d6431a65ac36fc024 KeyspaceIds [bucket_%2]. Err genServer.closed 2023-10-21T04:22:35.122+05:30 [Error] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:22:35.122+05:30 [Info] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Treating genServer.closed As Success 2023-10-21T04:22:35.122+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 32478 2023-10-21T04:22:35.122+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 23.42094ms 2023-10-21T04:22:35.122+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 100.222932ms 2023-10-21T04:22:35.145+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 137.426289ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:35.152+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 14271070195512901425 &{1} 2023-10-21T04:22:35.152+05:30 [Info] Indexer::handleDropIndex - IndexInstId 14271070195512901425 2023-10-21T04:22:35.152+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 11619165464560280276, instId: 14271070195512901425 2023-10-21T04:22:35.152+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [14271070195512901425] 2023-10-21T04:22:35.152+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [14271070195512901425] 2023-10-21T04:22:35.152+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:35.152+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [14271070195512901425] 2023-10-21T04:22:35.153+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 0 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:22:35.153+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 775.59µs 2023-10-21T04:22:35.153+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 815.221µs 2023-10-21T04:22:35.153+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14271070195512901425, PartitionId 0, IndexDefnId 11619165464560280276. 2023-10-21T04:22:35.153+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:35.153+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:35.154+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:35.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:22:35.154+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:22:35.154+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:22:35.153+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 14271070195512901425 2023-10-21T04:22:35.162+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.176+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.175+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:22:35.176+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:22:35.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex stopped 2023-10-21T04:22:35.176+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:35.176+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:35.176+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:35.176+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:35.180+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 35.880218ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:35.181+05:30 [Info] switched currmeta from 971 -> 974 force true 2023-10-21T04:22:35.182+05:30 [Info] ShardRebalancer::initiateLocalShardCleanup Initiating clean-up on local file system for ttid: ShardTokendc:78:31:73:6:81:ad:f2, shards: map[2864853750198994630:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/2864853750198994630/shards/shard2864853750198994630 12177293282243154215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardTokendc_78_31_73_6_81_ad_f2/12177293282243154215/shards/shard12177293282243154215] 2023-10-21T04:22:35.185+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:35.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:22:35.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex closed 2023-10-21T04:22:35.601+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:35.601+05:30 [Info] Indexer::cleanupIndexData IndexInst 14271070195512901425 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:35.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:22:35.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 164 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:22:35.602+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [12177293282243154215 2864853750198994630] 2023-10-21T04:22:35.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:22:35.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index destroyed 2023-10-21T04:22:35.609+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14271070195512901425, PartitionId 0, IndexDefnId 11619165464560280276. 2023-10-21T04:22:35.609+05:30 [Info] Indexer::cleanupIndexData IndexInst 14271070195512901425 Partition 0 Destroy Done 2023-10-21T04:22:35.616+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 16733302726553098964 &{1} 2023-10-21T04:22:35.616+05:30 [Info] Indexer::handleDropIndex - IndexInstId 16733302726553098964 2023-10-21T04:22:35.616+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 13856704970496601958, instId: 16733302726553098964 2023-10-21T04:22:35.616+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [16733302726553098964] 2023-10-21T04:22:35.616+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [16733302726553098964] 2023-10-21T04:22:35.616+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697841977675544947 bucket_1:1697841976960500515] 2023-10-21T04:22:35.616+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [16733302726553098964] 2023-10-21T04:22:35.616+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 4 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 4 ReplicaId: 1 PartitionContainer: &{map[0:{0 4 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:22:35.616+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 469.779µs 2023-10-21T04:22:35.616+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 511.481µs 2023-10-21T04:22:35.616+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16733302726553098964, PartitionId 0, IndexDefnId 13856704970496601958. 2023-10-21T04:22:35.617+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:22:35.617+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:22:35.617+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:22:35.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:22:35.617+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:22:35.617+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:22:35.617+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 16733302726553098964 2023-10-21T04:22:35.653+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 52.065218ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:35.655+05:30 [Info] ShardRebalancer::initiateLocalShardCleanup Initiating clean-up on local file system for ttid: ShardToken62:b9:b5:7e:51:46:53:89, shards: map[7659339677706884510:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/7659339677706884510/shards/shard7659339677706884510 10890719122371003479:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/88b52e29e23a9bc9a1f18d56ee3fbbd4_ShardToken62_b9_b5_7e_51_46_53_89/10890719122371003479/shards/shard10890719122371003479] 2023-10-21T04:22:35.655+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [10890719122371003479 7659339677706884510] 2023-10-21T04:22:35.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.662+05:30 [Info] switched currmeta from 974 -> 989 force true 2023-10-21T04:22:35.674+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.674+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.677+05:30 [Info] switched currmeta from 989 -> 989 force true 2023-10-21T04:22:35.682+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.682+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.687+05:30 [Info] switched currmeta from 989 -> 989 force true 2023-10-21T04:22:35.706+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:22:35.706+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:22:35.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex stopped 2023-10-21T04:22:35.706+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:22:35.706+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:22:35.706+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:22:35.706+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:22:35.719+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.719+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.719+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:22:35.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:22:35.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex closed 2023-10-21T04:22:35.720+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:22:35.720+05:30 [Info] Indexer::cleanupIndexData IndexInst 16733302726553098964 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:22:35.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:22:35.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 174 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:22:35.723+05:30 [Info] switched currmeta from 989 -> 990 force true 2023-10-21T04:22:35.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:22:35.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index destroyed 2023-10-21T04:22:35.723+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16733302726553098964, PartitionId 0, IndexDefnId 13856704970496601958. 2023-10-21T04:22:35.723+05:30 [Info] Indexer::cleanupIndexData IndexInst 16733302726553098964 Partition 0 Destroy Done 2023-10-21T04:22:35.773+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.773+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.804+05:30 [Info] switched currmeta from 990 -> 993 force true 2023-10-21T04:22:35.815+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.815+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.817+05:30 [Info] switched currmeta from 993 -> 993 force true 2023-10-21T04:22:35.824+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.824+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.832+05:30 [Info] switched currmeta from 993 -> 993 force true 2023-10-21T04:22:35.864+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.865+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.871+05:30 [Info] switched currmeta from 993 -> 995 force true 2023-10-21T04:22:35.902+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.902+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.910+05:30 [Info] switched currmeta from 995 -> 995 force true 2023-10-21T04:22:35.959+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.959+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.961+05:30 [Info] switched currmeta from 995 -> 997 force true 2023-10-21T04:22:35.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.984+05:30 [Info] switched currmeta from 997 -> 997 force true 2023-10-21T04:22:35.992+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9999998748688625 2023-10-21T04:22:35.993+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.993+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.995+05:30 [Info] switched currmeta from 997 -> 998 force true 2023-10-21T04:22:35.999+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:35.999+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:22:35.999+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 12177293282243154215 2023-10-21T04:22:35.999+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2864853750198994630 2023-10-21T04:22:35.999+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 10890719122371003479 2023-10-21T04:22:35.999+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 7659339677706884510 2023-10-21T04:22:35.999+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 7659339677706884510 2023-10-21T04:22:35.999+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 12177293282243154215 2023-10-21T04:22:35.999+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2864853750198994630 2023-10-21T04:22:35.999+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [12177293282243154215 2864853750198994630], elapsed: 397.312662ms 2023-10-21T04:22:35.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:35.999+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 (12177293282243154215) removed 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swapper stopped 2023-10-21T04:22:36.000+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': stopped 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.000+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.000+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 10890719122371003479 2023-10-21T04:22:36.001+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [10890719122371003479 7659339677706884510], elapsed: 345.091058ms 2023-10-21T04:22:36.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:36.001+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 (10890719122371003479) removed 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.001+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swapper stopped 2023-10-21T04:22:36.001+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': stopped 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.002+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.002+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.002+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.002+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.002+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.003+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.Close: Done close 2023-10-21T04:22:36.003+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.003+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.003+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.003+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.003+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.004+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.Close: Done close 2023-10-21T04:22:36.004+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.004+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.004+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.006+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.004+05:30 [Info] switched currmeta from 998 -> 998 force true 2023-10-21T04:22:36.004+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.Close: Done close 2023-10-21T04:22:36.006+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.007+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.007+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.007+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : All instance closed 2023-10-21T04:22:36.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shutdown completed 2023-10-21T04:22:36.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:36.008+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 (10890719122371003479) removed 2023-10-21T04:22:36.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : All instance closed 2023-10-21T04:22:36.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : All instance destroyed 2023-10-21T04:22:36.015+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.Close: Done close 2023-10-21T04:22:36.015+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.015+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.015+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : All instance closed 2023-10-21T04:22:36.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shutdown completed 2023-10-21T04:22:36.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:36.015+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 (12177293282243154215) removed 2023-10-21T04:22:36.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : All instance closed 2023-10-21T04:22:36.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : All instance destroyed 2023-10-21T04:22:36.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:22:36.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:36.030+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 (2864853750198994630) removed 2023-10-21T04:22:36.031+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.031+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.031+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.031+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.031+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.031+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.031+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:22:36.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:36.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.033+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 (7659339677706884510) removed 2023-10-21T04:22:36.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.034+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.034+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.034+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.Close: Done close 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.036+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.036+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.036+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.036+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:22:36.036+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:22:36.036+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.036+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.037+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.037+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:22:36.037+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:22:36.038+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:22:36.038+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.Close: Done close 2023-10-21T04:22:36.040+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:22:36.040+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:22:36.040+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:22:36.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.040+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.040+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.041+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.Close: Done close 2023-10-21T04:22:36.044+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.044+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.044+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.044+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:22:36.044+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.Close: Begin close 2023-10-21T04:22:36.046+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.Close: Done close 2023-10-21T04:22:36.046+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.046+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.046+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : All instance closed 2023-10-21T04:22:36.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shutdown completed 2023-10-21T04:22:36.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:36.046+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 (7659339677706884510) removed 2023-10-21T04:22:36.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : All instance closed 2023-10-21T04:22:36.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : All instance destroyed 2023-10-21T04:22:36.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:22:36.055+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': Cleaned up Tenant with name:bucket_%2_7007f0ab442ac00e52d057a30cd22644 2023-10-21T04:22:36.055+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [10890719122371003479 7659339677706884510], elapsed(sec): 0.399854058 2023-10-21T04:22:36.056+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:22:36.056+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:22:36.056+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:22:36.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : All instance closed 2023-10-21T04:22:36.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shutdown completed 2023-10-21T04:22:36.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:22:36.057+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 (2864853750198994630) removed 2023-10-21T04:22:36.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : All daemons stopped 2023-10-21T04:22:36.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : All instance closed 2023-10-21T04:22:36.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : All instance destroyed 2023-10-21T04:22:36.074+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.074+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:22:36.074+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:22:36.074+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_DONE] 2023-10-21T04:22:36.074+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 23.674µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:36.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:22:36.083+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': Cleaned up Tenant with name:bucket_1_db19c9050b4ab1232e6bc6654011618f 2023-10-21T04:22:36.083+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [12177293282243154215 2864853750198994630], elapsed(sec): 0.480783914 2023-10-21T04:22:36.125+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.125+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:22:36.125+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:22:36.125+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_DONE] 2023-10-21T04:22:36.125+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.915µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:36.277+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken62:b9:b5:7e:51:46:53:89 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken80:25:58:aa:f5:3d:42:3a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16489953270406880971 RealInstId: 14579235200590321485 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7256866756526982695 RealInstId: 10477796393418268202 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5701304422421248176 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18286344351415703073 RealInstId: 12762948908485355776 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 4086959971053976291 RealInstId: 17357232428819395534 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13554123636232028903 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18415077564616412601 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 4611895227247053723 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 13739166377675070800 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 1265729768899070816 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.303+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken62:b9:b5:7e:51:46:53:89 2023-10-21T04:22:36.349+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokendc:78:31:73:6:81:ad:f2 MasterId: bada28845870208460ab46b431032b01 SourceId: 4c6e4b64d0c01f9d6431a65ac36fc024 (127.0.0.1:9003) DestId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12652336003508392720 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17196107967903578507 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15512663964869546873 RealInstId: 5744899830895973442 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 1882546579949915688 RealInstId: 11375521620401506748 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 15074233131683912490 RealInstId: 759049595436785640 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 13452430413863876848 RealInstId: 4440544141961093904 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 11790381330881939912 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 7494249569853685720 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 12265503634969032942 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 7582778950442222621 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.385+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb7:ff:f0:86:22:97:7a:96 MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokendc:78:31:73:6:81:ad:f2 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5821535925846794556 RealInstId: 12510230312641977282 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 18214879127132723017 RealInstId: 16101189959443764523 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 17431410840353003128 RealInstId: 6919182200795062103 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 12091013817258869926 RealInstId: 5048773469222931610 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10320862121459941622 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 16723059298582460800 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8852459821271515836 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 7536553203860469796 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 2311200026499605920 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 1071957724495241089 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.412+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardTokendc:78:31:73:6:81:ad:f2 2023-10-21T04:22:36.463+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.607+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken80:25:58:aa:f5:3d:42:3a MasterId: bada28845870208460ab46b431032b01 SourceId: 458c2b8d760618b2a2f6a998f4c93e74 (127.0.0.1:9004) DestId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6708884665435111421 RealInstId: 5642987086862451891 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 7851315966801359237 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 6833281568105161438 RealInstId: 15461292247655814798 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 10248166972642413696 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 9323587521762890531 RealInstId: 12516458704114824147 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 5091980711413443680 RealInstId: 11863658526331737681 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [1 2 3 4 5 6 7 8] Versions: [5 5 5 5 5 5 5 5] InstId: 11304328130226754712 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [5] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [5] InstId: 8060737836868569583 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] InstId: 749864910613428348 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c2%/d Partitions: [0] Versions: [5] InstId: 2332114746415979372 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [5] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.733+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken80:25:58:aa:f5:3d:42:3a 2023-10-21T04:22:36.734+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardTokenb7:ff:f0:86:22:97:7a:96 2023-10-21T04:22:36.761+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken 2023-10-21T04:22:36.761+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token Deleted. Mark Done. 2023-10-21T04:22:36.761+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:22:36.762+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:22:36.762+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:22:36.762+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:22:36.766+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:22:36.766+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken30:b4:a5:29:32:8b:e6:df MasterId: SourceId: DestId: RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken80:25:58:aa:f5:3d:42:3a SiblingTokenId: ShardToken62:b9:b5:7e:51:46:53:89 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.766+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken51:b0:f9:2b:92:f4:ae:dc MasterId: SourceId: DestId: RebalId: 88b52e29e23a9bc9a1f18d56ee3fbbd4 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokendc:78:31:73:6:81:ad:f2 SiblingTokenId: ShardTokenb7:ff:f0:86:22:97:7a:96 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:22:36.771+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:22:36.771+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:22:36.771+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.569µs 2023-10-21T04:22:36.771+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:22:36.771+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:22:36.771+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:22:36.773+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:22:36.773+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:22:36.773+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:22:36.773+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:22:36.773+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 93.133µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:36.775+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:22:36.775+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:22:36.775+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:22:36.775+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:22:36.844+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:22:36.844+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/88b52e29e23a9bc9a1f18d56ee3fbbd4 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:88b52e29e23a9bc9a1f18d56ee3fbbd4]}]} 2023-10-21T04:22:36.846+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:22:36.846+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[bada28845870208460ab46b431032b01 f22df9bc00cfefa0906def8153acae8a] IsBalanced:true Messages:[]} 2023-10-21T04:22:36.848+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/88b52e29e23a9bc9a1f18d56ee3fbbd4, rev: [] 2023-10-21T04:22:36.848+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/88b52e29e23a9bc9a1f18d56ee3fbbd4, rev: [] 2023-10-21T04:22:36.848+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:22:36.848+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:22:36.848+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:22:36.849+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:22:36.849+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 27.309µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:22:36.850+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:22:36.850+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:22:36.850+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/88b52e29e23a9bc9a1f18d56ee3fbbd4, rev: [] call. err: 2023-10-21T04:22:36.850+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[bada28845870208460ab46b431032b01 f22df9bc00cfefa0906def8153acae8a] IsBalanced:true Messages:[]} 2023-10-21T04:22:36.850+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:22:36.862+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:22:36.862+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:22:36.864+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:22:36.864+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[bada28845870208460ab46b431032b01 f22df9bc00cfefa0906def8153acae8a] IsBalanced:true Messages:[]} 2023-10-21T04:22:36.892+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:22:36.893+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:22:36.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9998838683224286 2023-10-21T04:22:37.084+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:22:37.300+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57998. Reason = EOF. Cleaning Pipe. 2023-10-21T04:22:37.300+05:30 [Info] messageListener.start(): message channel closed. Remove peer 57:e3:77:c6:cc:fe:6f:46 as follower. 2023-10-21T04:22:37.330+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:37.331+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:22:37.331+05:30 [Info] UnwatchMetadata(): indexer 4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:22:37.331+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:22:37.331+05:30 [Info] UnwatchMetadata(): finish for indexer 4c6e4b64d0c01f9d6431a65ac36fc024 2023-10-21T04:22:37.331+05:30 [Info] UnwatchMetadata(): indexer 458c2b8d760618b2a2f6a998f4c93e74 2023-10-21T04:22:37.331+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:22:37.332+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:46346->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:22:37.332+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:46346. Reason = EOF. Cleaning Pipe. 2023-10-21T04:22:37.332+05:30 [Info] messageListener.start(): message channel closed. Remove peer fd:a6:e0:b8:fc:d2:a4:61 as follower. 2023-10-21T04:22:37.332+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:42732->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:22:37.333+05:30 [Info] UnwatchMetadata(): finish for indexer 458c2b8d760618b2a2f6a998f4c93e74 2023-10-21T04:22:37.353+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58064. Reason = EOF. Cleaning Pipe. 2023-10-21T04:22:37.353+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4c:d8:28:e5:1e:6b:c5:36 as follower. 2023-10-21T04:22:37.356+05:30 [Info] switched currmeta from 998 -> 1000 force true 2023-10-21T04:22:37.356+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:22:37.356+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] ... stopped 2023-10-21T04:22:37.356+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:22:37.356+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] ... stopped 2023-10-21T04:22:37.356+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:41416 closed 2023-10-21T04:22:37.357+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:57954 closed 2023-10-21T04:22:37.399+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:41382 closed 2023-10-21T04:22:37.459+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:46302. Reason = EOF. Cleaning Pipe. 2023-10-21T04:22:37.459+05:30 [Info] messageListener.start(): message channel closed. Remove peer c6:5a:9d:0:dd:d4:1f:d4 as follower. 2023-10-21T04:22:37.522+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:46306. Reason = EOF. Cleaning Pipe. 2023-10-21T04:22:37.522+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2b:8c:e7:c7:10:2c:1b:19 as follower. 2023-10-21T04:22:37.554+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:46308. Reason = EOF. Cleaning Pipe. 2023-10-21T04:22:37.554+05:30 [Info] messageListener.start(): message channel closed. Remove peer 28:17:e4:a8:a6:18:3a:74 as follower. 2023-10-21T04:22:37.572+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34758 closed 2023-10-21T04:22:37.577+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:57976 closed 2023-10-21T04:22:37.624+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:57966 closed 2023-10-21T04:22:37.800+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:22:37.822+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:37.954+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Stopping releaseConnsRoutine 2023-10-21T04:22:37.992+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9992447175228535 2023-10-21T04:22:38.000+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Stopping releaseConnsRoutine 2023-10-21T04:22:38.018+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:22:38.018+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:38.018+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:22:38.045+05:30 [Info] switched currmeta from 1000 -> 1000 force true 2023-10-21T04:22:38.253+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:38.253+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:22:38.255+05:30 [Info] switched currmeta from 1000 -> 1000 force true 2023-10-21T04:22:38.299+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:38.393+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:38.393+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:22:38.397+05:30 [Info] switched currmeta from 1000 -> 1000 force true 2023-10-21T04:22:38.448+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:57988 closed 2023-10-21T04:22:38.456+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:46400. Error = read tcp4 127.0.0.1:9118->127.0.0.1:46400: read: connection reset by peer. Kill Pipe. 2023-10-21T04:22:38.456+05:30 [Info] messageListener.start(): message channel closed. Remove peer 32:2e:c9:ac:80:c4:59:81 as follower. 2023-10-21T04:22:38.555+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:22:38.662+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:38.663+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:22:38.668+05:30 [Info] switched currmeta from 1000 -> 1000 force true 2023-10-21T04:22:38.713+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:38.720+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:38.720+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:22:38.724+05:30 [Info] switched currmeta from 1000 -> 1000 force true 2023-10-21T04:22:38.993+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9989212564724612 2023-10-21T04:22:39.556+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:22:39.564+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:22:39.564+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:22:39.568+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 226] updated cluster rebalancing status to false 2023-10-21T04:22:39.568+05:30 [Info] switched currmeta from 1000 -> 1000 force true 2023-10-21T04:24:57.539+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=2298f6cccd108c9b6151cc01f5a848f9 -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:24:57.540+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:24:57.540+05:30 [Info] Setting ipv6=false 2023-10-21T04:24:57.637+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:24:57.736+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:24:57.766+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:24:57.766+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:24:57.766+05:30 [Info] Setting log level to Info 2023-10-21T04:24:57.766+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:24:57.766+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:24:57.767+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:24:57.767+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:24:57.830+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:24:58 audit: created new audit service 2023-10-21T04:24:58.184+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:24:58.184+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:24:58.184+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:24:58.185+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:24:58.185+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:24:58.185+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:24:58.401+05:30 [Info] security context initialized 2023-10-21T04:24:58.401+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9120:19120 9499:19499 9999:9999] 2023-10-21T04:24:58.401+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:24:58.401+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:24:58.401+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:24:58.401+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:24:58.427+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:24:58.427+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:24:58.427+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:24:58.434+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:24:58.441+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:24:58.441+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:24:58.441+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:24:58.444+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:24:58.447+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:24:58.534+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 53] will report aggregate recorder stats every 5m0s 2023-10-21T04:24:58.536+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:24:58.548+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:24:58.548+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:24:58.549+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:24:58.580+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:24:58.580+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:24:58.581+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:24:58.587+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:24:58.589+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:24:58.599+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 251 us 2023-10-21T04:24:58.600+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:24:58.597+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:24:58.597+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:24:58.597+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:24:58.598+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:24:58.598+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:24:58.610+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:24:58.627+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:24:58.794+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:24:58.794+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:24:58.794+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:24:58.794+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:24:58.794+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:24:58.795+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerId Value 2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:24:58.797+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:24:58.797+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:24:58.797+05:30 [Info] Indexer Id 2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:24:58.804+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:24:58.805+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:24:58.817+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 226] updating cluster rev epoch 0 -> 1 2023-10-21T04:24:58.817+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 226] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:24:58.884+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 143] bucketNames has been updated (was map[], now map[bucket_%2:7007f0ab442ac00e52d057a30cd22644 bucket_1:db19c9050b4ab1232e6bc6654011618f]) 2023-10-21T04:24:58.884+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 144] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:24:58.885+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 145] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:24:58.885+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:24:58.885+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:24:58.886+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:24:58.964+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 143] updated cluster rebalancing status to true 2023-10-21T04:24:58.964+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 229] updating rebalancing status to true 2023-10-21T04:24:58.970+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:24:58.970+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:24:58.970+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:24:58.970+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:24:58.970+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning false RebalanceToken 2023-10-21T04:24:58.970+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:24:58.971+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:24:58.971+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:24:58.972+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:24:58.974+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:24:58.974+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:24:58.978+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:24:58.981+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:24:58.981+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:24:58.994+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap , elapsed: 22.559604ms 2023-10-21T04:24:58.994+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:24:58.994+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:24:58.994+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:24:58.994+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager false 2023-10-21T04:24:58.995+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:24:58.995+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:24:58.995+05:30 [Error] RebalanceServiceManager::getCachedIndexerNodeUUIDs: 127.0.0.1:9120/getCachedIndexerNodeUUIDs returned err: Unable to find given hostport in cbauth database: `127.0.0.1:9120' 2023-10-21T04:24:58.995+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:24:58.996+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:24:59.004+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9. Err projector.topicMissing 2023-10-21T04:24:59.004+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:24:59.004+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:24:59.005+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:24:59.005+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9. Err projector.topicMissing 2023-10-21T04:24:59.005+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:24:59.005+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:24:59.006+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:24:59.006+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:24:59.028+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:24:59.028+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:24:59.050+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:24:59.050+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/stats: no such file or directory 2023-10-21T04:24:59.050+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:24:59.050+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:24:59.050+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:24:59.052+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:24:59.052+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{2298f6cccd108c9b6151cc01f5a848f9 7060000 } 2023-10-21T04:24:59.052+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:24:59.057+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:24:59.058+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:24:59.059+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:24:59.059+05:30 [Info] progress stats collection done. 2023-10-21T04:24:59.059+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:24:59.061+05:30 [Info] send progress stats to clients 2023-10-21T04:24:59.067+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:24:59.067+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:24:59.069+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:24:59.084+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:24:59.089+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:24:59.090+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:24:59.091+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:24:59.092+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:24:59.096+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:24:59.097+05:30 [Info] Setting log level to Info 2023-10-21T04:24:59.099+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:24:59.101+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:24:59.104+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:24:59.109+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:24:59.126+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:24:59.126+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:24:59.126+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:24:59.132+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:24:59.157+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:24:59.157+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:24:59.181+05:30 [Info] security context initialized 2023-10-21T04:24:59.181+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9499:19499 9999:9999] 2023-10-21T04:24:59.186+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:24:59.281+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {1972139801924f1fdf1188af4e7137b9 [] topology-change-rebalance [{{2298f6cccd108c9b6151cc01f5a848f9 7060000 } recovery-full} {{e28a1725ce62fe1e923bede04f38724f 7060000 } recovery-full}] [{bada28845870208460ab46b431032b01 7060000 } {f22df9bc00cfefa0906def8153acae8a 7060000 }]} 2023-10-21T04:24:59.345+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:24:59.362+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40516 2023-10-21T04:24:59.386+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40528 2023-10-21T04:24:59.386+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:24:59.387+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:24:59.387+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:24:59.387+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:24:59.396+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:24:59.396+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:24:59.406+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40532 2023-10-21T04:24:59.412+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40536 2023-10-21T04:24:59.414+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40538 2023-10-21T04:24:59.426+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:24:59.460+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40548 2023-10-21T04:24:59.485+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 111.742529ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.486+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:24:59.486+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:24:59.486+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:24:59.487+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 234.341µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.491+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:24:59.491+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:24:59.510+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:24:59.534+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 136.60161ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.536+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 10.835µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.548+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52146" doAuth() ... 2023-10-21T04:24:59.550+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52146" doReceive() ... 2023-10-21T04:24:59.562+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:24:59.562+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:24:59.583+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 166.390389ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.585+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 13.173µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.609+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 191.388523ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.610+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.318µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.611+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52156" doAuth() ... 2023-10-21T04:24:59.612+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:24:59.615+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52156" doReceive() ... 2023-10-21T04:24:59.617+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 153.157023ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.620+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 9.515µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.623+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40578 2023-10-21T04:24:59.624+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:24:59.630+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 219.633388ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.632+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 9.921µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.656+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52170" doAuth() ... 2023-10-21T04:24:59.656+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52170" doReceive() ... 2023-10-21T04:24:59.664+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 34.774257ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.665+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:24:59.665+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:24:59.666+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 13.416µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:24:59.682+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:24:59.705+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52180" doAuth() ... 2023-10-21T04:24:59.705+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52180" doReceive() ... 2023-10-21T04:24:59.706+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52182" doAuth() ... 2023-10-21T04:24:59.706+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52182" doReceive() ... 2023-10-21T04:24:59.714+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:24:59.719+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:24:59.720+05:30 [Info] initialized currmeta 34 force true 2023-10-21T04:24:59.720+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:24:59.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:24:59.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0] 7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:24:59.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0] 7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:24:59.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0] 7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 {Index inst : [partitions]}: map[3539475206979968695:[0] 12882995014846270330:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:24:59.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0] 7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 {Index inst : [partitions]}: map[2281164589865190206:[0] 3539475206979968695:[0] 12882995014846270330:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:24:59.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0] 3593050551190633737:[2 3 4 5 6 7 8 1] 7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 {Index inst : [partitions]}: map[2281164589865190206:[0] 3539475206979968695:[0] 12882995014846270330:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:24:59.721+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0] 3593050551190633737:[2 3 4 5 6 7 8 1] 7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 {Index inst : [partitions]}: map[620420137039842024:[0] 2281164589865190206:[0] 3539475206979968695:[0] 12882995014846270330:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:24:59.730+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52188" doAuth() ... 2023-10-21T04:24:59.730+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52188" doReceive() ... 2023-10-21T04:24:59.735+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:24:59.735+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:24:59.735+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:24:59.736+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:24:59.744+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:24:59.744+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:24:59.744+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:24:59.746+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:24:59.770+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:24:59.771+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:24:59.771+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:24:59.772+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:24:59.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52196" doAuth() ... 2023-10-21T04:24:59.773+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:52196" doReceive() ... 2023-10-21T04:24:59.779+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:24:59.779+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:24:59.779+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:24:59.780+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:24:59.783+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:24:59.783+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:24:59.783+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:24:59.783+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:24:59.784+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:24:59.784+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:24:59.784+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:24:59.784+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 7.171µs 2023-10-21T04:24:59.784+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:24:59.784+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:24:59.784+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:24:59.786+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:24:59.789+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:24:59.789+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:24:59.789+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 4.90722ms 2023-10-21T04:24:59.789+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:24:59.789+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 506.368744ms in RebalanceServiceManager::prepareRebalance: 2023-10-21T04:24:59.790+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:24:59.790+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 3.922842ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 1 error 2023-10-21T04:24:59.790+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 73.573µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:24:59.790+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:24:59.791+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:24:59.792+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:24:59.792+05:30 [Info] Indexer::ReadMemstats Time Taken 1.683738ms 2023-10-21T04:24:59.793+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:24:59.794+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {1972139801924f1fdf1188af4e7137b9 [] topology-change-rebalance [{{2298f6cccd108c9b6151cc01f5a848f9 7060000 } recovery-full} {{e28a1725ce62fe1e923bede04f38724f 7060000 } recovery-full}] [{bada28845870208460ab46b431032b01 7060000 } {f22df9bc00cfefa0906def8153acae8a 7060000 }]} 2023-10-21T04:24:59.796+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:24:59.818+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"2298f6cccd108c9b6151cc01f5a848f9","RebalId":"1972139801924f1fdf1188af4e7137b9","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:24:59.820+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{2298f6cccd108c9b6151cc01f5a848f9 1972139801924f1fdf1188af4e7137b9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:24:59.822+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{2298f6cccd108c9b6151cc01f5a848f9 1972139801924f1fdf1188af4e7137b9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:24:59.944+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 259] aggregate storage info: map[bucket_%2:map[kv:5843136 index:77536762] bucket_1:map[kv:6629568 index:71637702]] 2023-10-21T04:24:59.986+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:24:59.998+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 0 NodeUUIDs: [] 2023-10-21T04:24:59.998+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 1 NodeUUIDs: [2298f6cccd108c9b6151cc01f5a848f9] 2023-10-21T04:24:59.998+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 1 NodeUUIDs: [2298f6cccd108c9b6151cc01f5a848f9] 2023-10-21T04:24:59.998+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:24:59.998+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:00.000+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:25:00.001+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:25:00.035+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:25:00.065+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 2023-10-21T04:25:00.065+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 {Index inst : [partitions]}: map[15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:25:00.065+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 {Index inst : [partitions]}: map[12004664282277897596:[0] 15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:25:00.065+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 {Index inst : [partitions]}: map[5004576769623010783:[0] 12004664282277897596:[0] 15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:25:00.065+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 4992762551574016313:[0]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 {Index inst : [partitions]}: map[5004576769623010783:[0] 12004664282277897596:[0] 15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:25:00.066+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 4992762551574016313:[0] 11460826864060385936:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 {Index inst : [partitions]}: map[5004576769623010783:[0] 12004664282277897596:[0] 15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:25:00.066+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 4992762551574016313:[0] 10979931881244616620:[0] 11460826864060385936:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 {Index inst : [partitions]}: map[5004576769623010783:[0] 12004664282277897596:[0] 15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:25:00.066+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 4992762551574016313:[0] 10979931881244616620:[0] 11460826864060385936:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842319187166966, Old rollbacktime: 0 {Index inst : [partitions]}: map[5004576769623010783:[0] 9912003609070511414:[2 3 4 5 6 7 8 1] 12004664282277897596:[0] 15667952562388529486:[0]]. New rollbacktime: 1697842319560471331, Old rollbacktime: 0 2023-10-21T04:25:00.084+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:25:00.110+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:25:00.118+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:25:00.132+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:25:00.165+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:25:00.855+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:25:00.865+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:25:00.865+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:25:00.874+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:25:00.874+05:30 [Info] NewShardRebalancer nodeId 2298f6cccd108c9b6151cc01f5a848f9 rebalToken &{2298f6cccd108c9b6151cc01f5a848f9 1972139801924f1fdf1188af4e7137b9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:25:00.874+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:25:00.874+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.080007556s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:25:00.874+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{2298f6cccd108c9b6151cc01f5a848f9 1972139801924f1fdf1188af4e7137b9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:25:00.874+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/1972139801924f1fdf1188af4e7137b9 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:00.874+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:00.875+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:25:00.878+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:25:00.878+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:00.878+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:25:00.878+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/1972139801924f1fdf1188af4e7137b9 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:00.883+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:25:00.883+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:25:00.942+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{bada28845870208460ab46b431032b01 bada28845870208460ab46b431032b01 plasma map[] [{26 bucket_1 _default _default [{bucket_1 _default _default idx_partitioned 573521267164877540 [{3593050551190633737 4 1 [{1 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 5 0 false plasma 0}]} {bucket_1 _default _default idx_partitioned_defer 17656144409930763141 [{14472287716825196064 1 0 [{1 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 5 1 false plasma 0}]} {bucket_1 _default _default #primary 12338036531800130039 [{10590902705185414921 4 1 [{0 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630] []}] 1 0 5 0 false plasma 0}]} {bucket_1 _default _default #primary_defer 11619165464560280276 [{16581200427024765500 1 0 [{0 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630] []}] 1 0 5 1 false plasma 0}]} {bucket_1 _default _default idx_secondary 1252359852432998131 [{7609950505007110692 4 1 [{0 0 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default _default idx_secondary_defer 8378417662949994532 [{2281269425900993179 1 0 [{0 0 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 1 false plasma 0}]}]} {38 bucket_%2 _default c1 []} {7 bucket_1 _system _query [{bucket_1 _system _query #primary 12833670934079048728 [{2428478379951969204 4 1 [{0 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [2864853750198994630] []}] 1 0 5 0 false plasma 0}]}]} {28 bucket_%2 _default _default [{bucket_%2 _default _default idx_partitioned_defer 15431181238515168303 [{15633219898206385003 1 0 [{1 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 5 1 false plasma 0}]} {bucket_%2 _default _default idx_partitioned 16162012140118277065 [{12882995014846270330 4 1 [{1 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 5 0 false plasma 0}]} {bucket_%2 _default _default #primary 8857269605176042330 [{620420137039842024 4 1 [{0 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479] []}] 1 0 5 0 false plasma 0}]} {bucket_%2 _default _default #primary_defer 13856704970496601958 [{1757261333586968909 1 0 [{0 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479] []}] 1 0 5 0 false plasma 0}]} {bucket_%2 _default _default idx_secondary 7525085113233084253 [{2281164589865190206 4 1 [{0 0 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default _default idx_secondary_defer 18026660730857541767 [{14296137856158897341 1 0 [{0 0 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 0 false plasma 0}]}]} {40 bucket_1 _default c1 []} {40 bucket_%2 _default c2% []} {40 bucket_1 _default c2% []} {7 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3674078063353481907 [{3539475206979968695 4 1 [{0 5 {[{0 0 bada28845870208460ab46b431032b01}]} {[] []} [10890719122371003479] []}] 1 0 5 0 false plasma 0}]}]}] [DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {f22df9bc00cfefa0906def8153acae8a f22df9bc00cfefa0906def8153acae8a plasma map[] [{7 bucket_1 _system _query [{bucket_1 _system _query #primary 12833670934079048728 [{5004576769623010783 4 1 [{0 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630] []}] 1 0 5 1 false plasma 0}]}]} {26 bucket_1 _default _default [{bucket_1 _default _default idx_partitioned_defer 17656144409930763141 [{7315818257769233877 1 0 [{1 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 5 0 false plasma 0}]} {bucket_1 _default _default idx_partitioned 573521267164877540 [{9912003609070511414 4 1 [{1 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {2 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {3 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {4 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {5 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {6 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {7 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []} {8 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []}] 8 0 5 1 false plasma 0}]} {bucket_1 _default _default #primary 12338036531800130039 [{15667952562388529486 4 1 [{0 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630] []}] 1 0 5 1 false plasma 0}]} {bucket_1 _default _default #primary_defer 11619165464560280276 [{14271070195512901425 1 0 [{0 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630] []}] 1 0 5 0 false plasma 0}]} {bucket_1 _default _default idx_secondary 1252359852432998131 [{12004664282277897596 4 1 [{0 0 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default _default idx_secondary_defer 8378417662949994532 [{17394655409336449770 1 0 [{0 0 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [2864853750198994630 12177293282243154215] []}] 1 0 0 0 false plasma 0}]}]} {26 bucket_%2 _default _default [{bucket_%2 _default _default idx_partitioned 16162012140118277065 [{11460826864060385936 4 1 [{1 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 5 1 false plasma 0}]} {bucket_%2 _default _default idx_partitioned_defer 15431181238515168303 [{13987057765505463258 1 0 [{1 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {2 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {3 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {4 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {5 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {6 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {7 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []} {8 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []}] 8 0 5 0 false plasma 0}]} {bucket_%2 _default _default #primary 8857269605176042330 [{2296526085582506469 4 1 [{0 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479] []}] 1 0 5 1 false plasma 0}]} {bucket_%2 _default _default #primary_defer 13856704970496601958 [{16733302726553098964 1 0 [{0 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479] []}] 1 0 5 1 false plasma 0}]} {bucket_%2 _default _default idx_secondary 7525085113233084253 [{10979931881244616620 4 1 [{0 0 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default _default idx_secondary_defer 18026660730857541767 [{1094343481594206610 1 0 [{0 0 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479 7659339677706884510] []}] 1 0 0 1 false plasma 0}]}]} {40 bucket_1 _default c2% []} {7 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3674078063353481907 [{4992762551574016313 4 1 [{0 5 {[{0 0 f22df9bc00cfefa0906def8153acae8a}]} {[] []} [10890719122371003479] []}] 1 0 5 1 false plasma 0}]}]} {40 bucket_%2 _default c2% []} {40 bucket_1 _default c1 []} {40 bucket_%2 _default c1 []}] [DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {2298f6cccd108c9b6151cc01f5a848f9 2298f6cccd108c9b6151cc01f5a848f9 plasma map[] [] [] 0 0 0 false} {e28a1725ce62fe1e923bede04f38724f e28a1725ce62fe1e923bede04f38724f plasma map[] [] [] 0 0 0 false}] map[]} 2023-10-21T04:25:01.034+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53726 2023-10-21T04:25:01.041+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:25:01.041+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:25:01.041+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:25:01.041+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:25:01.042+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53726 2023-10-21T04:25:01.063+05:30 [Info] Planner::processDeleteToken: processing 20 tokens for node 127.0.0.1:9002 2023-10-21T04:25:01.063+05:30 [Info] Planner::processDeleteToken: processing 20 tokens for node 127.0.0.1:9004 2023-10-21T04:25:01.063+05:30 [Info] Planner::processDeleteToken: processing 20 tokens for node 127.0.0.1:9001 2023-10-21T04:25:01.063+05:30 [Info] Planner::processDeleteToken: processing 20 tokens for node 127.0.0.1:9003 2023-10-21T04:25:01.066+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53726 2023-10-21T04:25:01.085+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:25:01.085+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:25:01.085+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:25:01.086+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:25:01.148+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:25:01.148+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:25:01.148+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:25:01.148+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:25:01.148+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:25:01.148+05:30 [Info] Moving index 1252359852432998131:7609950505007110692:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.148+05:30 [Info] Moving index 18026660730857541767:14296137856158897341:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.148+05:30 [Info] Moving index 12338036531800130039:10590902705185414921:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 15431181238515168303:15633219898206385003:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 13856704970496601958:1757261333586968909:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 3674078063353481907:3539475206979968695:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 8378417662949994532:2281269425900993179:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:14472287716825196064:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:12882995014846270330:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 8857269605176042330:620420137039842024:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 7525085113233084253:2281164589865190206:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 12833670934079048728:2428478379951969204:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 573521267164877540:3593050551190633737:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 11619165464560280276:16581200427024765500:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:25:01.149+05:30 [Info] Moving index 12338036531800130039:15667952562388529486:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 18026660730857541767:1094343481594206610:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 8857269605176042330:2296526085582506469:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 17656144409930763141:7315818257769233877:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.149+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 16162012140118277065:11460826864060385936:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 13856704970496601958:16733302726553098964:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 11619165464560280276:14271070195512901425:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 1252359852432998131:12004664282277897596:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 8378417662949994532:17394655409336449770:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 7525085113233084253:10979931881244616620:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 15431181238515168303:13987057765505463258:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 12833670934079048728:5004576769623010783:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 573521267164877540:9912003609070511414:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Moving index 3674078063353481907:4992762551574016313:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:25:01.150+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17603037 Units 0 2023-10-21T04:25:01.150+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 17602433 Units 0 2023-10-21T04:25:01.150+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 613085 Units 0 Tenants 0 2023-10-21T04:25:01.150+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 613739 Units 0 Tenants 0 2023-10-21T04:25:01.150+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:25:01.150+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:25:01.150+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:25:01.150+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:25:01.150+05:30 [Info] ---------------------------------------- 2023-10-21T04:25:01.150+05:30 [Info] 2023-10-21T04:25:01.150+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:e28a1725ce62fe1e923bede04f38724f, useLiveData:false 2023-10-21T04:25:01.150+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:42 shardCompatVersion: 1 2023-10-21T04:25:01.150+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:25:01.150+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.150+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:_default, defnId:1252359852432998131, instId:7609950505007110692, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.150+05:30 [Info] Index total memory:157660290 (150.357M), mem:112000 (109.375K), overhead:157548290 (150.25M), min:157660290 (150.357M), data:112000 (109.375K) cpu:0.0002 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.150+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.150+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.150+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:18026660730857541767, instId:14296137856158897341, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.150+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.150+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.150+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.150+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_default, collection:_default, defnId:12338036531800130039, instId:10590902705185414921, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:25:01.150+05:30 [Info] Index total memory:157315275 (150.028M), mem:25000 (24.4141K), overhead:157290275 (150.004M), min:157315275 (150.028M), data:25000 (24.4141K) cpu:0.0432 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.150+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.150+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.150+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.150+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.150+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.150+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.150+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.150+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:15633219898206385003, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:13856704970496601958, instId:1757261333586968909, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3674078063353481907, instId:3539475206979968695, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:157297515 (150.011M), mem:9624 (9.39844K), overhead:157287891 (150.001M), min:157297515 (150.011M), data:9624 (9.39844K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:8378417662949994532, instId:2281269425900993179, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.151+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.151+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.151+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.151+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:14472287716825196064, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157324989 (150.037M), mem:33411 (32.6279K), overhead:157291578 (150.005M), min:157324989 (150.037M), data:33411 (32.6279K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157324642 (150.036M), mem:33110 (32.334K), overhead:157291532 (150.005M), min:157324642 (150.036M), data:33110 (32.334K) cpu:0.0036 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157324840 (150.037M), mem:33282 (32.502K), overhead:157291558 (150.005M), min:157324840 (150.037M), data:33282 (32.502K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157326339 (150.038M), mem:34580 (33.7695K), overhead:157291759 (150.005M), min:157326339 (150.038M), data:34580 (33.7695K) cpu:0.0046 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157325254 (150.037M), mem:33640 (32.8516K), overhead:157291614 (150.005M), min:157325254 (150.037M), data:33640 (32.8516K) cpu:0.0038 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157324684 (150.037M), mem:33147 (32.3701K), overhead:157291537 (150.005M), min:157324684 (150.037M), data:33147 (32.3701K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157324515 (150.036M), mem:33000 (32.2266K), overhead:157291515 (150.005M), min:157324515 (150.036M), data:33000 (32.2266K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.152+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.152+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.152+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:12882995014846270330, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.152+05:30 [Info] Index total memory:157326215 (150.038M), mem:34472 (33.6641K), overhead:157291743 (150.005M), min:157326215 (150.038M), data:34472 (33.6641K) cpu:0.0040 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:8857269605176042330, instId:620420137039842024, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157315275 (150.028M), mem:25000 (24.4141K), overhead:157290275 (150.004M), min:157315275 (150.028M), data:25000 (24.4141K) cpu:0.0418 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:7525085113233084253, instId:2281164589865190206, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157668210 (150.364M), mem:112000 (109.375K), overhead:157556210 (150.257M), min:157668210 (150.364M), data:112000 (109.375K) cpu:0.0846 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:12833670934079048728, instId:2428478379951969204, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157297488 (150.011M), mem:9600 (9.375K), overhead:157287888 (150.001M), min:157297488 (150.011M), data:9600 (9.375K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157325711 (150.037M), mem:34036 (33.2383K), overhead:157291675 (150.005M), min:157325711 (150.037M), data:34036 (33.2383K) cpu:0.0028 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157325162 (150.037M), mem:33561 (32.7744K), overhead:157291601 (150.005M), min:157325162 (150.037M), data:33561 (32.7744K) cpu:0.0024 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157325429 (150.037M), mem:33792 (33K), overhead:157291637 (150.005M), min:157325429 (150.037M), data:33792 (33K) cpu:0.0028 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157326931 (150.039M), mem:35092 (34.2695K), overhead:157291839 (150.005M), min:157326931 (150.039M), data:35092 (34.2695K) cpu:0.0026 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157324861 (150.037M), mem:33300 (32.5195K), overhead:157291561 (150.005M), min:157324861 (150.037M), data:33300 (32.5195K) cpu:0.0024 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157326339 (150.038M), mem:34580 (33.7695K), overhead:157291759 (150.005M), min:157326339 (150.038M), data:34580 (33.7695K) cpu:0.0030 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157324948 (150.037M), mem:33375 (32.5928K), overhead:157291573 (150.005M), min:157324948 (150.037M), data:33375 (32.5928K) cpu:0.0026 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:3593050551190633737, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:157326509 (150.038M), mem:34727 (33.9131K), overhead:157291782 (150.005M), min:157326509 (150.038M), data:34727 (33.9131K) cpu:0.0026 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.153+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:11619165464560280276, instId:16581200427024765500, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:25:01.153+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.153+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.153+05:30 [Info] 2023-10-21T04:25:01.153+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:2298f6cccd108c9b6151cc01f5a848f9, useLiveData:false 2023-10-21T04:25:01.153+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:42 shardCompatVersion: 1 2023-10-21T04:25:01.154+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:12338036531800130039, instId:15667952562388529486, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:157315275 (150.028M), mem:25000 (24.4141K), overhead:157290275 (150.004M), min:157315275 (150.028M), data:25000 (24.4141K) cpu:0.0396 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_secondary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:18026660730857541767, instId:1094343481594206610, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:8857269605176042330, instId:2296526085582506469, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:157315275 (150.028M), mem:25000 (24.4141K), overhead:157290275 (150.004M), min:157315275 (150.028M), data:25000 (24.4141K) cpu:0.0404 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.154+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.154+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.154+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:_default, defnId:17656144409930763141, instId:7315818257769233877, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.154+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.155+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.155+05:30 [Info] Index total memory:157324989 (150.037M), mem:33411 (32.6279K), overhead:157291578 (150.005M), min:157324989 (150.037M), data:33411 (32.6279K) cpu:0.0124 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.155+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.155+05:30 [Info] Index total memory:157324642 (150.036M), mem:33110 (32.334K), overhead:157291532 (150.005M), min:157324642 (150.036M), data:33110 (32.334K) cpu:0.0106 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.155+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.155+05:30 [Info] Index total memory:157324840 (150.037M), mem:33282 (32.502K), overhead:157291558 (150.005M), min:157324840 (150.037M), data:33282 (32.502K) cpu:0.0124 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.155+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.155+05:30 [Info] Index total memory:157326339 (150.038M), mem:34580 (33.7695K), overhead:157291759 (150.005M), min:157326339 (150.038M), data:34580 (33.7695K) cpu:0.0134 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.155+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.155+05:30 [Info] Index total memory:157325254 (150.037M), mem:33640 (32.8516K), overhead:157291614 (150.005M), min:157325254 (150.037M), data:33640 (32.8516K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.155+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.155+05:30 [Info] Index total memory:157324684 (150.037M), mem:33147 (32.3701K), overhead:157291537 (150.005M), min:157324684 (150.037M), data:33147 (32.3701K) cpu:0.0122 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.155+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.155+05:30 [Info] Index total memory:157324515 (150.036M), mem:33000 (32.2266K), overhead:157291515 (150.005M), min:157324515 (150.036M), data:33000 (32.2266K) cpu:0.0120 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.155+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.155+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:16162012140118277065, instId:11460826864060385936, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:157326215 (150.038M), mem:34472 (33.6641K), overhead:157291743 (150.005M), min:157326215 (150.038M), data:34472 (33.6641K) cpu:0.0120 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:#primary_defer 0 (replica 1), bucket:bucket_%2, scope:_default, collection:_default, defnId:13856704970496601958, instId:16733302726553098964, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:#primary_defer 0, bucket:bucket_1, scope:_default, collection:_default, defnId:11619165464560280276, instId:14271070195512901425, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:1252359852432998131, instId:12004664282277897596, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:157651530 (150.348M), mem:111000 (108.398K), overhead:157540530 (150.242M), min:157651530 (150.348M), data:111000 (108.398K) cpu:0.0001 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_secondary_defer 0, bucket:bucket_1, scope:_default, collection:_default, defnId:8378417662949994532, instId:17394655409336449770, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:_default, defnId:7525085113233084253, instId:10979931881244616620, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:157671030 (150.367M), mem:111000 (108.398K), overhead:157560030 (150.261M), min:157671030 (150.367M), data:111000 (108.398K) cpu:0.0807 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.156+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.156+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.156+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.156+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.157+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.157+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.157+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.157+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.157+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.157+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:_default, defnId:15431181238515168303, instId:13987057765505463258, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479 7659339677706884510], shardIdsInDefn: map[] 2023-10-21T04:25:01.157+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.157+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.157+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.157+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:12833670934079048728, instId:5004576769623010783, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630], shardIdsInDefn: map[] 2023-10-21T04:25:01.158+05:30 [Info] Index total memory:157297488 (150.011M), mem:9600 (9.375K), overhead:157287888 (150.001M), min:157297488 (150.011M), data:9600 (9.375K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.158+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.158+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.158+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.158+05:30 [Info] Index total memory:157325711 (150.037M), mem:34036 (33.2383K), overhead:157291675 (150.005M), min:157325711 (150.037M), data:34036 (33.2383K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.158+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.158+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.158+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.158+05:30 [Info] Index total memory:157325162 (150.037M), mem:33561 (32.7744K), overhead:157291601 (150.005M), min:157325162 (150.037M), data:33561 (32.7744K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.158+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.158+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.159+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.159+05:30 [Info] Index total memory:157325429 (150.037M), mem:33792 (33K), overhead:157291637 (150.005M), min:157325429 (150.037M), data:33792 (33K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.159+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.159+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.159+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.159+05:30 [Info] Index total memory:157326931 (150.039M), mem:35092 (34.2695K), overhead:157291839 (150.005M), min:157326931 (150.039M), data:35092 (34.2695K) cpu:0.0048 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.159+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.159+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.159+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.159+05:30 [Info] Index total memory:157324861 (150.037M), mem:33300 (32.5195K), overhead:157291561 (150.005M), min:157324861 (150.037M), data:33300 (32.5195K) cpu:0.0042 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.159+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.159+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.159+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.159+05:30 [Info] Index total memory:157326339 (150.038M), mem:34580 (33.7695K), overhead:157291759 (150.005M), min:157326339 (150.038M), data:34580 (33.7695K) cpu:0.0054 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.159+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.159+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.159+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.159+05:30 [Info] Index total memory:157324948 (150.037M), mem:33375 (32.5928K), overhead:157291573 (150.005M), min:157324948 (150.037M), data:33375 (32.5928K) cpu:0.0048 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.159+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.159+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.159+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:_default, defnId:573521267164877540, instId:9912003609070511414, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [2864853750198994630 12177293282243154215], shardIdsInDefn: map[] 2023-10-21T04:25:01.159+05:30 [Info] Index total memory:157326509 (150.038M), mem:34727 (33.9131K), overhead:157291782 (150.005M), min:157326509 (150.038M), data:34727 (33.9131K) cpu:0.0046 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.159+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.159+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:25:01.159+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3674078063353481907, instId:4992762551574016313, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10890719122371003479], shardIdsInDefn: map[] 2023-10-21T04:25:01.159+05:30 [Info] Index total memory:157297515 (150.011M), mem:9624 (9.39844K), overhead:157287891 (150.001M), min:157297515 (150.011M), data:9624 (9.39844K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:25:01.159+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:25:01.159+05:30 [Info] **************************************** 2023-10-21T04:25:01.160+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 613085 Units 0 Tenants 0 2023-10-21T04:25:01.160+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 613739 Units 0 Tenants 0 2023-10-21T04:25:01.160+05:30 [Info] Generating Shard Transfer Token (ShardTokened:42:e0:2a:27:fe:e0:80) for rebalance ( MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: Region: InstId: 7609950505007110692 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 7609950505007110692 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2281269425900993179 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2281269425900993179 Defn: DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17516054303786468982 RealInstId: 14472287716825196064 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 14472287716825196064 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: InstId: 6637393687348361258 RealInstId: 3593050551190633737 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 3593050551190633737 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 10590902705185414921 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 10590902705185414921 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 2428478379951969204 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 2428478379951969204 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 16581200427024765500 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 16581200427024765500 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:25:01.162+05:30 [Info] Generating Shard Transfer Token (ShardToken36:37:b8:9d:2e:c3:90:4a) for rebalance ( MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: Region: InstId: 14296137856158897341 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14296137856158897341 Defn: DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 9963702112820124680 RealInstId: 15633219898206385003 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 15633219898206385003 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: InstId: 791854418175666346 RealInstId: 12882995014846270330 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 12882995014846270330 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 2281164589865190206 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2281164589865190206 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 1757261333586968909 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 1757261333586968909 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 3539475206979968695 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 3539475206979968695 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 620420137039842024 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 620420137039842024 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:25:01.163+05:30 [Info] Generating Shard Transfer Token (ShardToken34:c1:80:2c:eb:79:82:c5) for rebalance ( MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: Region: InstId: 1094343481594206610 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 1094343481594206610 Defn: DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 13895180128994009539 RealInstId: 11460826864060385936 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: InstId: 10979931881244616620 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12820935061706293349 RealInstId: 13987057765505463258 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 2296526085582506469 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: InstId: 16733302726553098964 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: InstId: 4992762551574016313 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:25:01.164+05:30 [Info] Generating Shard Transfer Token (ShardToken12:9b:ba:a8:e5:6a:47:bd) for rebalance ( MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: Region: InstId: 9885812108833241320 RealInstId: 7315818257769233877 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 12004664282277897596 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17394655409336449770 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17394655409336449770 Defn: DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 6611915605599465689 RealInstId: 9912003609070511414 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] Inst: InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: InstId: 15667952562388529486 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: InstId: 14271070195512901425 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 0 PartitionContainer: InstId: 5004576769623010783 RealInstId: 0 Partitions: [0] Versions: [6] Inst: InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 5 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:25:01.165+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:25:01.165+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:25:01.168+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokened:42:e0:2a:27:fe:e0:80 2023-10-21T04:25:01.210+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken36:37:b8:9d:2e:c3:90:4a 2023-10-21T04:25:01.239+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken34:c1:80:2c:eb:79:82:c5 2023-10-21T04:25:01.265+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken12:9b:ba:a8:e5:6a:47:bd 2023-10-21T04:25:01.265+05:30 [Info] ShardRebalancer::observeRebalance &{2298f6cccd108c9b6151cc01f5a848f9 1972139801924f1fdf1188af4e7137b9 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:25:01.317+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.317+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"2298f6cccd108c9b6151cc01f5a848f9","RebalId":"1972139801924f1fdf1188af4e7137b9","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:25:01.318+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:25:01.320+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.323+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.325+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.326+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.426+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.535+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.536+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:25:01.536+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:25:01.536+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:25:01.536+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 62.662µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:01.554+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.554+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:25:01.554+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:25:01.554+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:25:01.555+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 38.764µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:01.603+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.604+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.605+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.605+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:25:01.629+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{2298f6cccd108c9b6151cc01f5a848f9 1972139801924f1fdf1188af4e7137b9 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:25:01.629+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 2 2023-10-21T04:25:01.681+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[bada28845870208460ab46b431032b01:ShardTokened:42:e0:2a:27:fe:e0:80 f22df9bc00cfefa0906def8153acae8a:ShardToken12:9b:ba:a8:e5:6a:47:bd], currActiveTransfers: map[bada28845870208460ab46b431032b01:2 f22df9bc00cfefa0906def8153acae8a:2] 2023-10-21T04:25:01.681+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"2298f6cccd108c9b6151cc01f5a848f9","RebalId":"1972139801924f1fdf1188af4e7137b9","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:25:01.681+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:25:01.681+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:25:01.681+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:25:01.681+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:25:01.681+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:25:01.682+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.683+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.684+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:01.685+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 17.312µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:01.687+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:02.634+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:02.634+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:1 bada28845870208460ab46b431032b01:2 f22df9bc00cfefa0906def8153acae8a:1], batchSize: 2 2023-10-21T04:25:02.634+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:25:02.634+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:25:02.634+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:25:02.634+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 23.733µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:02.634+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2864853750198994630:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/2864853750198994630/shards/shard2864853750198994630 12177293282243154215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/12177293282243154215/shards/shard12177293282243154215] 1972139801924f1fdf1188af4e7137b9 ShardToken12:9b:ba:a8:e5:6a:47:bd /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0062c2ae0 0xc0062c2b40 map[] 0xc006ea9e60 0xc006ee3200 false} 2023-10-21T04:25:02.635+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:25:02.635+05:30 [Info] Directory /opt/build/ns_server/data/n_3/data/@2i/shards does not exist. Skip shard recovery. 2023-10-21T04:25:02.635+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/2864853750198994630/shards/shard2864853750198994630 2023-10-21T04:25:02.636+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/12177293282243154215/shards/shard12177293282243154215 2023-10-21T04:25:02.639+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:02.639+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:1 bada28845870208460ab46b431032b01:1 e28a1725ce62fe1e923bede04f38724f:1 f22df9bc00cfefa0906def8153acae8a:1], batchSize: 2 2023-10-21T04:25:02.659+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/2864853750198994630/shards/shard2864853750198994630/data: super block snapshot head offset :0 tail offset:10031104 sbgen :1640 2023-10-21T04:25:02.659+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/2864853750198994630/shards/shard2864853750198994630/data/recovery: super block snapshot head offset :0 tail offset:9707520 sbgen :1640 2023-10-21T04:25:02.661+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/12177293282243154215/shards/shard12177293282243154215/data: super block snapshot head offset :0 tail offset:9138176 sbgen :1446 2023-10-21T04:25:02.661+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/12177293282243154215/shards/shard12177293282243154215/data/recovery: super block snapshot head offset :0 tail offset:8777728 sbgen :1446 2023-10-21T04:25:02.680+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/12177293282243154215/shards/shard12177293282243154215: metadata saved successfully 2023-10-21T04:25:02.681+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 (12177293282243154215) added 2023-10-21T04:25:02.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 from metadata completed, tenant :bucket_1_db19c9050b4ab1232e6bc6654011618f 2023-10-21T04:25:02.681+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 389894196 totalBytes: 17935133 numShards: 2 duration: 46.613586ms 2023-10-21T04:25:02.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/12177293282243154215/shards/shard12177293282243154215, newshardId 12177293282243154215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215, shardsize 17935133, duration 45.887716ms 2023-10-21T04:25:02.681+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 12177293282243154215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215, err: , elapsed(sec): 0.047002871 2023-10-21T04:25:02.682+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 12177293282243154215 bytesWritten: 17935133, totalBytes: 17935133, transferRate: 3.985585111111111e+08 2023-10-21T04:25:02.697+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/2864853750198994630/shards/shard2864853750198994630: metadata saved successfully 2023-10-21T04:25:02.708+05:30 [Info] tenant 'bucket_1_db19c9050b4ab1232e6bc6654011618f': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 (2864853750198994630) added 2023-10-21T04:25:02.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 from metadata completed, tenant :bucket_1_db19c9050b4ab1232e6bc6654011618f 2023-10-21T04:25:02.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken12_9b_ba_a8_e5_6a_47_bd/2864853750198994630/shards/shard2864853750198994630, newshardId 2864853750198994630, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630, shardsize 19758278, duration 72.780249ms 2023-10-21T04:25:02.708+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2864853750198994630, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630, err: , elapsed(sec): 0.073319486 2023-10-21T04:25:02.708+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 1972139801924f1fdf1188af4e7137b9, transferId: ShardToken12:9b:ba:a8:e5:6a:47:bd 2023-10-21T04:25:02.708+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 1972139801924f1fdf1188af4e7137b9, transferId: ShardToken12:9b:ba:a8:e5:6a:47:bd, elapsed(sec): 0.000115391 2023-10-21T04:25:02.708+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2864853750198994630: 12177293282243154215:], shardPaths: map[2864853750198994630:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 12177293282243154215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.073514121 2023-10-21T04:25:02.708+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2864853750198994630 bytesWritten: 19758278, totalBytes: 19758278, transferRate: 2.744205277777778e+08 2023-10-21T04:25:02.708+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [2864853750198994630 12177293282243154215], ttid: ShardToken12:9b:ba:a8:e5:6a:47:bd, elapsed(sec): 0.073943189 2023-10-21T04:25:02.716+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:02.753+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:2 bada28845870208460ab46b431032b01:1 e28a1725ce62fe1e923bede04f38724f:1 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:02.753+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:25:02.753+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:25:02.753+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:25:02.753+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.735µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:02.753+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[7659339677706884510:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/7659339677706884510/shards/shard7659339677706884510 10890719122371003479:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/10890719122371003479/shards/shard10890719122371003479] 1972139801924f1fdf1188af4e7137b9 ShardToken34:c1:80:2c:eb:79:82:c5 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0062c2ae0 0xc0062c2b40 map[] 0xc0070e27e0 0xc006ee3b60 false} 2023-10-21T04:25:02.754+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/7659339677706884510/shards/shard7659339677706884510 2023-10-21T04:25:02.755+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/10890719122371003479/shards/shard10890719122371003479 2023-10-21T04:25:02.759+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:02.759+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:1 bada28845870208460ab46b431032b01:1 e28a1725ce62fe1e923bede04f38724f:1 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:02.759+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken12:9b:ba:a8:e5:6a:47:bd 2023-10-21T04:25:02.759+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2864853750198994630:true 12177293282243154215:true] 2023-10-21T04:25:02.759+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.927µs 2023-10-21T04:25:02.759+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [2864853750198994630 12177293282243154215], isLockedForRecovery: true 2023-10-21T04:25:02.759+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [2864853750198994630 12177293282243154215], errMap: map[2864853750198994630: 12177293282243154215:], elapsed: 8.155µs 2023-10-21T04:25:02.762+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 17656144409930763141, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:02.762+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 17656144409930763141, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:02.778+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/10890719122371003479/shards/shard10890719122371003479/data: super block snapshot head offset :0 tail offset:10674176 sbgen :1716 2023-10-21T04:25:02.778+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/10890719122371003479/shards/shard10890719122371003479/data/recovery: super block snapshot head offset :0 tail offset:10334208 sbgen :1716 2023-10-21T04:25:02.779+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/7659339677706884510/shards/shard7659339677706884510/data: super block snapshot head offset :0 tail offset:10063872 sbgen :1506 2023-10-21T04:25:02.780+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/7659339677706884510/shards/shard7659339677706884510/data/recovery: super block snapshot head offset :0 tail offset:9687040 sbgen :1506 2023-10-21T04:25:02.814+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:02.814+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:1 bada28845870208460ab46b431032b01:1 e28a1725ce62fe1e923bede04f38724f:0 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:02.827+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/7659339677706884510/shards/shard7659339677706884510: metadata saved successfully 2023-10-21T04:25:02.827+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 (7659339677706884510) added 2023-10-21T04:25:02.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 from metadata completed, tenant :bucket_%2_7007f0ab442ac00e52d057a30cd22644 2023-10-21T04:25:02.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/7659339677706884510/shards/shard7659339677706884510, newshardId 7659339677706884510, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510, shardsize 19770207, duration 73.635817ms 2023-10-21T04:25:02.828+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 7659339677706884510, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510, err: , elapsed(sec): 0.07398116 2023-10-21T04:25:02.828+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 7659339677706884510 bytesWritten: 19770207, totalBytes: 19770207, transferRate: 2.708247534246575e+08 2023-10-21T04:25:02.828+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/10890719122371003479/shards/shard10890719122371003479: metadata saved successfully 2023-10-21T04:25:02.830+05:30 [Info] tenant 'bucket_%2_7007f0ab442ac00e52d057a30cd22644': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 (10890719122371003479) added 2023-10-21T04:25:02.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 from metadata completed, tenant :bucket_%2_7007f0ab442ac00e52d057a30cd22644 2023-10-21T04:25:02.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1972139801924f1fdf1188af4e7137b9_ShardToken34_c1_80_2c_eb_79_82_c5/10890719122371003479/shards/shard10890719122371003479, newshardId 10890719122371003479, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479, shardsize 21028110, duration 75.364574ms 2023-10-21T04:25:02.830+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10890719122371003479, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479, err: , elapsed(sec): 0.076804245 2023-10-21T04:25:02.831+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 1972139801924f1fdf1188af4e7137b9, transferId: ShardToken34:c1:80:2c:eb:79:82:c5 2023-10-21T04:25:02.831+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 1972139801924f1fdf1188af4e7137b9, transferId: ShardToken34:c1:80:2c:eb:79:82:c5, elapsed(sec): 0.000296737 2023-10-21T04:25:02.831+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[7659339677706884510: 10890719122371003479:], shardPaths: map[7659339677706884510:/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 10890719122371003479:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.077493948 2023-10-21T04:25:02.831+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10890719122371003479 bytesWritten: 21028110, totalBytes: 21028110, transferRate: 2.803748e+08 2023-10-21T04:25:02.831+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [10890719122371003479 7659339677706884510], ttid: ShardToken34:c1:80:2c:eb:79:82:c5, elapsed(sec): 0.078286762 2023-10-21T04:25:02.852+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7315818257769233877,realInstId: 0, indexDefn DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:25:02.854+05:30 [Info] Indexer::handleRecoverIndex InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} 2023-10-21T04:25:02.857+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 3.351678ms 2023-10-21T04:25:02.857+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.547038ms 2023-10-21T04:25:02.857+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:02.858+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:02.858+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7315818257769233877, indexDefn DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:02.858+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 96.368435ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:02.858+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': added 2023-10-21T04:25:02.862+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:25:02.889+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner initialized 2023-10-21T04:25:02.889+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:02.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:02.893+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCleaner initialized for recovery 2023-10-21T04:25:02.893+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:02.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery 2023-10-21T04:25:02.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:02.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data], Shared [true] 2023-10-21T04:25:02.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[8777728] Data log Head [0] Tail [9138176] 2023-10-21T04:25:02.893+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [8777728] 2023-10-21T04:25:02.896+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.896+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:25:02.896+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is disabled 2023-10-21T04:25:02.896+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.896+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:25:02.896+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.896+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:25:02.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.897+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:25:02.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.897+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:25:02.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.897+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:25:02.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.897+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:25:02.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.897+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:25:02.932+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner initialized 2023-10-21T04:25:02.932+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:02.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:02.964+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCleaner initialized for recovery 2023-10-21T04:25:02.968+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:02.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery 2023-10-21T04:25:02.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:02.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data], Shared [true] 2023-10-21T04:25:02.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[9707520] Data log Head [0] Tail [10031104] 2023-10-21T04:25:02.968+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [9707520] 2023-10-21T04:25:02.968+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.971+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:02.977+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.977+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:02.977+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.977+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:02.978+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.978+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:02.978+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.978+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:02.978+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.978+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:02.979+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.979+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:25:02.979+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.979+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:25:02.980+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.980+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.981+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:02.981+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:25:02.995+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.996+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:25:02.996+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.996+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:25:02.996+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.997+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:25:02.997+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.997+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:25:02.997+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.997+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:25:02.998+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.998+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:25:02.998+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:02.998+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:25:02.981+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.004+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.006+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.006+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex 2023-10-21T04:25:03.015+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:03.016+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:0 bada28845870208460ab46b431032b01:1 e28a1725ce62fe1e923bede04f38724f:0 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:03.017+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken34:c1:80:2c:eb:79:82:c5 2023-10-21T04:25:03.018+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[7659339677706884510:true 10890719122371003479:true] 2023-10-21T04:25:03.024+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.652µs 2023-10-21T04:25:03.024+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [10890719122371003479 7659339677706884510], isLockedForRecovery: true 2023-10-21T04:25:03.027+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [10890719122371003479 7659339677706884510], errMap: map[7659339677706884510: 10890719122371003479:], elapsed: 15.537µs 2023-10-21T04:25:03.029+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.030+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:03.057+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : replayOffset: 9090628 (RP:9090628) (CP:8962048) 2023-10-21T04:25:03.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [9090628] took [164.51949ms] 2023-10-21T04:25:03.058+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053364, resident 1, memSz 8591, memSzIndex 0, numRecords 111 2023-10-21T04:25:03.058+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38461814 controller mem 4680 2023-10-21T04:25:03.059+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053364, resident 1, memSz 10953, memSzIndex 77, numRecords 140 2023-10-21T04:25:03.059+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38461814 controller mem 4680 2023-10-21T04:25:03.059+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053364, resident 1, memSz 8808, memSzIndex 0, numRecords 113 2023-10-21T04:25:03.059+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38461814 controller mem 4680 2023-10-21T04:25:03.059+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053288, resident 1, memSz 10069, memSzIndex 77, numRecords 132 2023-10-21T04:25:03.059+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38461814 controller mem 4680 2023-10-21T04:25:03.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.075+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.075+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.075+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.076+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.076+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053288, resident 1, memSz 9253, memSzIndex 92, numRecords 125 2023-10-21T04:25:03.077+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.077+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.078+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.079+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1056168, resident 1, memSz 68058, memSzIndex 874, numRecords 1000 2023-10-21T04:25:03.080+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053288, resident 1, memSz 8918, memSzIndex 0, numRecords 124 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053288, resident 1, memSz 9576, memSzIndex 0, numRecords 121 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1053288, resident 1, memSz 10282, memSzIndex 77, numRecords 134 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.094+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 38462047 controller mem 4680 2023-10-21T04:25:03.094+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [9138176] replayOffset [9090628] 2023-10-21T04:25:03.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [201.746423ms] 2023-10-21T04:25:03.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data], Shared [true]. Built [18] plasmas, took [202.875177ms] 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.093+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 18026660730857541767, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:03.097+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 18026660730857541767, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:03.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 171 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:03.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.110+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swapper started 2023-10-21T04:25:03.110+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:03.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex started 2023-10-21T04:25:03.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.112+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.159+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.160+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.160+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.160+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:03.160+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.161+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:03.161+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.161+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:03.162+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.162+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.163+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.164+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.164+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.164+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.165+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.165+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:03.166+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.167+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex 2023-10-21T04:25:03.172+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.173+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:03.189+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : replayOffset: 9994240 (RP:9994240) (CP:9986048) 2023-10-21T04:25:03.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [9994240] took [239.800159ms] 2023-10-21T04:25:03.209+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057276, resident 1, memSz 12945, memSzIndex 0, numRecords 125 2023-10-21T04:25:03.209+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.209+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057426, resident 1, memSz 11780, memSzIndex 0, numRecords 113 2023-10-21T04:25:03.210+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.211+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.211+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.212+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1055100, resident 1, memSz 62044, memSzIndex 355, numRecords 1000 2023-10-21T04:25:03.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.214+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.214+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057426, resident 1, memSz 14456, memSzIndex 0, numRecords 140 2023-10-21T04:25:03.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057276, resident 1, memSz 12516, memSzIndex 0, numRecords 121 2023-10-21T04:25:03.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.218+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057276, resident 1, memSz 13603, memSzIndex 0, numRecords 132 2023-10-21T04:25:03.218+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1055776, resident 0.23, memSz 1167, memSzIndex 0, numRecords 38 2023-10-21T04:25:03.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057276, resident 1, memSz 12838, memSzIndex 0, numRecords 124 2023-10-21T04:25:03.239+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.238+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:03.219+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 1094343481594206610,realInstId: 0, indexDefn DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:03.241+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057426, resident 1, memSz 11571, memSzIndex 0, numRecords 111 2023-10-21T04:25:03.242+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:03.243+05:30 [Info] Indexer::handleRecoverIndex InstId: 1094343481594206610 Defn: DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:03.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.251+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1057276, resident 1, memSz 13808, memSzIndex 0, numRecords 134 2023-10-21T04:25:03.255+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.255+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.251+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 8.490102ms 2023-10-21T04:25:03.251+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 8.52613ms 2023-10-21T04:25:03.252+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1094343481594206610 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:03.252+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 1094343481594206610, indexDefn DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:03.256+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 163.333106ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:03.256+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.257+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.257+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.257+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1055776, resident 1, memSz 66002, memSzIndex 474, numRecords 1000 2023-10-21T04:25:03.257+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.257+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.258+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: controller ('bucket_1_db19c9050b4ab1232e6bc6654011618f') quota after header replay 48953376 controller mem 5780 2023-10-21T04:25:03.258+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [10031104] replayOffset [9994240] 2023-10-21T04:25:03.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [290.025695ms] 2023-10-21T04:25:03.258+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.259+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.259+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.259+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.259+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.259+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.260+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.257+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 1094343481594206610 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:03.260+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.260+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.260+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': added 2023-10-21T04:25:03.260+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data], Shared [true]. Built [21] plasmas, took [292.824818ms] 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.261+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is disabled 2023-10-21T04:25:03.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.263+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner initialized 2023-10-21T04:25:03.263+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:03.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:03.265+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCleaner initialized for recovery 2023-10-21T04:25:03.265+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:03.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery 2023-10-21T04:25:03.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:03.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data], Shared [true] 2023-10-21T04:25:03.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[10334208] Data log Head [0] Tail [10674176] 2023-10-21T04:25:03.265+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [10334208] 2023-10-21T04:25:03.267+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.267+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:25:03.268+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.268+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:25:03.268+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.268+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:25:03.268+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.268+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:25:03.269+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.269+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:25:03.269+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.269+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:25:03.269+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.269+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:25:03.269+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.270+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:25:03.270+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.270+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:25:03.270+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.270+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:25:03.273+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner initialized 2023-10-21T04:25:03.273+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:03.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:03.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 190 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.277+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex started 2023-10-21T04:25:03.277+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.277+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.277+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:25:03.278+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:03.279+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.280+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.280+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:03.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.281+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:03.281+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:03.281+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex stopped 2023-10-21T04:25:03.282+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.282+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.282+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.282+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.285+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCleaner initialized for recovery 2023-10-21T04:25:03.285+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:25:03.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery 2023-10-21T04:25:03.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:03.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data], Shared [true] 2023-10-21T04:25:03.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[9687040] Data log Head [0] Tail [10063872] 2023-10-21T04:25:03.285+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [9687040] 2023-10-21T04:25:03.293+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:03.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex closed 2023-10-21T04:25:03.294+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:03.294+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.294+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.294+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:03.294+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:03.294+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:03.299+05:30 [Info] switched currmeta from 34 -> 38 force true 2023-10-21T04:25:03.312+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:03.312+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:03.312+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex stopped 2023-10-21T04:25:03.312+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.312+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.312+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.312+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.319+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.320+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:25:03.320+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.320+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:25:03.320+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.321+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:25:03.321+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.321+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:25:03.321+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.321+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:25:03.321+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.321+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:25:03.322+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.322+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:25:03.322+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.319+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:03.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex closed 2023-10-21T04:25:03.339+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:03.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:25:03.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 171 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:25:03.339+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:25:03.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.343+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex sucessfully destroyed 2023-10-21T04:25:03.343+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:25:03.343+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 190 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:25:03.347+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:03.348+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:03.355+05:30 [Info] switched currmeta from 38 -> 38 force true 2023-10-21T04:25:03.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index destroyed 2023-10-21T04:25:03.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.432+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.432+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.435+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.435+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:03.435+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.435+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:03.435+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.435+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:03.435+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.435+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:03.435+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.436+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:03.436+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.436+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:03.436+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.436+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:03.436+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.436+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:03.437+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.437+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:03.437+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.437+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.437+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:03.437+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex 2023-10-21T04:25:03.437+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.438+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:03.438+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.438+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:03.438+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.439+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:03.439+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.439+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.440+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : replayOffset: 10055680 (RP:9850880) (CP:10055680) 2023-10-21T04:25:03.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [10055680] took [154.418256ms] 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8462, memSzIndex 0, numRecords 110 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9091, memSzIndex 0, numRecords 116 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1056136, resident 1, memSz 67899, memSzIndex 856, numRecords 1000 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9582, memSzIndex 76, numRecords 127 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9253, memSzIndex 76, numRecords 125 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8911, memSzIndex 0, numRecords 124 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10903, memSzIndex 76, numRecords 140 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9787, memSzIndex 92, numRecords 129 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9808, memSzIndex 77, numRecords 129 2023-10-21T04:25:03.440+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 43704690 controller mem 4862 2023-10-21T04:25:03.440+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [10063872] replayOffset [10055680] 2023-10-21T04:25:03.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [155.46824ms] 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.441+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.442+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data], Shared [true]. Built [18] plasmas, took [156.328623ms] 2023-10-21T04:25:03.442+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.442+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.442+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.442+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:03.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:03.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:03.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:03.444+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.444+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex 2023-10-21T04:25:03.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.445+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.446+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : replayOffset: 10665984 (RP:10657792) (CP:10665984) 2023-10-21T04:25:03.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [10665984] took [180.620089ms] 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1055078, resident 1, memSz 61971, memSzIndex 338, numRecords 1000 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12831, memSzIndex 0, numRecords 124 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11446, memSzIndex 0, numRecords 110 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13319, memSzIndex 0, numRecords 129 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057276, resident 1, memSz 14407, memSzIndex 0, numRecords 140 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12051, memSzIndex 0, numRecords 116 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1055974, resident 1, memSz 70503, memSzIndex 386, numRecords 1000 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1055776, resident 1, memSz 2207, memSzIndex 0, numRecords 36 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12961, memSzIndex 0, numRecords 125 2023-10-21T04:25:03.446+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057426, resident 1, memSz 13354, memSzIndex 0, numRecords 129 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13138, memSzIndex 0, numRecords 127 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:25:03.447+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: controller ('bucket_%2_7007f0ab442ac00e52d057a30cd22644') quota after header replay 48953290 controller mem 5778 2023-10-21T04:25:03.447+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [10674176] replayOffset [10665984] 2023-10-21T04:25:03.449+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:03.449+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:0 bada28845870208460ab46b431032b01:0 e28a1725ce62fe1e923bede04f38724f:1 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:03.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.451+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [186.638258ms] 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.452+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.453+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:25:03.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data], Shared [true]. Built [21] plasmas, took [188.272632ms] 2023-10-21T04:25:03.454+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.454+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 192 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:03.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:03.458+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swapper started 2023-10-21T04:25:03.458+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex started 2023-10-21T04:25:03.458+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.458+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 219 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.460+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.460+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:03.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex started 2023-10-21T04:25:03.460+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.460+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 197 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:03.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.467+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:03.467+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:03.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex started 2023-10-21T04:25:03.467+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Warmup took 608.624673ms 2023-10-21T04:25:03.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.467+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 2 WriterThreads 1 2023-10-21T04:25:03.467+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 2 Stopped Billing WUs 2023-10-21T04:25:03.468+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index Index: 7315818257769233877 Partition: 2 2023-10-21T04:25:03.469+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:03.469+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:03.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.471+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 214 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:03.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:03.472+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.472+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex started 2023-10-21T04:25:03.473+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.473+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.473+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1094343481594206610 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:03.473+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1094343481594206610 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:03.473+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.473+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.473+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:03.474+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.493+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:03.494+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:03.494+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex stopped 2023-10-21T04:25:03.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.495+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.496+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:03.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex closed 2023-10-21T04:25:03.505+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:03.505+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.505+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.505+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:03.505+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.505+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:03.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 191 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.508+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex started 2023-10-21T04:25:03.508+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.509+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 172 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:03.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.517+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:03.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex started 2023-10-21T04:25:03.518+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:25:03.518+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:03.519+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.518+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.518+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.519+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.520+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:03.530+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.530+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:03.530+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:03.530+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex stopped 2023-10-21T04:25:03.531+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.531+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.531+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.531+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.526+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:03.532+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:0 bada28845870208460ab46b431032b01:0 e28a1725ce62fe1e923bede04f38724f:0 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:03.555+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:03.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex closed 2023-10-21T04:25:03.556+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:03.556+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.556+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.556+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:03.557+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:03.557+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:03.558+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:03.558+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:03.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex stopped 2023-10-21T04:25:03.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.559+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:03.560+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex stopped 2023-10-21T04:25:03.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.561+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.560+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.561+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.561+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.561+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.577+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.578+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:03.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex closed 2023-10-21T04:25:03.579+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:03.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:03.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex closed 2023-10-21T04:25:03.579+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:03.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:25:03.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 172 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:25:03.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex sucessfully destroyed 2023-10-21T04:25:03.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:25:03.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 191 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:25:03.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index destroyed 2023-10-21T04:25:03.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex ... 2023-10-21T04:25:03.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 192 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex ... 2023-10-21T04:25:03.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex sucessfully destroyed 2023-10-21T04:25:03.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex ... 2023-10-21T04:25:03.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 214 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex ... 2023-10-21T04:25:03.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index destroyed 2023-10-21T04:25:03.601+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:03.601+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:03.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.604+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.604+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.604+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.605+05:30 [Info] switched currmeta from 38 -> 39 force true 2023-10-21T04:25:03.604+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:03.606+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.606+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.609+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.609+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:03.611+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.611+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 219 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:03.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:03.614+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.614+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:03.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex started 2023-10-21T04:25:03.614+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.614+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 198 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:03.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.617+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:03.617+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:03.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex started 2023-10-21T04:25:03.619+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 220 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 197 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.619+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.621+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.621+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:03.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex started 2023-10-21T04:25:03.622+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Warmup took 152.133981ms 2023-10-21T04:25:03.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:03.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:03.622+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.622+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:03.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex started 2023-10-21T04:25:03.622+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 1094343481594206610 PartitionId 0 Warmup took 362.361638ms 2023-10-21T04:25:03.623+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1094343481594206610 partitionId 0 WriterThreads 4 2023-10-21T04:25:03.623+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1094343481594206610 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index Index: 1094343481594206610 Partition: 0 2023-10-21T04:25:03.623+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 1094343481594206610, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:25:03.623+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 1094343481594206610, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:25:03.623+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1094343481594206610 Defn: DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:03.623+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 1094343481594206610 2023-10-21T04:25:03.624+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1094343481594206610 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:25:03.624+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:03.624+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421] 2023-10-21T04:25:03.624+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1094343481594206610 Name: idx_secondary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:25:03.624+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 1094343481594206610 Defn: DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:03.624+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 1094343481594206610 Partitions [0] 2023-10-21T04:25:03.624+05:30 [Info] StorageMgr::openSnapshot IndexInst:1094343481594206610 Partition:0 No Snapshot Found. 2023-10-21T04:25:03.624+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 1094343481594206610 Adding Nil Snapshot. 2023-10-21T04:25:03.622+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.622+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.624+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.624+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.630+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 3 WriterThreads 1 2023-10-21T04:25:03.631+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 3 Stopped Billing WUs 2023-10-21T04:25:03.633+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index Index: 7315818257769233877 Partition: 3 2023-10-21T04:25:03.632+05:30 [Error] ScanCoordinator: Unable to compute index items_count for bucket_1/idx_partitioned_defer/7315818257769233877 state 0 (Index not found) 2023-10-21T04:25:03.636+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.234585ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:03.636+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 81.22µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:03.635+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:03.637+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:03.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.637+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.638+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.638+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.638+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 173 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:03.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.650+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:03.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex started 2023-10-21T04:25:03.651+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 192 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.652+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex started 2023-10-21T04:25:03.652+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:25:03.652+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.652+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.652+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:03.652+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.652+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.653+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:03.653+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:03.653+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:03.653+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex stopped 2023-10-21T04:25:03.653+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.653+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.653+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.653+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.652+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.657+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:03.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex closed 2023-10-21T04:25:03.657+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:03.657+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.657+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:03.658+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:03.658+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:03.658+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:03.658+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:03.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex stopped 2023-10-21T04:25:03.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.666+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:03.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex closed 2023-10-21T04:25:03.666+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:03.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:25:03.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 173 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:25:03.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex sucessfully destroyed 2023-10-21T04:25:03.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:25:03.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 192 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:25:03.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index destroyed 2023-10-21T04:25:03.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.691+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.692+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.696+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.696+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:03.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:03.705+05:30 [Info] switched currmeta from 39 -> 40 force true 2023-10-21T04:25:03.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 221 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.723+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.723+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:03.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex started 2023-10-21T04:25:03.723+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.723+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 199 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:03.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.748+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:03.748+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:03.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex started 2023-10-21T04:25:03.749+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Warmup took 111.616968ms 2023-10-21T04:25:03.749+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.749+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.750+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 4 WriterThreads 1 2023-10-21T04:25:03.751+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 4 Stopped Billing WUs 2023-10-21T04:25:03.751+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index Index: 7315818257769233877 Partition: 4 2023-10-21T04:25:03.751+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:03.752+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:03.768+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:03.789+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:03.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.790+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.790+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.792+05:30 [Info] switched currmeta from 40 -> 41 force true 2023-10-21T04:25:03.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 193 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.795+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex started 2023-10-21T04:25:03.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.795+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.795+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.795+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.797+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 174 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.809+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex started 2023-10-21T04:25:03.810+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:25:03.811+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:03.811+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.810+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.810+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.812+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.813+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:03.813+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.814+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:03.814+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:03.817+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex stopped 2023-10-21T04:25:03.819+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.819+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.819+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.820+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.826+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:03.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex closed 2023-10-21T04:25:03.827+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:03.828+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.828+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.828+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:03.829+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.829+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:03.830+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:03.830+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex stopped 2023-10-21T04:25:03.830+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.831+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.831+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.831+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.848+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:03.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex closed 2023-10-21T04:25:03.849+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:03.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:25:03.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 174 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:25:03.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex sucessfully destroyed 2023-10-21T04:25:03.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:25:03.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 193 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:25:03.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index destroyed 2023-10-21T04:25:03.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.877+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.877+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.882+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.882+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 222 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.885+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.885+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:03.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex started 2023-10-21T04:25:03.885+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.885+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 200 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.889+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.889+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:03.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex started 2023-10-21T04:25:03.889+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Warmup took 99.600011ms 2023-10-21T04:25:03.889+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 5 WriterThreads 1 2023-10-21T04:25:03.889+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 5 Stopped Billing WUs 2023-10-21T04:25:03.889+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index Index: 7315818257769233877 Partition: 5 2023-10-21T04:25:03.889+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:03.889+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.889+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.890+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:03.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.891+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.891+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.894+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.894+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 167 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:03.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.901+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:03.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex started 2023-10-21T04:25:03.901+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.901+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 186 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.906+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex started 2023-10-21T04:25:03.907+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:25:03.907+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:03.907+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.907+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:03.907+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:03.907+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex stopped 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.908+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:03.908+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:03.911+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:03.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex closed 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.912+05:30 [Info] switched currmeta from 41 -> 42 force true 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:03.912+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:03.912+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:03.912+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:03.912+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:03.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex stopped 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.912+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.913+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno cache..., established connections: map[172.31.5.112:58424:172.31.5.112:12000] 2023-10-21T04:25:03.915+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:03.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex closed 2023-10-21T04:25:03.915+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:03.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:25:03.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 167 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:25:03.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex sucessfully destroyed 2023-10-21T04:25:03.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:25:03.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 186 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:25:03.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index destroyed 2023-10-21T04:25:03.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.923+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.924+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:03.926+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:03.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 201 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:03.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.932+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:03.933+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:03.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex started 2023-10-21T04:25:03.948+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 223 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.949+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.949+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:03.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex started 2023-10-21T04:25:03.949+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Warmup took 58.742784ms 2023-10-21T04:25:03.950+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 6 WriterThreads 1 2023-10-21T04:25:03.950+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 6 Stopped Billing WUs 2023-10-21T04:25:03.950+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index Index: 7315818257769233877 Partition: 6 2023-10-21T04:25:03.950+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:03.950+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:03.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:03.950+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.950+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 168 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:03.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:03.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:03.954+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:03.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex started 2023-10-21T04:25:03.955+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.955+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.955+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:03.955+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:03.955+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:03.955+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 187 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:03.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:03.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:03.960+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:03.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex started 2023-10-21T04:25:03.960+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:25:03.960+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:03.960+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:03.960+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:03.960+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:03.960+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:03.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex stopped 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.965+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:03.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex closed 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:03.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:03.978+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:03.978+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:03.978+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:03.978+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:03.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex stopped 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:03.978+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:03.979+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:03.988+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex closed 2023-10-21T04:25:03.988+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:25:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 168 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:25:03.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex sucessfully destroyed 2023-10-21T04:25:03.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:25:03.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 187 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:25:03.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:03.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index destroyed 2023-10-21T04:25:04.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:04.002+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:04.002+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:04.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 202 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:04.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:04.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:04.014+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:04.014+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:04.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex started 2023-10-21T04:25:04.014+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.014+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.038+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.076+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.081+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:04.085+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:04.085+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:04.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 224 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:04.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:04.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:04.092+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:04.093+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:04.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex started 2023-10-21T04:25:04.093+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Warmup took 143.086923ms 2023-10-21T04:25:04.094+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.094+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.095+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 7 WriterThreads 1 2023-10-21T04:25:04.095+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 7 Stopped Billing WUs 2023-10-21T04:25:04.095+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index Index: 7315818257769233877 Partition: 7 2023-10-21T04:25:04.095+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:04.095+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:04.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:04.095+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:04.096+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:04.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:04.097+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:04.097+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:04.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 169 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:04.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:04.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:04.103+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:04.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex started 2023-10-21T04:25:04.103+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.103+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 188 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:04.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:04.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:04.108+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:04.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex started 2023-10-21T04:25:04.109+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:25:04.109+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:04.109+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:04.109+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.109+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.110+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:04.110+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:04.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:04.110+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:04.110+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:04.111+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:04.111+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:04.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex stopped 2023-10-21T04:25:04.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:04.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:04.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:04.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:04.117+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:04.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:04.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex closed 2023-10-21T04:25:04.118+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:04.119+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:04.119+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:04.119+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:04.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:04.120+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:04.120+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:04.120+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:04.121+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:04.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex stopped 2023-10-21T04:25:04.123+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:04.123+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:04.123+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:04.123+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:04.142+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:04.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:04.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex closed 2023-10-21T04:25:04.142+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:04.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:25:04.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 169 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:25:04.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex sucessfully destroyed 2023-10-21T04:25:04.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... 2023-10-21T04:25:04.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 188 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... 2023-10-21T04:25:04.161+05:30 [Info] switched currmeta from 42 -> 43 force true 2023-10-21T04:25:04.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index destroyed 2023-10-21T04:25:04.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:04.180+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:04.180+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:04.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:04.190+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:04.191+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:04.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 203 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:04.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:04.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:04.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:04.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:04.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex started 2023-10-21T04:25:04.193+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.193+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 225 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:04.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:04.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:04.202+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:04.202+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:04.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex started 2023-10-21T04:25:04.202+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Warmup took 107.236575ms 2023-10-21T04:25:04.203+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 8 WriterThreads 1 2023-10-21T04:25:04.203+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 8 Stopped Billing WUs 2023-10-21T04:25:04.203+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index Index: 7315818257769233877 Partition: 8 2023-10-21T04:25:04.204+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7315818257769233877 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:04.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.204+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7315818257769233877 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:04.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:04.205+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:04.206+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:04.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:04.206+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:04.207+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:04.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 189 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:04.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:04.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:04.216+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:04.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex started 2023-10-21T04:25:04.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 170 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:04.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:04.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:04.218+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:04.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex started 2023-10-21T04:25:04.218+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.218+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.219+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:25:04.219+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:04.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:04.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:04.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:04.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:04.220+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:04.220+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:04.220+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:04.220+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:04.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex stopped 2023-10-21T04:25:04.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:04.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:04.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:04.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:04.228+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:04.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:04.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex closed 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:04.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:04.229+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:04.229+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:04.229+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:04.229+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:04.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex stopped 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:04.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:04.245+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:04.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:04.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex closed 2023-10-21T04:25:04.246+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:04.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:25:04.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 170 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:25:04.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex sucessfully destroyed 2023-10-21T04:25:04.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... 2023-10-21T04:25:04.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 189 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... 2023-10-21T04:25:04.259+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[1094343481594206610:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:25:04.259+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1094343481594206610 Defn: DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006457440 2023-10-21T04:25:04.259+05:30 [Info] handleUpdateIndexRState: Index instance 1094343481594206610 rstate moved to ACTIVE 2023-10-21T04:25:04.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index destroyed 2023-10-21T04:25:04.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:04.269+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:04.269+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:04.272+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.439146ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:04.272+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16162012140118277065, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:04.272+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16162012140118277065, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:04.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:04.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:04.275+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:04.275+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:04.284+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.284+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 204 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:04.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:04.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:04.285+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:04.285+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:04.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex started 2023-10-21T04:25:04.285+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.285+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 226 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:04.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:04.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:04.288+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:04.288+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:04.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex started 2023-10-21T04:25:04.288+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Warmup took 83.53571ms 2023-10-21T04:25:04.289+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7315818257769233877 partitionId 1 WriterThreads 1 2023-10-21T04:25:04.289+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7315818257769233877 PartitionId 1 Stopped Billing WUs 2023-10-21T04:25:04.289+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7315818257769233877 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index Index: 7315818257769233877 Partition: 1 2023-10-21T04:25:04.289+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7315818257769233877, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:25:04.289+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7315818257769233877, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:25:04.289+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:04.289+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7315818257769233877 2023-10-21T04:25:04.289+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7315818257769233877 Name: idx_partitioned_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 0 2023-10-21T04:25:04.289+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:25:04.289+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:04.290+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7315818257769233877 Name: idx_partitioned_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 0 2023-10-21T04:25:04.290+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:04.290+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7315818257769233877 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:25:04.290+05:30 [Info] StorageMgr::openSnapshot IndexInst:7315818257769233877 Partition:5 No Snapshot Found. 2023-10-21T04:25:04.290+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7315818257769233877 Adding Nil Snapshot. 2023-10-21T04:25:04.290+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.290+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.293+05:30 [Info] switched currmeta from 43 -> 44 force true 2023-10-21T04:25:04.300+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 11460826864060385936,realInstId: 0, indexDefn DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:25:04.300+05:30 [Info] Indexer::handleRecoverIndex InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} 2023-10-21T04:25:04.300+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 332.835µs 2023-10-21T04:25:04.300+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 378.389µs 2023-10-21T04:25:04.300+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.301+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.301+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 240 2023-10-21T04:25:04.301+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 125 metaSz: 2575 rp [1] sn: 1800009 count: 125 metaSz: 2575 rp [2] sn: 1800010 count: 125 metaSz: 2575 2023-10-21T04:25:04.301+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 524233 recovery UsedSpace 314964 2023-10-21T04:25:04.302+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 11460826864060385936, indexDefn DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:04.302+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 30.024223ms len(expediates) 1 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:04.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.303+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:25:04.303+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 125 metaSz: 2575 rp [1] sn: 1800010 count: 125 metaSz: 2575 2023-10-21T04:25:04.303+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 502600 recovery UsedSpace 289630 2023-10-21T04:25:04.311+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.641357ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:04.316+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.348+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:25:04.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:25:04.347+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 211.697µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:04.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.349+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:25:04.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex started 2023-10-21T04:25:04.349+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.349+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.349+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:25:04.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex started 2023-10-21T04:25:04.349+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:25:04.351+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.351+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.358+05:30 [Info] switched currmeta from 44 -> 46 force true 2023-10-21T04:25:04.373+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.374+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.377+05:30 [Info] switched currmeta from 46 -> 46 force true 2023-10-21T04:25:04.379+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 7 Warmup took 77.868884ms 2023-10-21T04:25:04.379+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 7 WriterThreads 1 2023-10-21T04:25:04.379+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index Index: 11460826864060385936 Partition: 7 2023-10-21T04:25:04.380+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.382+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.383+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:25:04.383+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 124 metaSz: 2575 rp [1] sn: 1800009 count: 124 metaSz: 2575 rp [2] sn: 1800010 count: 124 metaSz: 2575 2023-10-21T04:25:04.383+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 499966 recovery UsedSpace 314184 2023-10-21T04:25:04.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.384+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:04.384+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 124 metaSz: 2575 rp [1] sn: 1800010 count: 124 metaSz: 2575 2023-10-21T04:25:04.385+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 501166 recovery UsedSpace 289630 2023-10-21T04:25:04.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:25:04.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.400+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:25:04.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex started 2023-10-21T04:25:04.400+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.400+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:25:04.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.401+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:25:04.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex started 2023-10-21T04:25:04.401+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:25:04.402+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.402+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.426+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 8 Warmup took 43.546974ms 2023-10-21T04:25:04.426+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 8 WriterThreads 1 2023-10-21T04:25:04.426+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index Index: 11460826864060385936 Partition: 8 2023-10-21T04:25:04.426+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.426+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.427+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:25:04.427+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 129 metaSz: 2575 rp [1] sn: 1800009 count: 129 metaSz: 2575 rp [2] sn: 1800010 count: 129 metaSz: 2575 2023-10-21T04:25:04.427+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 531166 recovery UsedSpace 331536 2023-10-21T04:25:04.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:04.427+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 129 metaSz: 2575 rp [1] sn: 1800010 count: 129 metaSz: 2575 2023-10-21T04:25:04.428+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 503233 recovery UsedSpace 289630 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.436+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex started 2023-10-21T04:25:04.436+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.436+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.436+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:25:04.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex started 2023-10-21T04:25:04.436+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:25:04.438+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.438+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.459+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 1 Warmup took 32.2822ms 2023-10-21T04:25:04.459+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 1 WriterThreads 1 2023-10-21T04:25:04.459+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index Index: 11460826864060385936 Partition: 1 2023-10-21T04:25:04.459+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.459+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.460+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:25:04.460+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 110 metaSz: 2575 rp [1] sn: 1800009 count: 110 metaSz: 2575 rp [2] sn: 1800010 count: 110 metaSz: 2575 2023-10-21T04:25:04.460+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 491133 recovery UsedSpace 334664 2023-10-21T04:25:04.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.460+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:04.460+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 110 metaSz: 2575 rp [1] sn: 1800010 count: 110 metaSz: 2575 2023-10-21T04:25:04.460+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 491200 recovery UsedSpace 289630 2023-10-21T04:25:04.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:25:04.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.473+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:25:04.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex started 2023-10-21T04:25:04.474+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.474+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:25:04.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.477+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:25:04.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex started 2023-10-21T04:25:04.477+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:25:04.479+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.479+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.507+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 2 Warmup took 47.271368ms 2023-10-21T04:25:04.507+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 2 WriterThreads 1 2023-10-21T04:25:04.508+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index Index: 11460826864060385936 Partition: 2 2023-10-21T04:25:04.508+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.508+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.509+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:25:04.509+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 129 metaSz: 2575 rp [1] sn: 1800009 count: 129 metaSz: 2575 rp [2] sn: 1800010 count: 129 metaSz: 2575 2023-10-21T04:25:04.509+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 503800 recovery UsedSpace 281416 2023-10-21T04:25:04.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.511+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:25:04.514+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 129 metaSz: 2575 rp [1] sn: 1800009 count: 129 metaSz: 2575 rp [2] sn: 1800010 count: 129 metaSz: 2575 2023-10-21T04:25:04.514+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 532600 recovery UsedSpace 335646 2023-10-21T04:25:04.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:25:04.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.521+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:25:04.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex started 2023-10-21T04:25:04.521+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.521+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:25:04.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.527+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:25:04.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex started 2023-10-21T04:25:04.527+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:25:04.528+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.528+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.571+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.571+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.571+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 3 Warmup took 62.36672ms 2023-10-21T04:25:04.571+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 3 WriterThreads 1 2023-10-21T04:25:04.571+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index Index: 11460826864060385936 Partition: 3 2023-10-21T04:25:04.571+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.573+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.573+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:25:04.573+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 140 metaSz: 2575 rp [1] sn: 1800009 count: 140 metaSz: 2575 rp [2] sn: 1800010 count: 140 metaSz: 2575 2023-10-21T04:25:04.573+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 539733 recovery UsedSpace 335632 2023-10-21T04:25:04.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.574+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:04.574+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 140 metaSz: 2575 rp [1] sn: 1800010 count: 140 metaSz: 2575 2023-10-21T04:25:04.574+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 510466 recovery UsedSpace 293726 2023-10-21T04:25:04.579+05:30 [Info] switched currmeta from 46 -> 47 force true 2023-10-21T04:25:04.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:25:04.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.587+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:25:04.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex started 2023-10-21T04:25:04.587+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.587+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:25:04.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.588+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:25:04.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex started 2023-10-21T04:25:04.588+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:25:04.589+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.590+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.619+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 4 Warmup took 46.319967ms 2023-10-21T04:25:04.619+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 4 WriterThreads 1 2023-10-21T04:25:04.619+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index Index: 11460826864060385936 Partition: 4 2023-10-21T04:25:04.619+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.620+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.620+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:25:04.620+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 116 metaSz: 2575 rp [1] sn: 1800009 count: 116 metaSz: 2575 rp [2] sn: 1800010 count: 116 metaSz: 2575 2023-10-21T04:25:04.620+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 495766 recovery UsedSpace 293704 2023-10-21T04:25:04.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.621+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:04.621+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 116 metaSz: 2575 rp [1] sn: 1800010 count: 116 metaSz: 2575 2023-10-21T04:25:04.621+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 495600 recovery UsedSpace 289630 2023-10-21T04:25:04.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:25:04.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.636+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:25:04.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex started 2023-10-21T04:25:04.636+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.636+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:25:04.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.637+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:25:04.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex started 2023-10-21T04:25:04.637+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:25:04.638+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.638+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.643+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.643+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.649+05:30 [Info] switched currmeta from 47 -> 48 force true 2023-10-21T04:25:04.663+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 5 Warmup took 42.989949ms 2023-10-21T04:25:04.663+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 5 WriterThreads 1 2023-10-21T04:25:04.663+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index Index: 11460826864060385936 Partition: 5 2023-10-21T04:25:04.663+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 11460826864060385936 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:04.663+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 11460826864060385936 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:04.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:04.664+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:25:04.664+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 32 numRPs: 3 rp [0] sn: 1440010 count: 127 metaSz: 2575 rp [1] sn: 1800009 count: 127 metaSz: 2575 rp [2] sn: 1800010 count: 127 metaSz: 2575 2023-10-21T04:25:04.664+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 530000 recovery UsedSpace 335632 2023-10-21T04:25:04.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:04.664+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:04.664+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 127 metaSz: 2575 rp [1] sn: 1800010 count: 127 metaSz: 2575 2023-10-21T04:25:04.664+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 502566 recovery UsedSpace 289630 2023-10-21T04:25:04.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:25:04.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:04.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:04.676+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:25:04.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex started 2023-10-21T04:25:04.677+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.677+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:25:04.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:04.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:04.678+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:25:04.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex started 2023-10-21T04:25:04.678+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:25:04.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.678+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.703+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58516:172.31.5.112:12000] 2023-10-21T04:25:04.704+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 11460826864060385936 PartitionId 6 Warmup took 40.640462ms 2023-10-21T04:25:04.704+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 11460826864060385936 partitionId 6 WriterThreads 1 2023-10-21T04:25:04.704+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 11460826864060385936 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index Index: 11460826864060385936 Partition: 6 2023-10-21T04:25:04.704+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 11460826864060385936, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:25:04.705+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 11460826864060385936, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:25:04.705+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:04.705+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 11460826864060385936 2023-10-21T04:25:04.705+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:04.705+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:25:04.705+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:04.706+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:04.706+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:04.706+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 11460826864060385936 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:25:04.707+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:1 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:25:04.708+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:2 Attempting to open snapshot (SnapshotInfo: count:110 committed:false) 2023-10-21T04:25:04.712+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:3 Attempting to open snapshot (SnapshotInfo: count:129 committed:false) 2023-10-21T04:25:04.725+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.498125ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:04.725+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 150.228µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:04.735+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:4 Attempting to open snapshot (SnapshotInfo: count:140 committed:false) 2023-10-21T04:25:04.738+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:5 Attempting to open snapshot (SnapshotInfo: count:116 committed:false) 2023-10-21T04:25:04.750+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:25:04.761+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:7 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:25:04.762+05:30 [Info] StorageMgr::openSnapshot IndexInst:11460826864060385936 Partition:8 Attempting to open snapshot (SnapshotInfo: count:124 committed:false) 2023-10-21T04:25:04.793+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.794+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.803+05:30 [Info] switched currmeta from 48 -> 50 force true 2023-10-21T04:25:04.820+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.820+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.827+05:30 [Info] switched currmeta from 50 -> 50 force true 2023-10-21T04:25:04.865+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[7315818257769233877:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:25:04.865+05:30 [Info] Indexer::handleMergePartition Source 9885812108833241320 Target 7315818257769233877 2023-10-21T04:25:04.865+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0067d4600 2023-10-21T04:25:04.866+05:30 [Info] MergePartition: sent async request to update index instance 7315818257769233877 rstate moved to ACTIVE 2023-10-21T04:25:04.882+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.61221ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:04.883+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 9885812108833241320, realInst: 7315818257769233877 2023-10-21T04:25:04.903+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 1252359852432998131, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:04.903+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 1252359852432998131, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:04.911+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.911+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.913+05:30 [Info] switched currmeta from 50 -> 51 force true 2023-10-21T04:25:04.935+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12004664282277897596,realInstId: 0, indexDefn DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:04.935+05:30 [Info] Indexer::handleRecoverIndex InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:04.935+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 242.804µs 2023-10-21T04:25:04.936+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 286.004µs 2023-10-21T04:25:04.936+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12004664282277897596 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:04.936+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12004664282277897596 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:04.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:04.937+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1056168 memSz 2092 2023-10-21T04:25:04.937+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 9 count: 1000 metaSz: 2580 rp [1] sn: 10 count: 1000 metaSz: 2578 2023-10-21T04:25:04.937+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 890833 recovery UsedSpace 41551 2023-10-21T04:25:04.937+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12004664282277897596, indexDefn DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:04.938+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 35.047341ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:04.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:04.938+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1068 2023-10-21T04:25:04.938+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 9 count: 1000 metaSz: 2580 rp [1] sn: 10 count: 1000 metaSz: 2578 2023-10-21T04:25:04.938+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 862800 recovery UsedSpace 39401 2023-10-21T04:25:04.947+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:04.947+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:04.958+05:30 [Info] switched currmeta from 51 -> 52 force true 2023-10-21T04:25:04.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 213 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex 2023-10-21T04:25:04.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:04.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:04.961+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex 2023-10-21T04:25:04.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex started 2023-10-21T04:25:04.962+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.962+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 191 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex 2023-10-21T04:25:04.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:04.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:04.965+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex 2023-10-21T04:25:04.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex started 2023-10-21T04:25:04.965+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:04.965+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12004664282277897596 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:04.965+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:04.990+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12004664282277897596 PartitionId 0 Warmup took 53.952179ms 2023-10-21T04:25:04.991+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12004664282277897596 partitionId 0 WriterThreads 4 2023-10-21T04:25:04.991+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12004664282277897596 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index Index: 12004664282277897596 Partition: 0 2023-10-21T04:25:04.991+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12004664282277897596, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:25:04.991+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12004664282277897596, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:25:04.991+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:04.991+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12004664282277897596 2023-10-21T04:25:04.991+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:25:04.991+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:04.991+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:04.991+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:25:04.992+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:04.992+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12004664282277897596 Partitions [0] 2023-10-21T04:25:04.993+05:30 [Info] StorageMgr::openSnapshot IndexInst:12004664282277897596 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:25:05.003+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.912896ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:05.003+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 88.561µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:05.037+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.037+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.041+05:30 [Info] switched currmeta from 52 -> 53 force true 2023-10-21T04:25:05.073+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.999717039383065, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:25:05.102+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.102+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.107+05:30 [Info] switched currmeta from 53 -> 55 force true 2023-10-21T04:25:05.151+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.151+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.168+05:30 [Info] switched currmeta from 55 -> 55 force true 2023-10-21T04:25:05.207+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58530:172.31.5.112:12000] 2023-10-21T04:25:05.234+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58532:172.31.5.112:12000] 2023-10-21T04:25:05.290+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58534:172.31.5.112:12000] 2023-10-21T04:25:05.322+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[11460826864060385936:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:05.322+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 7525085113233084253, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:05.322+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 7525085113233084253, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:05.326+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58536:172.31.5.112:12000] 2023-10-21T04:25:05.354+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10979931881244616620,realInstId: 0, indexDefn DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:05.354+05:30 [Info] Indexer::handleRecoverIndex InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:05.354+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 261.788µs 2023-10-21T04:25:05.354+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 302.298µs 2023-10-21T04:25:05.355+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10979931881244616620 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:05.355+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10979931881244616620 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:05.355+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:05.355+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1056136 memSz 2040 2023-10-21T04:25:05.355+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 3 count: 1000 metaSz: 2515 rp [1] sn: 4 count: 1000 metaSz: 2582 rp [2] sn: 5 count: 1000 metaSz: 2580 2023-10-21T04:25:05.355+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 873600 recovery UsedSpace 33156 2023-10-21T04:25:05.356+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10979931881244616620, indexDefn DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:05.356+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 33.758278ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:05.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:05.356+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1055974 memSz 884 2023-10-21T04:25:05.356+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 4 count: 1000 metaSz: 2582 rp [1] sn: 5 count: 1000 metaSz: 2580 2023-10-21T04:25:05.356+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 854400 recovery UsedSpace 39099 2023-10-21T04:25:05.373+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.373+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 213 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex 2023-10-21T04:25:05.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:05.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:05.374+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex 2023-10-21T04:25:05.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex started 2023-10-21T04:25:05.374+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:05.375+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:05.377+05:30 [Info] switched currmeta from 55 -> 56 force true 2023-10-21T04:25:05.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 191 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex 2023-10-21T04:25:05.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:05.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:05.388+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex 2023-10-21T04:25:05.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex started 2023-10-21T04:25:05.388+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10979931881244616620 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:05.389+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:05.389+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:05.411+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10979931881244616620 PartitionId 0 Warmup took 56.157037ms 2023-10-21T04:25:05.411+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10979931881244616620 partitionId 0 WriterThreads 4 2023-10-21T04:25:05.411+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10979931881244616620 PartitionId 0 Stopped Billing WUs 2023-10-21T04:25:05.411+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10979931881244616620 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index Index: 10979931881244616620 Partition: 0 2023-10-21T04:25:05.411+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 10979931881244616620, shardIdMap: map[0:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:25:05.411+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 10979931881244616620, partnShardIdMap: map[0:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:25:05.412+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:05.412+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10979931881244616620 2023-10-21T04:25:05.412+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:25:05.412+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:05.412+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:05.413+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:25:05.413+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:05.415+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10979931881244616620 Partitions [0] 2023-10-21T04:25:05.416+05:30 [Info] StorageMgr::openSnapshot IndexInst:10979931881244616620 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:25:05.427+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.833863ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:05.427+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 95.108µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:05.440+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.440+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.444+05:30 [Info] switched currmeta from 56 -> 57 force true 2023-10-21T04:25:05.486+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 0 InstId: 11460826864060385936 RealInstId: 11460826864060385936 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:25:05.486+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 0 InstId: 12004664282277897596 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:05.486+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 0 InstId: 7609950505007110692 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:05.489+05:30 [Info] ShardRebalancer::computeProgress 0.7028392857142858 2023-10-21T04:25:05.498+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.505+05:30 [Info] switched currmeta from 57 -> 58 force true 2023-10-21T04:25:05.543+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.543+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.546+05:30 [Info] switched currmeta from 58 -> 59 force true 2023-10-21T04:25:05.585+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:05.586+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:05.595+05:30 [Info] switched currmeta from 59 -> 60 force true 2023-10-21T04:25:05.640+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58546:172.31.5.112:12000] 2023-10-21T04:25:05.688+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58548:172.31.5.112:12000] 2023-10-21T04:25:05.732+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58550:172.31.5.112:12000] 2023-10-21T04:25:05.793+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58558:172.31.5.112:12000] 2023-10-21T04:25:05.822+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58560:172.31.5.112:12000] 2023-10-21T04:25:05.877+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/1972139801924f1fdf1188af4e7137b9 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7028392857142858 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:05.877+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:05.879+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:25:05.880+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:25:05.944+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[12004664282277897596:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:05.945+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary_defer, defnId: 8378417662949994532, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:05.945+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary_defer, defnId: 8378417662949994532, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:05.970+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17394655409336449770,realInstId: 0, indexDefn DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:05.970+05:30 [Info] Indexer::handleRecoverIndex InstId: 17394655409336449770 Defn: DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:05.971+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 365.843µs 2023-10-21T04:25:05.971+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 413.596µs 2023-10-21T04:25:05.971+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17394655409336449770 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:05.971+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17394655409336449770 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:05.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:05.971+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:05.972+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:05.974+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17394655409336449770, indexDefn DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:05.974+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.395424ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:05.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:05.980+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:05.982+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:05.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 192 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:05.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:05.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:05.989+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:05.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex started 2023-10-21T04:25:05.989+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:05.989+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:05.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 214 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:05.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:05.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:05.990+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:05.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex started 2023-10-21T04:25:05.992+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:05.992+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:05.992+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17394655409336449770 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:05.992+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17394655409336449770 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:05.992+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:05.992+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:05.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:05.992+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:05.992+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:05.992+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:05.992+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:05.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex stopped 2023-10-21T04:25:05.993+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:05.993+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:05.993+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:05.993+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:05.993+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:05.997+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:05.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:05.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex closed 2023-10-21T04:25:05.997+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:05.997+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:05.997+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:05.997+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:05.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:05.998+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:05.998+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:05.998+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:05.998+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:05.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex stopped 2023-10-21T04:25:05.998+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:05.998+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:05.998+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:05.998+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.005+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:06.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex closed 2023-10-21T04:25:06.005+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:06.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex ... 2023-10-21T04:25:06.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 192 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex ... 2023-10-21T04:25:06.012+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.012+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex sucessfully destroyed 2023-10-21T04:25:06.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex ... 2023-10-21T04:25:06.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 214 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex ... 2023-10-21T04:25:06.017+05:30 [Info] switched currmeta from 60 -> 61 force true 2023-10-21T04:25:06.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index destroyed 2023-10-21T04:25:06.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:06.034+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.035+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:06.043+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.043+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 205 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:06.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:06.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:06.047+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:06.047+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:06.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex started 2023-10-21T04:25:06.047+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.047+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 227 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:06.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:06.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:06.056+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:06.056+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:06.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex started 2023-10-21T04:25:06.056+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17394655409336449770 PartitionId 0 Warmup took 85.0464ms 2023-10-21T04:25:06.056+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.056+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.058+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17394655409336449770 partitionId 0 WriterThreads 4 2023-10-21T04:25:06.067+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17394655409336449770 PartitionId 0 Stopped Billing WUs 2023-10-21T04:25:06.068+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17394655409336449770 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index Index: 17394655409336449770 Partition: 0 2023-10-21T04:25:06.068+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17394655409336449770, shardIdMap: map[0:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:25:06.068+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17394655409336449770, partnShardIdMap: map[0:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:25:06.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.995443760324373, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:25:06.070+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17394655409336449770 Defn: DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:06.076+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17394655409336449770 2023-10-21T04:25:06.079+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17394655409336449770 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:25:06.083+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:06.077+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.861685ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:06.089+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:06.090+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17394655409336449770 Name: idx_secondary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:25:06.093+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17394655409336449770 Defn: DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:06.093+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17394655409336449770 Partitions [0] 2023-10-21T04:25:06.093+05:30 [Info] StorageMgr::openSnapshot IndexInst:17394655409336449770 Partition:0 No Snapshot Found. 2023-10-21T04:25:06.093+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17394655409336449770 Adding Nil Snapshot. 2023-10-21T04:25:06.094+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 81.942µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:06.110+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.110+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.121+05:30 [Info] switched currmeta from 61 -> 62 force true 2023-10-21T04:25:06.187+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.187+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.193+05:30 [Info] switched currmeta from 62 -> 63 force true 2023-10-21T04:25:06.243+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.243+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.247+05:30 [Info] switched currmeta from 63 -> 64 force true 2023-10-21T04:25:06.310+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.310+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.318+05:30 [Info] switched currmeta from 64 -> 65 force true 2023-10-21T04:25:06.362+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[10979931881244616620:INDEX_STATE_RECOVERED 11460826864060385936:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:06.363+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 15431181238515168303, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:06.363+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 15431181238515168303, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:06.447+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 13987057765505463258,realInstId: 0, indexDefn DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:25:06.447+05:30 [Info] Indexer::handleRecoverIndex InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} 2023-10-21T04:25:06.448+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 507.42µs 2023-10-21T04:25:06.448+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 552.759µs 2023-10-21T04:25:06.448+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:06.448+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 2 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:06.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.449+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.449+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.449+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 13987057765505463258, indexDefn DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:06.449+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 86.327367ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:06.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.450+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.450+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 188 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:06.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.456+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:06.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex started 2023-10-21T04:25:06.456+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.456+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 169 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:06.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.467+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:06.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex started 2023-10-21T04:25:06.467+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.470+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:25:06.470+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:06.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:06.470+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:06.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex stopped 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.470+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.471+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.474+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:06.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex closed 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:06.475+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:06.475+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:06.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex stopped 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.475+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.482+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:06.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex closed 2023-10-21T04:25:06.485+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:06.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:25:06.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 169 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:25:06.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex sucessfully destroyed 2023-10-21T04:25:06.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:25:06.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 188 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:25:06.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index destroyed 2023-10-21T04:25:06.496+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.496+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.499+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.499+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.501+05:30 [Info] switched currmeta from 65 -> 66 force true 2023-10-21T04:25:06.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.505+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.505+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 198 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:06.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.507+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:06.507+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:06.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex started 2023-10-21T04:25:06.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 220 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:06.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.518+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:06.518+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:06.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex started 2023-10-21T04:25:06.518+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.518+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Warmup took 69.830065ms 2023-10-21T04:25:06.518+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.519+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 2 WriterThreads 1 2023-10-21T04:25:06.519+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 2 Stopped Billing WUs 2023-10-21T04:25:06.519+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index Index: 13987057765505463258 Partition: 2 2023-10-21T04:25:06.519+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:06.519+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 3 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:06.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.519+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.520+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.520+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.526+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 170 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:06.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.529+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:06.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex started 2023-10-21T04:25:06.530+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.530+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 189 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:06.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.539+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:06.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex started 2023-10-21T04:25:06.539+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:25:06.539+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:06.539+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:06.539+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:06.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex stopped 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.539+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.540+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.554+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:06.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex closed 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:06.555+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:06.555+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:06.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex stopped 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.555+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.566+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:06.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex closed 2023-10-21T04:25:06.571+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:06.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:25:06.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 170 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:25:06.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex sucessfully destroyed 2023-10-21T04:25:06.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:25:06.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 189 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:25:06.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index destroyed 2023-10-21T04:25:06.586+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.587+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.591+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.591+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.594+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.594+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.598+05:30 [Info] switched currmeta from 66 -> 67 force true 2023-10-21T04:25:06.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 199 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:06.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.598+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:06.598+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:06.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex started 2023-10-21T04:25:06.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 221 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:06.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.600+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:06.600+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:06.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex started 2023-10-21T04:25:06.600+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Warmup took 81.100283ms 2023-10-21T04:25:06.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.601+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 3 WriterThreads 1 2023-10-21T04:25:06.601+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 3 Stopped Billing WUs 2023-10-21T04:25:06.601+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index Index: 13987057765505463258 Partition: 3 2023-10-21T04:25:06.601+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:06.601+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 4 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:06.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.601+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.602+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 171 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:06.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.608+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:06.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex started 2023-10-21T04:25:06.609+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.609+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.625+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:06.625+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:06.628+05:30 [Info] switched currmeta from 67 -> 68 force true 2023-10-21T04:25:06.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 190 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:06.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.629+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:06.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex started 2023-10-21T04:25:06.629+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.629+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:06.629+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:06.629+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:06.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex stopped 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.629+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.630+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:06.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex closed 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:06.644+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:06.644+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:06.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex stopped 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.644+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.645+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.658+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:06.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex closed 2023-10-21T04:25:06.659+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:06.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:25:06.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 171 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:25:06.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex sucessfully destroyed 2023-10-21T04:25:06.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:25:06.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 190 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:25:06.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index destroyed 2023-10-21T04:25:06.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.682+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.682+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.686+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.686+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 222 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:06.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.693+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:06.693+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:06.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex started 2023-10-21T04:25:06.693+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.693+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 200 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:06.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.700+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:06.700+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:06.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex started 2023-10-21T04:25:06.700+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Warmup took 98.99346ms 2023-10-21T04:25:06.700+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 4 WriterThreads 1 2023-10-21T04:25:06.700+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 4 Stopped Billing WUs 2023-10-21T04:25:06.701+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index Index: 13987057765505463258 Partition: 4 2023-10-21T04:25:06.701+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:06.701+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 5 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:06.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.701+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.702+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.702+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.702+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.702+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.702+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 191 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.713+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex started 2023-10-21T04:25:06.713+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.713+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 172 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.713+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:06.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex started 2023-10-21T04:25:06.714+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:25:06.714+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:06.714+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:06.714+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:06.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex stopped 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.714+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:06.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex closed 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:06.721+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:06.721+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:06.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex stopped 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.721+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.733+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:06.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex closed 2023-10-21T04:25:06.733+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:06.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:25:06.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 172 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:25:06.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex sucessfully destroyed 2023-10-21T04:25:06.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:25:06.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 191 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:25:06.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index destroyed 2023-10-21T04:25:06.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.746+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.746+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.754+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 201 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:06.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.768+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:06.768+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:06.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex started 2023-10-21T04:25:06.768+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.768+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 223 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:06.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.775+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:06.775+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:06.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex started 2023-10-21T04:25:06.775+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Warmup took 73.926516ms 2023-10-21T04:25:06.775+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 5 WriterThreads 1 2023-10-21T04:25:06.775+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 5 Stopped Billing WUs 2023-10-21T04:25:06.775+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index Index: 13987057765505463258 Partition: 5 2023-10-21T04:25:06.775+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:06.776+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 6 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:06.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.776+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.776+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.776+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.777+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 192 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.792+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex started 2023-10-21T04:25:06.792+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.792+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 173 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.792+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex started 2023-10-21T04:25:06.792+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:25:06.792+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:06.792+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.792+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.792+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:06.792+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:06.792+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:06.792+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:06.792+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:06.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex stopped 2023-10-21T04:25:06.793+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.793+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.793+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.793+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.793+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.793+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.804+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:06.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex closed 2023-10-21T04:25:06.804+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:06.804+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.804+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.804+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:06.804+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:06.804+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:06.804+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:06.804+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:06.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex stopped 2023-10-21T04:25:06.805+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.805+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.805+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.805+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.817+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:06.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex closed 2023-10-21T04:25:06.818+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:06.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:25:06.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 173 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:25:06.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex sucessfully destroyed 2023-10-21T04:25:06.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:25:06.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 192 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:25:06.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index destroyed 2023-10-21T04:25:06.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.838+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.838+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.844+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.844+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 202 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:06.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.848+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:06.848+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:06.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex started 2023-10-21T04:25:06.849+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 224 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:06.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.850+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:06.850+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:06.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex started 2023-10-21T04:25:06.850+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Warmup took 74.444537ms 2023-10-21T04:25:06.850+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 6 WriterThreads 1 2023-10-21T04:25:06.851+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 6 Stopped Billing WUs 2023-10-21T04:25:06.851+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index Index: 13987057765505463258 Partition: 6 2023-10-21T04:25:06.851+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:06.851+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 7 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:06.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.851+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.851+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.852+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.852+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.852+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.852+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.854+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 193 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.857+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex started 2023-10-21T04:25:06.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 174 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.857+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex started 2023-10-21T04:25:06.857+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:25:06.857+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:06.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.857+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:06.858+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:06.858+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:06.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex stopped 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.858+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.875+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:06.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex closed 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:06.876+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:06.876+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:06.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex stopped 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.876+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.887+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:06.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:06.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex closed 2023-10-21T04:25:06.887+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:06.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:25:06.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 174 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:25:06.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex sucessfully destroyed 2023-10-21T04:25:06.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:25:06.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 193 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:25:06.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index destroyed 2023-10-21T04:25:06.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.936+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.936+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:06.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.942+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:06.942+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:06.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 203 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:06.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.946+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:06.946+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:06.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex started 2023-10-21T04:25:06.946+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.946+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 225 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:06.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.960+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:06.960+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:06.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex started 2023-10-21T04:25:06.960+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Warmup took 109.125728ms 2023-10-21T04:25:06.960+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 7 WriterThreads 1 2023-10-21T04:25:06.960+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 7 Stopped Billing WUs 2023-10-21T04:25:06.960+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index Index: 13987057765505463258 Partition: 7 2023-10-21T04:25:06.960+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:06.961+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 8 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:06.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:06.961+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.961+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.962+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.962+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:06.962+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:06.962+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:06.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 186 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:06.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:06.976+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex started 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 167 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:06.976+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex started 2023-10-21T04:25:06.976+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:25:06.976+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:06.976+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:06.976+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:06.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex stopped 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:06.976+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:06.979+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[17394655409336449770:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:25:06.979+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17394655409336449770 Defn: DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009341440 2023-10-21T04:25:06.979+05:30 [Info] handleUpdateIndexRState: Index instance 17394655409336449770 rstate moved to ACTIVE 2023-10-21T04:25:06.997+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.406098ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.006+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 573521267164877540, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:07.006+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 573521267164877540, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:07.025+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:07.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:07.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex closed 2023-10-21T04:25:07.025+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:07.025+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:07.025+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:07.025+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:07.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:07.025+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:07.025+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:07.025+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:07.025+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:07.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex stopped 2023-10-21T04:25:07.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:07.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:07.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:07.026+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:07.047+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 9912003609070511414,realInstId: 0, indexDefn DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:25:07.047+05:30 [Info] Indexer::handleRecoverIndex InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} 2023-10-21T04:25:07.048+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 528.806µs 2023-10-21T04:25:07.048+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 586.208µs 2023-10-21T04:25:07.048+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:07.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:07.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex closed 2023-10-21T04:25:07.048+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.049+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:07.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:25:07.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 167 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:25:07.049+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 7 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.049+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 9912003609070511414, indexDefn DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:07.049+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 43.495973ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex sucessfully destroyed 2023-10-21T04:25:07.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:25:07.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 186 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:25:07.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index destroyed 2023-10-21T04:25:07.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:25:07.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 125 metaSz: 2576 rp [1] sn: 1800010 count: 125 metaSz: 2576 2023-10-21T04:25:07.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 522033 recovery UsedSpace 285942 2023-10-21T04:25:07.073+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9939471755319096 2023-10-21T04:25:07.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:07.076+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:07.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:25:07.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.079+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:07.079+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 125 metaSz: 2576 rp [1] sn: 1800010 count: 125 metaSz: 2576 2023-10-21T04:25:07.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:07.079+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:07.079+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:07.079+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:07.079+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 501166 recovery UsedSpace 256490 2023-10-21T04:25:07.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.091+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.092+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:25:07.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex started 2023-10-21T04:25:07.092+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.092+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.097+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.097+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.103+05:30 [Info] switched currmeta from 68 -> 70 force true 2023-10-21T04:25:07.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 226 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:07.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:07.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:07.110+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:07.110+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:07.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex started 2023-10-21T04:25:07.110+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.110+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.110+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.110+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 204 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:07.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:07.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:07.111+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:07.111+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:07.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex started 2023-10-21T04:25:07.111+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Warmup took 150.573821ms 2023-10-21T04:25:07.112+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 8 WriterThreads 1 2023-10-21T04:25:07.112+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 8 Stopped Billing WUs 2023-10-21T04:25:07.112+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index Index: 13987057765505463258 Partition: 8 2023-10-21T04:25:07.112+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13987057765505463258 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:07.113+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 13987057765505463258 PartitionId: 1 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:07.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:07.114+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:07.114+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:07.113+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.113+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:07.115+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:07.115+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:07.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:25:07.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.119+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:25:07.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex started 2023-10-21T04:25:07.119+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:25:07.119+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.119+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 168 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:07.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:07.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:07.122+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:07.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex started 2023-10-21T04:25:07.123+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 187 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:07.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:07.123+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:07.124+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:07.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex started 2023-10-21T04:25:07.124+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.124+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.124+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:25:07.125+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:07.125+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:07.125+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:07.125+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:07.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:07.126+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:07.126+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:07.126+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:07.127+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:07.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex stopped 2023-10-21T04:25:07.127+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:07.127+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:07.127+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:07.127+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:07.136+05:30 [Info] switched currmeta from 70 -> 70 force true 2023-10-21T04:25:07.137+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:07.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:07.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex closed 2023-10-21T04:25:07.138+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:07.138+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:07.138+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:07.138+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:07.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:07.139+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:07.139+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:07.139+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:07.139+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:07.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex stopped 2023-10-21T04:25:07.140+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:07.140+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:07.140+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:07.140+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:07.146+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:07.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:07.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex closed 2023-10-21T04:25:07.147+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:07.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:25:07.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 168 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:25:07.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex sucessfully destroyed 2023-10-21T04:25:07.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:25:07.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 187 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:25:07.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index destroyed 2023-10-21T04:25:07.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery) 2023-10-21T04:25:07.161+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:07.162+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:07.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:07.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:07.170+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:07.170+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:07.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Assign plasmaId 205 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:07.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510 2023-10-21T04:25:07.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data 2023-10-21T04:25:07.187+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:07.188+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:07.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex started 2023-10-21T04:25:07.188+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.188+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 227 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:07.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:07.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:07.192+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:07.192+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:07.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex started 2023-10-21T04:25:07.192+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Warmup took 79.254934ms 2023-10-21T04:25:07.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.193+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13987057765505463258 partitionId 1 WriterThreads 1 2023-10-21T04:25:07.193+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 13987057765505463258 PartitionId 1 Stopped Billing WUs 2023-10-21T04:25:07.193+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13987057765505463258 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index Index: 13987057765505463258 Partition: 1 2023-10-21T04:25:07.193+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 13987057765505463258, shardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]] err: 2023-10-21T04:25:07.193+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 13987057765505463258, partnShardIdMap: map[1:[10890719122371003479 7659339677706884510] 2:[10890719122371003479 7659339677706884510] 3:[10890719122371003479 7659339677706884510] 4:[10890719122371003479 7659339677706884510] 5:[10890719122371003479 7659339677706884510] 6:[10890719122371003479 7659339677706884510] 7:[10890719122371003479 7659339677706884510] 8:[10890719122371003479 7659339677706884510]], err: 2023-10-21T04:25:07.193+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:07.193+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13987057765505463258 2023-10-21T04:25:07.194+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13987057765505463258 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 0 2023-10-21T04:25:07.194+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:25:07.194+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:07.194+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13987057765505463258 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 0 2023-10-21T04:25:07.194+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:07.194+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13987057765505463258 Partitions [7 8 1 2 3 4 5 6] 2023-10-21T04:25:07.194+05:30 [Info] StorageMgr::openSnapshot IndexInst:13987057765505463258 Partition:7 No Snapshot Found. 2023-10-21T04:25:07.195+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 13987057765505463258 Adding Nil Snapshot. 2023-10-21T04:25:07.201+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 7 Warmup took 152.34962ms 2023-10-21T04:25:07.202+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 7 WriterThreads 1 2023-10-21T04:25:07.202+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index Index: 9912003609070511414 Partition: 7 2023-10-21T04:25:07.202+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.202+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 8 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.202+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:25:07.202+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 121 metaSz: 2576 rp [1] sn: 1800010 count: 121 metaSz: 2576 2023-10-21T04:25:07.202+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 497100 recovery UsedSpace 285162 2023-10-21T04:25:07.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:07.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 121 metaSz: 2576 rp [1] sn: 1800010 count: 121 metaSz: 2576 2023-10-21T04:25:07.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 496200 recovery UsedSpace 256490 2023-10-21T04:25:07.218+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 23.244759ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.218+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 129.626µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:07.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:25:07.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.223+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:25:07.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex started 2023-10-21T04:25:07.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:25:07.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.224+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:25:07.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex started 2023-10-21T04:25:07.224+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:25:07.226+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.227+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.240+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 8 Warmup took 38.165856ms 2023-10-21T04:25:07.240+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 8 WriterThreads 1 2023-10-21T04:25:07.240+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index Index: 9912003609070511414 Partition: 8 2023-10-21T04:25:07.241+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.241+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 1 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:25:07.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 134 metaSz: 2576 rp [1] sn: 1800010 count: 134 metaSz: 2576 2023-10-21T04:25:07.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 534600 recovery UsedSpace 286144 2023-10-21T04:25:07.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:07.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 134 metaSz: 2576 rp [1] sn: 1800010 count: 134 metaSz: 2576 2023-10-21T04:25:07.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 506033 recovery UsedSpace 256490 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.247+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex started 2023-10-21T04:25:07.247+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.247+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.247+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:25:07.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex started 2023-10-21T04:25:07.247+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:25:07.250+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.250+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.274+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.275+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.275+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 1 Warmup took 34.632197ms 2023-10-21T04:25:07.276+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 1 WriterThreads 1 2023-10-21T04:25:07.276+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index Index: 9912003609070511414 Partition: 1 2023-10-21T04:25:07.276+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.276+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 2 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.277+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:25:07.277+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 113 metaSz: 2576 rp [1] sn: 1800010 count: 113 metaSz: 2576 2023-10-21T04:25:07.277+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 492633 recovery UsedSpace 285162 2023-10-21T04:25:07.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.278+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:25:07.278+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 113 metaSz: 2576 rp [1] sn: 1800010 count: 113 metaSz: 2576 2023-10-21T04:25:07.278+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 493966 recovery UsedSpace 256490 2023-10-21T04:25:07.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:25:07.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.293+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:25:07.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex started 2023-10-21T04:25:07.293+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.293+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:25:07.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.296+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:25:07.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex started 2023-10-21T04:25:07.296+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:25:07.298+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.298+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.310+05:30 [Info] switched currmeta from 70 -> 73 force true 2023-10-21T04:25:07.322+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.338+05:30 [Info] switched currmeta from 73 -> 73 force true 2023-10-21T04:25:07.346+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 2 Warmup took 69.847491ms 2023-10-21T04:25:07.347+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 2 WriterThreads 1 2023-10-21T04:25:07.347+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index Index: 9912003609070511414 Partition: 2 2023-10-21T04:25:07.347+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.348+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 3 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.349+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 258 2023-10-21T04:25:07.349+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 132 metaSz: 2576 rp [1] sn: 1800010 count: 132 metaSz: 2576 2023-10-21T04:25:07.349+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 533033 recovery UsedSpace 286144 2023-10-21T04:25:07.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:07.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 132 metaSz: 2576 rp [1] sn: 1800010 count: 132 metaSz: 2576 2023-10-21T04:25:07.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 504766 recovery UsedSpace 256490 2023-10-21T04:25:07.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:25:07.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.367+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:25:07.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex started 2023-10-21T04:25:07.367+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.367+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:25:07.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.368+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:25:07.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex started 2023-10-21T04:25:07.368+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:25:07.370+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.370+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.374+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:58728:172.31.5.112:12000] 2023-10-21T04:25:07.392+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 3 Warmup took 43.38061ms 2023-10-21T04:25:07.393+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 3 WriterThreads 1 2023-10-21T04:25:07.393+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index Index: 9912003609070511414 Partition: 3 2023-10-21T04:25:07.393+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.393+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 4 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.393+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:25:07.394+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 124 metaSz: 2576 rp [1] sn: 1800010 count: 124 metaSz: 2576 2023-10-21T04:25:07.394+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 497600 recovery UsedSpace 281066 2023-10-21T04:25:07.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.395+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:25:07.395+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 124 metaSz: 2576 rp [1] sn: 1800010 count: 124 metaSz: 2576 2023-10-21T04:25:07.395+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 498900 recovery UsedSpace 256490 2023-10-21T04:25:07.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:25:07.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.399+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:25:07.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex started 2023-10-21T04:25:07.399+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.399+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:25:07.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.401+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:25:07.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex started 2023-10-21T04:25:07.402+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:25:07.402+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.402+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.418+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 4 Warmup took 24.759849ms 2023-10-21T04:25:07.418+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 4 WriterThreads 1 2023-10-21T04:25:07.419+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index Index: 9912003609070511414 Partition: 4 2023-10-21T04:25:07.419+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.419+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 5 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.420+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:25:07.420+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 111 metaSz: 2576 rp [1] sn: 1800010 count: 111 metaSz: 2576 2023-10-21T04:25:07.421+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 492066 recovery UsedSpace 256490 2023-10-21T04:25:07.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.421+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:25:07.421+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 111 metaSz: 2576 rp [1] sn: 1800010 count: 111 metaSz: 2576 2023-10-21T04:25:07.421+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 491666 recovery UsedSpace 285162 2023-10-21T04:25:07.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:25:07.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.428+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:25:07.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex started 2023-10-21T04:25:07.428+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.428+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:25:07.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.429+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:25:07.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex started 2023-10-21T04:25:07.429+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:25:07.429+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.429+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.450+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.450+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.458+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[13987057765505463258:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:25:07.458+05:30 [Info] Indexer::handleMergePartition Source 12820935061706293349 Target 13987057765505463258 2023-10-21T04:25:07.458+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009fda960 2023-10-21T04:25:07.459+05:30 [Info] MergePartition: sent async request to update index instance 13987057765505463258 rstate moved to ACTIVE 2023-10-21T04:25:07.465+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 5 Warmup took 45.827151ms 2023-10-21T04:25:07.466+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 5 WriterThreads 1 2023-10-21T04:25:07.466+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index Index: 9912003609070511414 Partition: 5 2023-10-21T04:25:07.466+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9912003609070511414 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:07.466+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9912003609070511414 PartitionId: 6 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:07.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery) 2023-10-21T04:25:07.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:25:07.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 140 metaSz: 2576 rp [1] sn: 1800010 count: 140 metaSz: 2576 2023-10-21T04:25:07.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 539900 recovery UsedSpace 265664 2023-10-21T04:25:07.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:07.468+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:25:07.468+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 140 metaSz: 2576 rp [1] sn: 1800010 count: 140 metaSz: 2576 2023-10-21T04:25:07.468+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 510966 recovery UsedSpace 256490 2023-10-21T04:25:07.469+05:30 [Info] switched currmeta from 73 -> 74 force true 2023-10-21T04:25:07.483+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 24.287408ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.484+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 12820935061706293349, realInst: 13987057765505463258 2023-10-21T04:25:07.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:25:07.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215 2023-10-21T04:25:07.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data 2023-10-21T04:25:07.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:25:07.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex started 2023-10-21T04:25:07.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.486+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 8857269605176042330, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:07.486+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 8857269605176042330, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:07.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:25:07.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:07.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:07.491+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:25:07.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex started 2023-10-21T04:25:07.491+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:25:07.493+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.493+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.513+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9912003609070511414 PartitionId 6 Warmup took 47.107303ms 2023-10-21T04:25:07.514+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9912003609070511414 partitionId 6 WriterThreads 1 2023-10-21T04:25:07.514+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9912003609070511414 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index Index: 9912003609070511414 Partition: 6 2023-10-21T04:25:07.514+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 9912003609070511414, shardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]] err: 2023-10-21T04:25:07.514+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 9912003609070511414, partnShardIdMap: map[1:[2864853750198994630 12177293282243154215] 2:[2864853750198994630 12177293282243154215] 3:[2864853750198994630 12177293282243154215] 4:[2864853750198994630 12177293282243154215] 5:[2864853750198994630 12177293282243154215] 6:[2864853750198994630 12177293282243154215] 7:[2864853750198994630 12177293282243154215] 8:[2864853750198994630 12177293282243154215]], err: 2023-10-21T04:25:07.514+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:07.515+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 9912003609070511414 2023-10-21T04:25:07.515+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:07.516+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:25:07.516+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:07.519+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:07.519+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2296526085582506469,realInstId: 0, indexDefn DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:07.525+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:07.525+05:30 [Info] Indexer::handleRecoverIndex InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:07.526+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2296526085582506469, indexDefn DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:07.527+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 40.893171ms len(expediates) 1 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.526+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 6.650418ms 2023-10-21T04:25:07.526+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 833.51µs 2023-10-21T04:25:07.528+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 9912003609070511414 Partitions [5 6 7 8 1 2 3 4] 2023-10-21T04:25:07.529+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:5 Attempting to open snapshot (SnapshotInfo: count:111 committed:false) 2023-10-21T04:25:07.530+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:6 Attempting to open snapshot (SnapshotInfo: count:140 committed:false) 2023-10-21T04:25:07.531+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:7 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:25:07.526+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2296526085582506469 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:07.533+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2296526085582506469 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:07.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:07.533+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1055078 memSz 820 2023-10-21T04:25:07.533+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 33 numRPs: 2 rp [0] sn: 1800009 count: 1000 metaSz: 2554 rp [1] sn: 1800010 count: 1000 metaSz: 2553 2023-10-21T04:25:07.534+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 867533 recovery UsedSpace 293675 2023-10-21T04:25:07.537+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:8 Attempting to open snapshot (SnapshotInfo: count:121 committed:false) 2023-10-21T04:25:07.538+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:1 Attempting to open snapshot (SnapshotInfo: count:134 committed:false) 2023-10-21T04:25:07.540+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:2 Attempting to open snapshot (SnapshotInfo: count:113 committed:false) 2023-10-21T04:25:07.541+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:3 Attempting to open snapshot (SnapshotInfo: count:132 committed:false) 2023-10-21T04:25:07.571+05:30 [Info] StorageMgr::openSnapshot IndexInst:9912003609070511414 Partition:4 Attempting to open snapshot (SnapshotInfo: count:124 committed:false) 2023-10-21T04:25:07.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:25:07.545+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.799664ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.548+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.586+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:07.587+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 141.084µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.587+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:07.591+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:25:07.589+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:07.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex started 2023-10-21T04:25:07.594+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2296526085582506469 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:07.594+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:07.595+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:07.618+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2296526085582506469 PartitionId 0 Warmup took 84.893663ms 2023-10-21T04:25:07.618+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2296526085582506469 partitionId 0 WriterThreads 4 2023-10-21T04:25:07.618+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2296526085582506469 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index Index: 2296526085582506469 Partition: 0 2023-10-21T04:25:07.618+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2296526085582506469, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:25:07.619+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2296526085582506469, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:25:07.619+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:07.620+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2296526085582506469 2023-10-21T04:25:07.620+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:07.620+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:07.621+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:07.621+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:07.621+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:07.622+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2296526085582506469 Partitions [0] 2023-10-21T04:25:07.623+05:30 [Info] StorageMgr::openSnapshot IndexInst:2296526085582506469 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:25:07.625+05:30 [Info] switched currmeta from 74 -> 77 force true 2023-10-21T04:25:07.643+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.77818ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:07.643+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 75.208µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:07.658+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.710+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.720+05:30 [Info] switched currmeta from 77 -> 80 force true 2023-10-21T04:25:07.732+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:07.732+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:07.739+05:30 [Info] switched currmeta from 80 -> 80 force true 2023-10-21T04:25:08.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9901326559737428 2023-10-21T04:25:08.072+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[9912003609070511414:INDEX_STATE_RECOVERED 12004664282277897596:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:08.073+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 12338036531800130039, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:08.073+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 12338036531800130039, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:08.093+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15667952562388529486,realInstId: 0, indexDefn DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:08.093+05:30 [Info] Indexer::handleRecoverIndex InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:08.093+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 168.959µs 2023-10-21T04:25:08.093+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 208.237µs 2023-10-21T04:25:08.093+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15667952562388529486 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:08.094+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15667952562388529486 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:08.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:08.094+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1055100 memSz 774 2023-10-21T04:25:08.094+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 1000 metaSz: 2557 rp [1] sn: 1800010 count: 1000 metaSz: 2556 2023-10-21T04:25:08.094+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 862266 recovery UsedSpace 260515 2023-10-21T04:25:08.094+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15667952562388529486, indexDefn DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:08.095+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 21.671184ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:08.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.125+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:25:08.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:08.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:08.128+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:25:08.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex started 2023-10-21T04:25:08.128+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15667952562388529486 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:08.128+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:08.128+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:08.136+05:30 [Info] switched currmeta from 80 -> 81 force true 2023-10-21T04:25:08.157+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15667952562388529486 PartitionId 0 Warmup took 63.046637ms 2023-10-21T04:25:08.157+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15667952562388529486 partitionId 0 WriterThreads 4 2023-10-21T04:25:08.157+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15667952562388529486 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index Index: 15667952562388529486 Partition: 0 2023-10-21T04:25:08.157+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15667952562388529486, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:25:08.157+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15667952562388529486, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:25:08.157+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:08.158+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15667952562388529486 2023-10-21T04:25:08.158+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:08.158+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:08.158+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:08.158+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:08.158+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:08.158+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15667952562388529486 Partitions [0] 2023-10-21T04:25:08.159+05:30 [Info] StorageMgr::openSnapshot IndexInst:15667952562388529486 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:25:08.171+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.89116ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:08.173+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 150.939µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:08.184+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.184+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.186+05:30 [Info] switched currmeta from 81 -> 82 force true 2023-10-21T04:25:08.225+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58832:172.31.5.112:12000] 2023-10-21T04:25:08.228+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58836:172.31.5.112:12000] 2023-10-21T04:25:08.262+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58838:172.31.5.112:12000] 2023-10-21T04:25:08.270+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58840:172.31.5.112:12000] 2023-10-21T04:25:08.295+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58842:172.31.5.112:12000] 2023-10-21T04:25:08.306+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58844:172.31.5.112:12000] 2023-10-21T04:25:08.309+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.309+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.316+05:30 [Info] switched currmeta from 82 -> 83 force true 2023-10-21T04:25:08.354+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.354+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.359+05:30 [Info] switched currmeta from 83 -> 84 force true 2023-10-21T04:25:08.374+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58846:172.31.5.112:12000] 2023-10-21T04:25:08.411+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58848:172.31.5.112:12000] 2023-10-21T04:25:08.427+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.427+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.433+05:30 [Info] switched currmeta from 84 -> 85 force true 2023-10-21T04:25:08.449+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58852:172.31.5.112:12000] 2023-10-21T04:25:08.452+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:58854:172.31.5.112:12000] 2023-10-21T04:25:08.552+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2296526085582506469:INDEX_STATE_RECOVERED 10979931881244616620:INDEX_STATE_RECOVERED 11460826864060385936:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:08.553+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 13856704970496601958, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:08.553+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 13856704970496601958, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:08.572+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 16733302726553098964,realInstId: 0, indexDefn DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:08.574+05:30 [Info] Indexer::handleRecoverIndex InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:08.578+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 4.089766ms 2023-10-21T04:25:08.578+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 4.385198ms 2023-10-21T04:25:08.581+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16733302726553098964 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:08.581+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 16733302726553098964, indexDefn DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:08.582+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 28.958289ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:08.586+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16733302726553098964 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:08.586+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:08.587+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:08.588+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:08.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 194 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:08.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:08.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:08.601+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:08.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex started 2023-10-21T04:25:08.601+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16733302726553098964 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:08.601+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16733302726553098964 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:08.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:08.601+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:08.601+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:08.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex stopped 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:08.601+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:08.604+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.602+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:08.604+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.608+05:30 [Info] switched currmeta from 85 -> 86 force true 2023-10-21T04:25:08.609+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:08.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:08.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex closed 2023-10-21T04:25:08.609+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:08.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:25:08.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 194 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:25:08.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:08.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index destroyed 2023-10-21T04:25:08.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:08.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:08.615+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:08.615+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:08.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 228 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:08.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:08.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:08.619+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:08.619+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:08.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex started 2023-10-21T04:25:08.619+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16733302726553098964 PartitionId 0 Warmup took 32.859633ms 2023-10-21T04:25:08.619+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:08.619+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16733302726553098964 partitionId 0 WriterThreads 4 2023-10-21T04:25:08.619+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:08.619+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16733302726553098964 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index Index: 16733302726553098964 Partition: 0 2023-10-21T04:25:08.619+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 16733302726553098964, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:25:08.619+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 16733302726553098964, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:25:08.619+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:08.620+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 16733302726553098964 2023-10-21T04:25:08.620+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16733302726553098964 Name: #primary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:08.620+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:08.620+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:08.620+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16733302726553098964 Name: #primary_defer Keyspace: bucket_%2:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:08.621+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:08.621+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 16733302726553098964 Partitions [0] 2023-10-21T04:25:08.621+05:30 [Info] StorageMgr::openSnapshot IndexInst:16733302726553098964 Partition:0 No Snapshot Found. 2023-10-21T04:25:08.621+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 16733302726553098964 Adding Nil Snapshot. 2023-10-21T04:25:08.627+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.888012ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:08.627+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 71.191µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:08.658+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.658+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.661+05:30 [Info] switched currmeta from 86 -> 87 force true 2023-10-21T04:25:08.714+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.714+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.719+05:30 [Info] switched currmeta from 87 -> 88 force true 2023-10-21T04:25:08.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:08.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:08.766+05:30 [Info] switched currmeta from 88 -> 89 force true 2023-10-21T04:25:09.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8525434678892561, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:25:09.100+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[9912003609070511414:INDEX_STATE_RECOVERED 12004664282277897596:INDEX_STATE_RECOVERED 15667952562388529486:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:09.101+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary_defer, defnId: 11619165464560280276, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:09.101+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary_defer, defnId: 11619165464560280276, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:25:09.116+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 14271070195512901425,realInstId: 0, indexDefn DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:09.117+05:30 [Info] Indexer::handleRecoverIndex InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:09.117+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 161.828µs 2023-10-21T04:25:09.117+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 204.336µs 2023-10-21T04:25:09.117+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 14271070195512901425, indexDefn DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:09.117+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14271070195512901425 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:09.117+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 16.013807ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:09.117+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 14271070195512901425 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:09.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:09.117+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:25:09.118+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 273066 recovery UsedSpace 8236 2023-10-21T04:25:09.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 194 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:09.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:09.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:09.129+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:09.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex started 2023-10-21T04:25:09.129+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:09.129+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Unable to find recovery point. Resetting store .. 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:09.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:09.129+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:09.129+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:09.129+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:09.129+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:09.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex stopped 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:09.129+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:09.130+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:09.138+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:09.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:09.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex closed 2023-10-21T04:25:09.139+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:09.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:25:09.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 194 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:25:09.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:09.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index destroyed 2023-10-21T04:25:09.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:09.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:09.147+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:25:09.148+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:25:09.155+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.155+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 228 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:09.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:09.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:09.161+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:09.161+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:09.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex started 2023-10-21T04:25:09.161+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:09.161+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Warmup took 43.747117ms 2023-10-21T04:25:09.161+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:09.161+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14271070195512901425 partitionId 0 WriterThreads 4 2023-10-21T04:25:09.161+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Stopped Billing WUs 2023-10-21T04:25:09.161+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14271070195512901425 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index Index: 14271070195512901425 Partition: 0 2023-10-21T04:25:09.161+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 14271070195512901425, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:25:09.161+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 14271070195512901425, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:25:09.161+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:09.162+05:30 [Info] switched currmeta from 89 -> 90 force true 2023-10-21T04:25:09.162+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14271070195512901425 2023-10-21T04:25:09.162+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14271070195512901425 Name: #primary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 0 2023-10-21T04:25:09.168+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:09.169+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:09.169+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14271070195512901425 Name: #primary_defer Keyspace: bucket_1:_default:_default State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 0 2023-10-21T04:25:09.169+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:09.169+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14271070195512901425 Partitions [0] 2023-10-21T04:25:09.169+05:30 [Info] StorageMgr::openSnapshot IndexInst:14271070195512901425 Partition:0 No Snapshot Found. 2023-10-21T04:25:09.169+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 14271070195512901425 Adding Nil Snapshot. 2023-10-21T04:25:09.182+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.259903ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:09.182+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 86.62µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:09.248+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.248+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.251+05:30 [Info] switched currmeta from 90 -> 91 force true 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.296+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:25:09.375+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.376+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.382+05:30 [Info] switched currmeta from 91 -> 92 force true 2023-10-21T04:25:09.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.464+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.468+05:30 [Info] switched currmeta from 92 -> 94 force true 2023-10-21T04:25:09.477+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.552+05:30 [Info] switched currmeta from 94 -> 97 force true 2023-10-21T04:25:09.571+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.571+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.574+05:30 [Info] switched currmeta from 97 -> 97 force true 2023-10-21T04:25:09.587+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[16733302726553098964:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:25:09.587+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00731a720 2023-10-21T04:25:09.587+05:30 [Info] handleUpdateIndexRState: Index instance 16733302726553098964 rstate moved to ACTIVE 2023-10-21T04:25:09.596+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.220114ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:09.596+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [16162012140118277065 7525085113233084253 8857269605176042330] belonging to collectionId: 0, ttid: ShardToken34:c1:80:2c:eb:79:82:c5. Initiating build 2023-10-21T04:25:09.637+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.637+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.640+05:30 [Info] switched currmeta from 97 -> 100 force true 2023-10-21T04:25:09.645+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.645+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.650+05:30 [Info] switched currmeta from 100 -> 100 force true 2023-10-21T04:25:09.660+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [11460826864060385936 10979931881244616620 2296526085582506469] &{2} 2023-10-21T04:25:09.667+05:30 [Info] Indexer::handleBuildRecoveredIndexes [11460826864060385936 10979931881244616620 2296526085582506469] 2023-10-21T04:25:09.668+05:30 [Info] bulkUpdateRState: Index instance 11460826864060385936 rstate moved to PENDING 2023-10-21T04:25:09.668+05:30 [Info] bulkUpdateRState: Index instance 10979931881244616620 rstate moved to PENDING 2023-10-21T04:25:09.668+05:30 [Info] bulkUpdateRState: Index instance 2296526085582506469 rstate moved to PENDING 2023-10-21T04:25:09.670+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.671+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.671+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [11460826864060385936 10979931881244616620 2296526085582506469] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:25:09.671+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:09.671+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:09.671+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:09.679+05:30 [Info] switched currmeta from 100 -> 101 force true 2023-10-21T04:25:09.710+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.710+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.717+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2 2023-10-21T04:25:09.719+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:09.721+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:25:09.722+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:25:09.720+05:30 [Info] switched currmeta from 101 -> 102 force true 2023-10-21T04:25:09.730+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:09.731+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:09.731+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:25:09.731+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:25:09.731+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2 SessionId 10601 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} 2023-10-21T04:25:09.731+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_%2. OSO not supported for default scope/collection. 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:25:09.731+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream INIT_STREAM 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:1]] 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 11460826864060385936 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:2]] 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 10979931881244616620 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:3]] 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 2296526085582506469 stream INIT_STREAM keyspaceId bucket_%2 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:09.731+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2 2023-10-21T04:25:09.732+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [0=56,1=66,2=52,3=47,4=60,5=57,6=63,7=54,8=58,9=66,10=52,11=58,12=57,13=56,14=65,15=69,16=58,17=48,18=48,19=48,20=54,21=56,22=57,23=55,24=54,25=63,26=61,27=57,28=56,29=51,30=48,31=62,32=69,33=58,34=63,35=53,36=54,37=61,38=55,39=52,40=72,41=57,42=57,43=68,44=60,45=66,46=58,47=57,48=58,49=57,50=68,51=53,52=56,53=63,54=62,55=69,56=63,57=65,58=48,59=65,60=48,61=41,62=60,63=75] IndexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 10601 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} NumVBuckets: 64 2023-10-21T04:25:09.732+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:25:09.733+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:25:09.733+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:09.733+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 66.081542ms 2023-10-21T04:25:09.734+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 66.129927ms 2023-10-21T04:25:09.734+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [11460826864060385936 10979931881244616620 2296526085582506469] 2023-10-21T04:25:09.734+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 136.160339ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:09.737+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[2296526085582506469:true 10979931881244616620:true 11460826864060385936:true] 2023-10-21T04:25:09.738+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_%2 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:25:09.745+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2 Check pending stream merge. 2023-10-21T04:25:09.766+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 32.334576ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:09.780+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.781+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.782+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.765891ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:09.789+05:30 [Info] switched currmeta from 102 -> 107 force true 2023-10-21T04:25:09.801+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.801+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.809+05:30 [Info] switched currmeta from 107 -> 107 force true 2023-10-21T04:25:09.813+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:42064" doAuth() ... 2023-10-21T04:25:09.814+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:42064" auth successful 2023-10-21T04:25:09.814+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:42064" +1 2023-10-21T04:25:09.817+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 34.496891ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:09.823+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.824+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.830+05:30 [Info] switched currmeta from 107 -> 108 force true 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 96.56191243621288 InstId: 12004664282277897596 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 96.56191243621288 InstId: 9912003609070511414 RealInstId: 9912003609070511414 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 96.56191243621288 InstId: 15667952562388529486 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 96.56191243621288 InstId: 7609950505007110692 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 96.56191243621288 InstId: 3593050551190633737 RealInstId: 3593050551190633737 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 96.56191243621288 InstId: 10590902705185414921 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 12882995014846270330 RealInstId: 12882995014846270330 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 2281164589865190206 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 620420137039842024 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 11460826864060385936 RealInstId: 11460826864060385936 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 10979931881244616620 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 2296526085582506469 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:09.920+05:30 [Info] ShardRebalancer::computeProgress 0.7029889490042593 2023-10-21T04:25:09.973+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:09.978+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:09.982+05:30 [Info] switched currmeta from 108 -> 111 force true 2023-10-21T04:25:10.013+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.013+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.036+05:30 [Info] switched currmeta from 111 -> 111 force true 2023-10-21T04:25:10.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8166025863771471, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:25:10.129+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[14271070195512901425:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:25:10.130+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc004d7c480 2023-10-21T04:25:10.130+05:30 [Info] handleUpdateIndexRState: Index instance 14271070195512901425 rstate moved to ACTIVE 2023-10-21T04:25:10.143+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.069599ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.143+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [1252359852432998131 573521267164877540 12338036531800130039] belonging to collectionId: 0, ttid: ShardToken12:9b:ba:a8:e5:6a:47:bd. Initiating build 2023-10-21T04:25:10.163+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.163+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.169+05:30 [Info] switched currmeta from 111 -> 112 force true 2023-10-21T04:25:10.193+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.193+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.198+05:30 [Info] switched currmeta from 112 -> 113 force true 2023-10-21T04:25:10.258+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.263+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.270+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [12004664282277897596 9912003609070511414 15667952562388529486] &{2} 2023-10-21T04:25:10.270+05:30 [Info] Indexer::handleBuildRecoveredIndexes [12004664282277897596 9912003609070511414 15667952562388529486] 2023-10-21T04:25:10.270+05:30 [Info] bulkUpdateRState: Index instance 12004664282277897596 rstate moved to PENDING 2023-10-21T04:25:10.270+05:30 [Info] bulkUpdateRState: Index instance 9912003609070511414 rstate moved to PENDING 2023-10-21T04:25:10.270+05:30 [Info] bulkUpdateRState: Index instance 15667952562388529486 rstate moved to PENDING 2023-10-21T04:25:10.270+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [12004664282277897596 9912003609070511414 15667952562388529486] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:25:10.270+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:10.270+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:10.271+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:10.323+05:30 [Info] switched currmeta from 113 -> 115 force true 2023-10-21T04:25:10.326+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1 2023-10-21T04:25:10.326+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:10.326+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:10.326+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:10.326+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:25:10.326+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:25:10.326+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:10.326+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:10.326+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0004a8540 }] 2023-10-21T04:25:10.326+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:10.327+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:10.327+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:25:10.327+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1 SessionId 28676 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:25:10.327+05:30 [Info] Indexer::useOSOForKeyspaceStream INIT_STREAM bucket_1. OSO not supported for default scope/collection. 2023-10-21T04:25:10.327+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:10.327+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:10.327+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream INIT_STREAM 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:3 bucket_1:1]] 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 9912003609070511414 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:3 bucket_1:2]] 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15667952562388529486 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:3 bucket_1:3]] 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 12004664282277897596 stream INIT_STREAM keyspaceId bucket_1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:10.327+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1 2023-10-21T04:25:10.329+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.329+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.327+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [0=56,1=60,2=54,3=35,4=58,5=53,6=62,7=63,8=45,9=55,10=56,11=51,12=42,13=47,14=75,15=51,16=60,17=51,18=63,19=48,20=50,21=48,22=51,23=59,24=59,25=72,26=52,27=50,28=58,29=57,30=58,31=67,32=60,33=67,34=60,35=47,36=58,37=54,38=60,39=68,40=62,41=58,42=46,43=55,44=67,45=69,46=54,47=53,48=60,49=34,50=64,51=54,52=56,53=56,54=55,55=72,56=54,57=55,58=54,59=53,60=58,61=53,62=57,63=56] IndexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 28676 CollectionId: 0 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:25:10.331+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0004a8540 } bucket_1:{0xc000508780 }] 2023-10-21T04:25:10.332+05:30 [Info] switched currmeta from 115 -> 115 force true 2023-10-21T04:25:10.332+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:10.332+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 62.357076ms 2023-10-21T04:25:10.332+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 62.398431ms 2023-10-21T04:25:10.332+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [12004664282277897596 9912003609070511414 15667952562388529486] 2023-10-21T04:25:10.332+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 183.818978ms len(expediates) 3 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.333+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[9912003609070511414:true 12004664282277897596:true 15667952562388529486:true] 2023-10-21T04:25:10.334+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:25:10.340+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1 Check pending stream merge. 2023-10-21T04:25:10.349+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.522861ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.369+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.747441ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.391+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.208495ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.407+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.408+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.416+05:30 [Info] switched currmeta from 115 -> 118 force true 2023-10-21T04:25:10.465+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.465+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.469+05:30 [Info] switched currmeta from 118 -> 118 force true 2023-10-21T04:25:10.610+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.610+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.616+05:30 [Info] switched currmeta from 118 -> 119 force true 2023-10-21T04:25:10.637+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.638+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.644+05:30 [Info] switched currmeta from 119 -> 120 force true 2023-10-21T04:25:10.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.664+05:30 [Info] switched currmeta from 120 -> 121 force true 2023-10-21T04:25:10.697+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 0 263268055970459 56 10601 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:10.697+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 1 176639880977228 66 10601 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:10.697+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 2 35403626585217 52 10601 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:10.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 3 215749515705641 47 10601 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:25:10.701+05:30 [Info] NewTsVbuuidCached: bucket_%2, 64 TsVbuuidPoolMap is updated from: map[] to: map[64:0xc006c8c810] 2023-10-21T04:25:10.703+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 6 261493429196201 63 10601 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:10.703+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 5 3874519919278 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.703+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 4 159113185399719 60 10601 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:10.706+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 7 170288557775498 54 10601 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:10.709+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 9 236398693961229 66 10601 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:10.709+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 8 264975169139761 58 10601 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:10.711+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 11 234758907181738 58 10601 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:10.712+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 10 206533515052028 52 10601 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:10.716+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 12 116104661063478 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.716+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 13 102350060311343 56 10601 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:10.718+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 14 140236896721046 65 10601 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:10.722+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 15 162158720181435 69 10601 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:10.725+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 17 98820809009123 48 10601 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:10.726+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 16 200752780087280 58 10601 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:10.730+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 20 92956357968546 54 10601 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:10.730+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 18 207134287529256 48 10601 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:10.730+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 19 208836052671833 48 10601 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:10.736+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 22 192215088742243 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.736+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 21 27979933591532 56 10601 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:10.740+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 23 154934968850942 55 10601 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:10.741+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 24 271667630317209 54 10601 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:10.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 26 120989102157727 61 10601 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:25:10.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 25 168740244208559 63 10601 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:10.748+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 27 5715487545729 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.750+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 28 236953356346174 56 10601 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:10.754+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 29 130231725986856 51 10601 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:10.754+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 30 60545336618717 48 10601 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:10.756+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 31 73432771436458 62 10601 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:10.760+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 34 51776324008509 63 10601 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:10.760+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 33 278426329668882 58 10601 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:10.760+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 32 67362620867705 69 10601 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:10.766+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 36 43594649874772 54 10601 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:10.766+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 35 24368176313355 53 10601 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:10.766+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 37 83647976044922 61 10601 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:25:10.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 39 100659545510465 52 10601 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:10.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 38 181267443431019 55 10601 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:10.774+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 41 116785907755771 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.774+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 40 153478150002150 72 10601 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:25:10.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 45 49796766757346 66 10601 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:10.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 42 45479338375128 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 43 249919763471169 68 10601 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:25:10.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 44 166675095997741 60 10601 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:10.789+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 47 149022976193937 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.789+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 46 124194574848641 58 10601 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:10.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 48 19274425379955 58 10601 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:10.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 49 184487117398647 57 10601 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:10.799+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 50 199728167534433 68 10601 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:25:10.799+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 51 196777743741606 53 10601 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:10.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 55 61798873988101 69 10601 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:10.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 52 158076890855634 56 10601 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:10.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 53 138935694816848 63 10601 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:10.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 54 24856128127173 62 10601 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:10.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 56 29071415952852 63 10601 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:10.814+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 57 191719959781498 65 10601 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:10.817+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 59 140942236111856 65 10601 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:10.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 58 103336286678362 48 10601 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:10.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 60 56626109228904 48 10601 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:10.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 61 119871837460216 41 10601 172.31.5.112:9000. HWT [41-41,41]. 2023-10-21T04:25:10.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 63 252537041863196 75 10601 172.31.5.112:9000. HWT [75-75,75]. 2023-10-21T04:25:10.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2 62 177105814270799 60 10601 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:10.827+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_%2 InstanceIds [11460826864060385936 10979931881244616620 2296526085582506469] 2023-10-21T04:25:10.828+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2 Cid 0 SessionId 10601 2023-10-21T04:25:10.828+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 10601 2023-10-21T04:25:10.828+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:25:10.828+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2 2023-10-21T04:25:10.829+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 11460826864060385936 INDEX_STATE_CATCHUP 2023-10-21T04:25:10.829+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 10979931881244616620 INDEX_STATE_CATCHUP 2023-10-21T04:25:10.829+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2 2296526085582506469 INDEX_STATE_CATCHUP 2023-10-21T04:25:10.829+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 11460826864060385936 true 2023-10-21T04:25:10.829+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 10979931881244616620 true 2023-10-21T04:25:10.829+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2 2296526085582506469 true 2023-10-21T04:25:10.829+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 11460826864060385936 Stream: INIT_STREAM KeyspaceId: bucket_%2 Session: 10601 BuildTS: [0=56,1=66,2=52,3=47,4=60,5=57,6=63,7=54,8=58,9=66,10=52,11=58,12=57,13=56,14=65,15=69,16=58,17=48,18=48,19=48,20=54,21=56,22=57,23=55,24=54,25=63,26=61,27=57,28=56,29=51,30=48,31=62,32=69,33=58,34=63,35=53,36=54,37=61,38=55,39=52,40=72,41=57,42=57,43=68,44=60,45=66,46=58,47=57,48=58,49=57,50=68,51=53,52=56,53=63,54=62,55=69,56=63,57=65,58=48,59=65,60=48,61=41,62=60,63=75] 2023-10-21T04:25:10.830+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} , fetchKVSeq false 2023-10-21T04:25:10.830+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:25:10.830+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:25:10.830+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2 SessionId 10601 STREAM_ACTIVE 2023-10-21T04:25:10.830+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2 Stream: INIT_STREAM SessionId: 10601 2023-10-21T04:25:10.830+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2 10601 2023-10-21T04:25:10.830+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:25:10.835+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:25:10.836+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:10.836+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:25:10.836+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:10.837+05:30 [Info] send progress stats to clients 2023-10-21T04:25:10.837+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 10601 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:10.837+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 11460826864060385936 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:25:10.837+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 10979931881244616620 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:25:10.837+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 2296526085582506469 from stream INIT_STREAM keyspaceId bucket_%2 2023-10-21T04:25:10.837+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2 2023-10-21T04:25:10.837+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream INIT_STREAM 2023-10-21T04:25:10.837+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 0 New 0 2023-10-21T04:25:10.837+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 10601 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:10.837+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc000508780 }] 2023-10-21T04:25:10.837+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:10.837+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:10.837+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:10.837+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:10.838+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2 2023-10-21T04:25:10.838+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_%2] 2023-10-21T04:25:10.846+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.940346ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.852+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.440391ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.859+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.212538ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:10.874+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.874+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.876+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]} {Rev:[0 0 0 0 0 0 0 2] ID:rebalance/1972139801924f1fdf1188af4e7137b9 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029889490042593 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:10.876+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:10.877+05:30 [Info] switched currmeta from 121 -> 124 force true 2023-10-21T04:25:10.878+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:25:10.880+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:25:10.902+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:10.902+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:10.914+05:30 [Info] switched currmeta from 124 -> 124 force true 2023-10-21T04:25:11.418+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:11.418+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:11.424+05:30 [Info] switched currmeta from 124 -> 125 force true 2023-10-21T04:25:11.432+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 1 178840478062605 60 28676 172.31.5.112:9000. HWT [59-60,60]. 2023-10-21T04:25:11.432+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 0 186517626268899 56 28676 172.31.5.112:9000. HWT [55-56,56]. 2023-10-21T04:25:11.477+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 3 215374859564337 35 28676 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:25:11.477+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 5 39552632604610 53 28676 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:25:11.500+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 4 28605411869539 58 28676 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:11.515+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:11.518+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:11.521+05:30 [Info] switched currmeta from 125 -> 127 force true 2023-10-21T04:25:11.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 8 50961953886801 45 28676 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:25:11.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 9 225280996447309 55 28676 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:25:11.523+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 7 191903564840858 63 28676 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:11.534+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:11.534+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:11.547+05:30 [Info] switched currmeta from 127 -> 127 force true 2023-10-21T04:25:11.556+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 11 66881023717063 51 28676 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:11.556+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 12 4429827396609 42 28676 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:25:11.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 23 216462437971699 59 28676 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:25:11.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 28 102578907139247 58 28676 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:11.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 60 75566139929862 58 28676 172.31.5.112:9000. HWT [56-58,58]. 2023-10-21T04:25:11.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 29 45957426777107 57 28676 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:11.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 27 184056281482149 50 28676 172.31.5.112:9000. HWT [49-50,50]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 30 99433130332778 58 28676 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 62 10527297481171 57 28676 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 25 280874278672777 72 28676 172.31.5.112:9000. HWT [71-72,72]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 26 219132834127088 52 28676 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 2 148075163102199 54 28676 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 6 242327631330054 62 28676 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 15 127185710837163 51 28676 172.31.5.112:9000. HWT [50-51,51]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 20 230239359684242 50 28676 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 17 55290066231400 51 28676 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 10 274410200699189 56 28676 172.31.5.112:9000. HWT [54-56,56]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 61 106932309485550 53 28676 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 13 183148377680315 47 28676 172.31.5.112:9000. HWT [46-47,47]. 2023-10-21T04:25:11.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 19 210306088158507 48 28676 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:11.627+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 16 257031483891250 60 28676 172.31.5.112:9000. HWT [55-60,60]. 2023-10-21T04:25:11.627+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 14 226814108379280 75 28676 172.31.5.112:9000. HWT [75-75,75]. 2023-10-21T04:25:11.627+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 44 158545118983293 67 28676 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:25:11.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 54 279362078165491 55 28676 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:11.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 22 260036488774725 51 28676 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:11.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 40 5915864305121 62 28676 172.31.5.112:9000. HWT [60-62,62]. 2023-10-21T04:25:11.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 21 209771427122473 48 28676 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:25:11.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 24 160961818008256 59 28676 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:25:11.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 18 242578104468179 63 28676 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:11.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 58 16044440367455 54 28676 172.31.5.112:9000. HWT [53-54,54]. 2023-10-21T04:25:11.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 35 70213628416850 47 28676 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:25:11.648+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 39 222555662236724 68 28676 172.31.5.112:9000. HWT [67-68,68]. 2023-10-21T04:25:11.652+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 45 12301831552869 69 28676 172.31.5.112:9000. HWT [68-69,69]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 52 188718367400378 56 28676 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 51 200436537417141 54 28676 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 49 215467668600630 34 28676 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 48 227378777358533 60 28676 172.31.5.112:9000. HWT [57-60,60]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 43 146267938201125 55 28676 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 59 125009072873642 53 28676 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 56 264835102027079 54 28676 172.31.5.112:9000. HWT [52-54,54]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 46 242994859770444 54 28676 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 31 217381278987664 67 28676 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 55 170056388266420 72 28676 172.31.5.112:9000. HWT [70-72,72]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 53 204867593700407 56 28676 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 50 207367350302991 64 28676 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 32 82469662608686 60 28676 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:11.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1 47 271074273345339 53 28676 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:11.657+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_1 InstanceIds [11460826864060385936 10979931881244616620 2296526085582506469 12004664282277897596 9912003609070511414 15667952562388529486] 2023-10-21T04:25:11.657+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1 Cid 0 SessionId 28676 2023-10-21T04:25:11.657+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 28676 2023-10-21T04:25:11.657+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 9912003609070511414 INDEX_STATE_CATCHUP 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 15667952562388529486 INDEX_STATE_CATCHUP 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1 12004664282277897596 INDEX_STATE_CATCHUP 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 15667952562388529486 true 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 12004664282277897596 true 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1 9912003609070511414 true 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 9912003609070511414 Stream: INIT_STREAM KeyspaceId: bucket_1 Session: 28676 BuildTS: [0=56,1=60,2=54,3=35,4=58,5=53,6=62,7=63,8=45,9=55,10=56,11=51,12=42,13=47,14=75,15=51,16=60,17=51,18=63,19=48,20=50,21=48,22=51,23=59,24=59,25=72,26=52,27=50,28=58,29=57,30=58,31=67,32=60,33=67,34=60,35=47,36=58,37=54,38=60,39=68,40=62,41=58,42=46,43=55,44=67,45=69,46=54,47=53,48=60,49=34,50=64,51=54,52=56,53=56,54=55,55=72,56=54,57=55,58=54,59=53,60=58,61=53,62=57,63=56] 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:25:11.658+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1 2023-10-21T04:25:11.658+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1 SessionId 28676 STREAM_ACTIVE 2023-10-21T04:25:11.658+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1 Stream: INIT_STREAM SessionId: 28676 2023-10-21T04:25:11.658+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1 28676 2023-10-21T04:25:11.658+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:11.659+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:11.659+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:11.659+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:11.659+05:30 [Info] send progress stats to clients 2023-10-21T04:25:11.659+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 28676 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:11.659+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 12004664282277897596 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:25:11.660+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 9912003609070511414 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:25:11.660+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15667952562388529486 from stream INIT_STREAM keyspaceId bucket_1 2023-10-21T04:25:11.660+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1 2023-10-21T04:25:11.660+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream INIT_STREAM 2023-10-21T04:25:11.660+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:11.660+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 28676 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:11.660+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:25:11.660+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 35:0 39:0 40:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 58:0 59:0 60:0 61:0 62:0] 2023-10-21T04:25:11.660+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 35:0 39:0 40:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 58:0 59:0 60:0 61:0 62:0] 2023-10-21T04:25:11.660+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:25:11.660+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:25:11.662+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:25:11.662+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:42064" closed ! 2023-10-21T04:25:11.662+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:25:11.663+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_1] 2023-10-21T04:25:11.663+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1 2023-10-21T04:25:11.666+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:42064" exit: dataport.workerKilled 2023-10-21T04:25:11.667+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.533723ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:11.677+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.841631ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:11.682+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.206796ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:11.721+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:11.721+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:11.732+05:30 [Info] switched currmeta from 127 -> 130 force true 2023-10-21T04:25:11.757+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:11.757+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:11.773+05:30 [Info] switched currmeta from 130 -> 130 force true 2023-10-21T04:25:11.790+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_1]. Err projector.topicMissing 2023-10-21T04:25:11.790+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:25:11.790+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1] Treating projector.topicMissing As Success 2023-10-21T04:25:11.790+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1 SessionId 28676 2023-10-21T04:25:11.790+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1 2023-10-21T04:25:11.790+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:25:11.790+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:11.790+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:25:11.790+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:25:11.790+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:11.790+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:11.790+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:25:11.790+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:25:11.790+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:25:11.790+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 51229 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:25:11.791+05:30 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2023-10-21T04:25:11.791+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:25:11.791+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:25:11.791+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:25:11.791+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:25:11.791+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:25:11.791+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 51229 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:25:11.792+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:25:11.792+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:25:11.794+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_%2] 2023-10-21T04:25:11.795+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2 SessionId 10601 2023-10-21T04:25:11.795+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2 2023-10-21T04:25:11.795+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:25:11.795+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:11.795+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:25:11.795+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:11.795+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:11.795+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc009280000 }] 2023-10-21T04:25:11.795+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:11.795+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:11.795+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:25:11.795+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 35695 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} 2023-10-21T04:25:11.795+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:11.795+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:11.795+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1 bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[]] 2023-10-21T04:25:11.795+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:25:11.795+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 35695 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} NumVBuckets: 64 2023-10-21T04:25:11.796+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0068e1080 } bucket_1:{0xc009280000 }] 2023-10-21T04:25:11.797+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:25:11.801+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_%2 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:25:11.823+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:37978" doAuth() ... 2023-10-21T04:25:11.823+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:37978" auth successful 2023-10-21T04:25:11.824+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:37978" +1 2023-10-21T04:25:11.832+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:25:11.999+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:11.999+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.020+05:30 [Info] switched currmeta from 130 -> 132 force true 2023-10-21T04:25:12.072+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.075+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.086+05:30 [Info] switched currmeta from 132 -> 133 force true 2023-10-21T04:25:12.087+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:25:12.087+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:25:12.087+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:25:12.088+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842319179728392, Old rollbacktime: 0 2023-10-21T04:25:12.094+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9806955144518954, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:25:12.095+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.097+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.104+05:30 [Info] switched currmeta from 133 -> 133 force true 2023-10-21T04:25:12.121+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.121+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.129+05:30 [Info] switched currmeta from 133 -> 134 force true 2023-10-21T04:25:12.174+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.174+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.190+05:30 [Info] switched currmeta from 134 -> 135 force true 2023-10-21T04:25:12.344+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:12.358+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009256360 2023-10-21T04:25:12.358+05:30 [Info] handleUpdateIndexRState: Index instance 12004664282277897596 rstate moved to ACTIVE 2023-10-21T04:25:12.369+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.370+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.385+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.442087ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.387+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009256480 2023-10-21T04:25:12.387+05:30 [Info] handleUpdateIndexRState: Index instance 15667952562388529486 rstate moved to ACTIVE 2023-10-21T04:25:12.387+05:30 [Info] Indexer::handleMergePartition Source 6611915605599465689 Target 9912003609070511414 2023-10-21T04:25:12.387+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9912003609070511414 Defn: DefnId: 573521267164877540 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006c00480 2023-10-21T04:25:12.388+05:30 [Info] MergePartition: sent async request to update index instance 9912003609070511414 rstate moved to ACTIVE 2023-10-21T04:25:12.398+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.588423ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.399+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 5004576769623010783. Retrying... 2023-10-21T04:25:12.399+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[9912003609070511414:true 12004664282277897596:true 15667952562388529486:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:25:12.425+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.018896ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.426+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 6611915605599465689, realInst: 9912003609070511414 2023-10-21T04:25:12.433+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 12833670934079048728, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:12.433+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 12833670934079048728, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:12.461+05:30 [Info] switched currmeta from 135 -> 138 force true 2023-10-21T04:25:12.461+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 1697842319179728392 2023-10-21T04:25:12.461+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10590902705185414921:[0]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 0 {Index inst : [partitions]}: map[3593050551190633737:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 1697842319179728392 2023-10-21T04:25:12.461+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3593050551190633737:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 1697842319179728392 {Index inst : [partitions]}: map[7609950505007110692:[0] 10590902705185414921:[0]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 0 2023-10-21T04:25:12.465+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 5004576769623010783,realInstId: 0, indexDefn DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:12.465+05:30 [Info] Indexer::handleRecoverIndex InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:12.465+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 145.275µs 2023-10-21T04:25:12.465+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 192.986µs 2023-10-21T04:25:12.465+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 5004576769623010783, indexDefn DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:12.465+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 32.897368ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.465+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 5004576769623010783 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[2864853750198994630 12177293282243154215]] 2023-10-21T04:25:12.466+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 5004576769623010783 PartitionId: 0 using shardIds: [2864853750198994630 12177293282243154215], rebalance running: true 2023-10-21T04:25:12.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery) 2023-10-21T04:25:12.466+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 64 2023-10-21T04:25:12.466+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: RP version: 29 numRPs: 2 rp [0] sn: 1800009 count: 26 metaSz: 2548 rp [1] sn: 1800012 count: 24 metaSz: 2548 2023-10-21T04:25:12.467+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: doInit: data UsedSpace 436466 recovery UsedSpace 257012 2023-10-21T04:25:12.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:25:12.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630 2023-10-21T04:25:12.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data 2023-10-21T04:25:12.483+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:25:12.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex started 2023-10-21T04:25:12.483+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5004576769623010783 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:12.483+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:12.483+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:12.506+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 5004576769623010783 PartitionId 0 Warmup took 39.915429ms 2023-10-21T04:25:12.506+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 5004576769623010783 partitionId 0 WriterThreads 4 2023-10-21T04:25:12.506+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 5004576769623010783 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index Index: 5004576769623010783 Partition: 0 2023-10-21T04:25:12.506+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 5004576769623010783, shardIdMap: map[0:[2864853750198994630]] err: 2023-10-21T04:25:12.506+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 5004576769623010783, partnShardIdMap: map[0:[2864853750198994630]], err: 2023-10-21T04:25:12.506+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:12.507+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 5004576769623010783 2023-10-21T04:25:12.507+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:12.507+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:12.507+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:12.507+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:12.507+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:12.508+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 5004576769623010783 Partitions [0] 2023-10-21T04:25:12.508+05:30 [Info] StorageMgr::openSnapshot IndexInst:5004576769623010783 Partition:0 Attempting to open snapshot (SnapshotInfo: count:24 committed:false) 2023-10-21T04:25:12.515+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.929709ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.516+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 98.138µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:12.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.517+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.530+05:30 [Info] switched currmeta from 138 -> 143 force true 2023-10-21T04:25:12.560+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.560+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.576+05:30 [Info] switched currmeta from 143 -> 144 force true 2023-10-21T04:25:12.627+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.627+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.637+05:30 [Info] switched currmeta from 144 -> 145 force true 2023-10-21T04:25:12.637+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[620420137039842024:[0]]. New rollbacktime: 1697842319286825244, Old rollbacktime: 0 2023-10-21T04:25:12.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.665+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.662+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:25:12.669+05:30 [Info] switched currmeta from 145 -> 145 force true 2023-10-21T04:25:12.669+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9912003609070511414:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:12.669+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9912003609070511414:[4 5 6 7 8 1 2 3] 12004664282277897596:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:12.669+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9912003609070511414:[4 5 6 7 8 1 2 3] 12004664282277897596:[0] 15667952562388529486:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:12.683+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 186517626268899 56 51229 172.31.5.112:9000. HWT [55-56,56]. 2023-10-21T04:25:12.686+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 178840478062605 60 51229 172.31.5.112:9000. HWT [59-60,60]. 2023-10-21T04:25:12.687+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 148075163102199 54 51229 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:12.689+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 215374859564337 35 51229 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:25:12.692+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 39552632604610 53 51229 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:25:12.692+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 242327631330054 62 51229 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:12.695+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 191903564840858 63 51229 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:12.699+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 225280996447309 55 51229 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:25:12.699+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 50961953886801 45 51229 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:25:12.699+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 28605411869539 58 51229 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:12.701+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 66881023717063 51 51229 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:12.701+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 274410200699189 56 51229 172.31.5.112:9000. HWT [54-56,56]. 2023-10-21T04:25:12.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 127185710837163 51 51229 172.31.5.112:9000. HWT [50-51,51]. 2023-10-21T04:25:12.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 183148377680315 47 51229 172.31.5.112:9000. HWT [46-47,47]. 2023-10-21T04:25:12.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 226814108379280 75 51229 172.31.5.112:9000. HWT [75-75,75]. 2023-10-21T04:25:12.709+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 4429827396609 42 51229 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:25:12.714+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 257031483891250 60 51229 172.31.5.112:9000. HWT [55-60,60]. 2023-10-21T04:25:12.714+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 55290066231400 51 51229 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:12.717+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 242578104468179 63 51229 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:12.719+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 210306088158507 48 51229 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:12.724+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 230239359684242 50 51229 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:25:12.724+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 209771427122473 48 51229 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:25:12.730+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 216462437971699 59 51229 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:25:12.730+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 260036488774725 51 51229 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:12.732+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 280874278672777 72 51229 172.31.5.112:9000. HWT [71-72,72]. 2023-10-21T04:25:12.739+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 219132834127088 52 51229 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:12.739+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 184056281482149 50 51229 172.31.5.112:9000. HWT [49-50,50]. 2023-10-21T04:25:12.743+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 45957426777107 57 51229 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:12.743+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 102578907139247 58 51229 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:12.743+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 160961818008256 59 51229 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:25:12.744+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 99433130332778 58 51229 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:12.748+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 217381278987664 67 51229 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:25:12.750+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 136958029391999 67 51229 172.31.5.112:9000. HWT [65-67,67]. 2023-10-21T04:25:12.751+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 70213628416850 47 51229 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:25:12.751+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 28417669987133 60 51229 172.31.5.112:9000. HWT [58-60,60]. 2023-10-21T04:25:12.754+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 253494845579900 54 51229 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:12.755+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 188107826286720 60 51229 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:12.758+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 222555662236724 68 51229 172.31.5.112:9000. HWT [67-68,68]. 2023-10-21T04:25:12.761+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 259381390638966 58 51229 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:12.762+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 82469662608686 60 51229 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:12.762+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 253845164956592 58 51229 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:12.762+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 252931432434966 46 51229 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:25:12.763+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 5915864305121 62 51229 172.31.5.112:9000. HWT [60-62,62]. 2023-10-21T04:25:12.763+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 146267938201125 55 51229 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:12.767+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 158545118983293 67 51229 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:25:12.768+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 12301831552869 69 51229 172.31.5.112:9000. HWT [68-69,69]. 2023-10-21T04:25:12.769+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 242994859770444 54 51229 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:12.770+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 271074273345339 53 51229 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:12.773+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 215467668600630 34 51229 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:25:12.775+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 200436537417141 54 51229 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:12.775+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 207367350302991 64 51229 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:25:12.779+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 204867593700407 56 51229 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:12.779+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 188718367400378 56 51229 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:12.779+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 227378777358533 60 51229 172.31.5.112:9000. HWT [57-60,60]. 2023-10-21T04:25:12.786+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009341080 2023-10-21T04:25:12.787+05:30 [Info] handleUpdateIndexRState: Index instance 10979931881244616620 rstate moved to ACTIVE 2023-10-21T04:25:12.788+05:30 [Info] Indexer::handleMergePartition Source 13895180128994009539 Target 11460826864060385936 2023-10-21T04:25:12.788+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc008ff7560 2023-10-21T04:25:12.789+05:30 [Info] MergePartition: sent async request to update index instance 11460826864060385936 rstate moved to ACTIVE 2023-10-21T04:25:12.789+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 279362078165491 55 51229 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:12.789+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 170056388266420 72 51229 172.31.5.112:9000. HWT [70-72,72]. 2023-10-21T04:25:12.789+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 129411666769864 55 51229 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:12.789+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 264835102027079 54 51229 172.31.5.112:9000. HWT [52-54,54]. 2023-10-21T04:25:12.790+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 16044440367455 54 51229 172.31.5.112:9000. HWT [53-54,54]. 2023-10-21T04:25:12.794+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 125009072873642 53 51229 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:12.794+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 75566139929862 58 51229 172.31.5.112:9000. HWT [56-58,58]. 2023-10-21T04:25:12.796+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 106932309485550 53 51229 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:12.798+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.83418ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.800+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2296526085582506469 Defn: DefnId: 8857269605176042330 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009707920 2023-10-21T04:25:12.800+05:30 [Info] handleUpdateIndexRState: Index instance 2296526085582506469 rstate moved to ACTIVE 2023-10-21T04:25:12.801+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_1 InstanceIds [9912003609070511414 15667952562388529486 12004664282277897596] 2023-10-21T04:25:12.801+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 51229 2023-10-21T04:25:12.801+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 51229 2023-10-21T04:25:12.801+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:25:12.801+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:25:12.801+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:25:12.801+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:25:12.801+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 51229 2023-10-21T04:25:12.801+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 51229 STREAM_ACTIVE 2023-10-21T04:25:12.801+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:25:12.810+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.833+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.813+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.784872ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.813+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 13895180128994009539, realInst: 11460826864060385936 2023-10-21T04:25:12.836+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:25:12.840+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 133748675726909 56 51229 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:12.840+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 10527297481171 57 51229 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:12.840+05:30 [Info] switched currmeta from 145 -> 147 force true 2023-10-21T04:25:12.850+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.853+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.979908ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.853+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 4992762551574016313. Retrying... 2023-10-21T04:25:12.853+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[2296526085582506469:true 10979931881244616620:true 11460826864060385936:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:25:12.856+05:30 [Info] switched currmeta from 147 -> 148 force true 2023-10-21T04:25:12.857+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 3674078063353481907, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:12.857+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 3674078063353481907, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:25:12.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.864+05:30 [Info] switched currmeta from 148 -> 148 force true 2023-10-21T04:25:12.886+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2281164589865190206:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:12.887+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2281164589865190206:[0] 12882995014846270330:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:12.887+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2281164589865190206:[0] 12882995014846270330:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 {Index inst : [partitions]}: map[620420137039842024:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 1697842319286825244 2023-10-21T04:25:12.903+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4992762551574016313,realInstId: 0, indexDefn DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:25:12.903+05:30 [Info] Indexer::handleRecoverIndex InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:12.904+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4992762551574016313 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10890719122371003479 7659339677706884510]] 2023-10-21T04:25:12.904+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4992762551574016313 PartitionId: 0 using shardIds: [10890719122371003479 7659339677706884510], rebalance running: true 2023-10-21T04:25:12.904+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4992762551574016313, indexDefn DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:25:12.904+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 47.31239ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery) 2023-10-21T04:25:12.903+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 318.063µs 2023-10-21T04:25:12.905+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 2.333254ms 2023-10-21T04:25:12.916+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:12.917+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:12.920+05:30 [Info] switched currmeta from 148 -> 149 force true 2023-10-21T04:25:12.922+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 64 2023-10-21T04:25:12.923+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: RP version: 37 numRPs: 2 rp [0] sn: 1800009 count: 27 metaSz: 2546 rp [1] sn: 1800014 count: 24 metaSz: 2546 2023-10-21T04:25:12.923+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: doInit: data UsedSpace 427233 recovery UsedSpace 323466 2023-10-21T04:25:12.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:25:12.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479 2023-10-21T04:25:12.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data 2023-10-21T04:25:12.934+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:25:12.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex started 2023-10-21T04:25:12.935+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:25:12.935+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:25:12.935+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:25:12.956+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4992762551574016313 PartitionId 0 Warmup took 52.314143ms 2023-10-21T04:25:12.957+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4992762551574016313 partitionId 0 WriterThreads 4 2023-10-21T04:25:12.957+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4992762551574016313 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index Index: 4992762551574016313 Partition: 0 2023-10-21T04:25:12.982+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4992762551574016313, shardIdMap: map[0:[10890719122371003479]] err: 2023-10-21T04:25:12.982+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4992762551574016313, partnShardIdMap: map[0:[10890719122371003479]], err: 2023-10-21T04:25:12.983+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:12.983+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4992762551574016313 2023-10-21T04:25:12.983+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:12.983+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:25:12.983+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:12.984+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:12.984+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:12.984+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4992762551574016313 Partitions [0] 2023-10-21T04:25:12.986+05:30 [Info] StorageMgr::openSnapshot IndexInst:4992762551574016313 Partition:0 Attempting to open snapshot (SnapshotInfo: count:24 committed:false) 2023-10-21T04:25:12.998+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.446287ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:12.998+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 79.991µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:25:13.039+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:13.039+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:13.042+05:30 [Info] switched currmeta from 149 -> 150 force true 2023-10-21T04:25:13.071+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9902809143142643, throttleDelayMs (new, old, change): (7, 3, 4) 2023-10-21T04:25:13.135+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:13.135+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:13.135+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:13.135+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 10979931881244616620:[0]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:13.135+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2296526085582506469:[0] 10979931881244616620:[0] 11460826864060385936:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:13.162+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:13.162+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:13.165+05:30 [Info] switched currmeta from 150 -> 152 force true 2023-10-21T04:25:13.197+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:13.197+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:13.202+05:30 [Info] switched currmeta from 152 -> 152 force true 2023-10-21T04:25:13.251+05:30 [Info] Timekeeper::sendNewStabilityTs: forcing an INMEM_SNAP even though there is no change in flushTs timestamp. streamId: MAINT_STREAM, keyspaceId: bucket_1, flushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 5 58 55 58} { 1 a2a78b2a980d 5 62 59 62} { 2 86ac6f7f2bf7 5 56 54 56} { 3 c3e1ddff7531 5 37 35 37} { 4 1a043769c363 5 60 58 60} { 5 23f9109dc7c2 5 55 52 55} { 6 dc654c627b06 5 64 62 64} { 7 ae8907d03f9a 5 65 63 65} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 5 57 53 57} { 10 f9931a891135 5 58 54 58} { 11 3cd3f3a92ec7 5 53 51 53} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 5 77 75 77} { 15 73acbb1d6dab 4 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 5 65 63 65} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 5 53 51 53} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:25:13.478+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[5004576769623010783:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:13.478+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [12833670934079048728] belonging to collectionId: 9, ttid: ShardToken12:9b:ba:a8:e5:6a:47:bd. Initiating build 2023-10-21T04:25:13.503+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [5004576769623010783] &{2} 2023-10-21T04:25:13.503+05:30 [Info] Indexer::handleBuildRecoveredIndexes [5004576769623010783] 2023-10-21T04:25:13.503+05:30 [Info] bulkUpdateRState: Index instance 5004576769623010783 rstate moved to PENDING 2023-10-21T04:25:13.503+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [5004576769623010783] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:25:13.504+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:13.505+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:13.505+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:13.506+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:25:13.506+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.506+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:13.519+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:13.519+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:25:13.519+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:25:13.519+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:13.519+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:13.519+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:25:13.519+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.519+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:13.519+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:25:13.519+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:25:13.519+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 20581 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:25:13.519+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.519+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:13.520+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:25:13.520+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:25:13.520+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 5004576769623010783 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:13.520+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:25:13.520+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=56,1=60,2=54,3=35,4=58,5=53,6=62,7=63,8=45,9=55,10=56,11=51,12=42,13=47,14=75,15=51,16=60,17=51,18=63,19=48,20=50,21=48,22=51,23=59,24=59,25=72,26=52,27=50,28=58,29=57,30=58,31=67,32=60,33=67,34=60,35=47,36=58,37=54,38=60,39=68,40=62,41=58,42=46,43=55,44=67,45=69,46=54,47=53,48=60,49=34,50=64,51=54,52=56,53=56,54=55,55=72,56=54,57=55,58=54,59=53,60=58,61=53,62=57,63=56] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 20581 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} NumVBuckets: 64 2023-10-21T04:25:13.521+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:25:13.521+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:25:13.522+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:13.522+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 18.831073ms 2023-10-21T04:25:13.522+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 18.877744ms 2023-10-21T04:25:13.522+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [5004576769623010783] 2023-10-21T04:25:13.522+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 43.904855ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:13.524+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[5004576769623010783:true] 2023-10-21T04:25:13.530+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:25:13.541+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.929238ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:13.743+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 263268055970459 56 35695 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:13.745+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 176639880977228 66 35695 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:13.745+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 35403626585217 52 35695 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:13.747+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:13.747+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:13.747+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 215749515705641 47 35695 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:25:13.749+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 159113185399719 60 35695 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:13.751+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 3874519919278 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.752+05:30 [Info] switched currmeta from 152 -> 156 force true 2023-10-21T04:25:13.754+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 261493429196201 63 35695 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:13.757+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 170288557775498 54 35695 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:13.757+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 264975169139761 58 35695 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:13.757+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:13.758+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:13.759+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 236398693961229 66 35695 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:13.761+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 206533515052028 52 35695 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:13.763+05:30 [Info] switched currmeta from 156 -> 156 force true 2023-10-21T04:25:13.763+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 234758907181738 58 35695 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:13.766+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 116104661063478 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.769+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 162158720181435 69 35695 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:13.769+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 140236896721046 65 35695 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:13.769+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 102350060311343 56 35695 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:13.771+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 200752780087280 58 35695 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:13.773+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 98820809009123 48 35695 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:13.776+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 208836052671833 48 35695 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:13.776+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 207134287529256 48 35695 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:13.779+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 92956357968546 54 35695 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:13.782+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 192215088742243 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.782+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 27979933591532 56 35695 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:13.785+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 154934968850942 55 35695 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:13.787+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 271667630317209 54 35695 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:13.789+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 168740244208559 63 35695 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:13.791+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 120989102157727 61 35695 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:25:13.794+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 5715487545729 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.796+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 236953356346174 56 35695 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:13.798+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 130231725986856 51 35695 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:13.800+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 60545336618717 48 35695 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:13.802+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:25:13.802+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:25:13.804+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 73432771436458 62 35695 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:13.804+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 67362620867705 69 35695 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:13.805+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 278426329668882 58 35695 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:13.808+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 51776324008509 63 35695 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:13.810+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 24368176313355 53 35695 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:13.812+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 43594649874772 54 35695 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:13.814+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 83647976044922 61 35695 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:25:13.816+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 181267443431019 55 35695 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:13.818+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 100659545510465 52 35695 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:13.821+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 153478150002150 72 35695 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:25:13.823+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 116785907755771 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.825+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 45479338375128 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.827+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 249919763471169 68 35695 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:25:13.829+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 166675095997741 60 35695 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:13.831+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 49796766757346 66 35695 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:13.833+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 124194574848641 58 35695 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:13.835+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 149022976193937 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.838+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 19274425379955 58 35695 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:13.840+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 184487117398647 57 35695 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:13.842+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 199728167534433 68 35695 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:25:13.845+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 158076890855634 56 35695 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:13.845+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 196777743741606 53 35695 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:13.848+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 138935694816848 63 35695 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:13.848+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 24856128127173 62 35695 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:13.850+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 61798873988101 69 35695 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:13.852+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 29071415952852 63 35695 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:13.854+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 191719959781498 65 35695 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:13.857+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 103336286678362 48 35695 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:13.859+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 56626109228904 48 35695 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:13.859+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 140942236111856 65 35695 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:13.862+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 119871837460216 41 35695 172.31.5.112:9000. HWT [41-41,41]. 2023-10-21T04:25:13.867+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_%2 InstanceIds [9912003609070511414 15667952562388529486 12004664282277897596 10979931881244616620 2296526085582506469 11460826864060385936] 2023-10-21T04:25:13.867+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 35695 2023-10-21T04:25:13.867+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 252537041863196 75 35695 172.31.5.112:9000. HWT [75-75,75]. 2023-10-21T04:25:13.867+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 177105814270799 60 35695 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:13.867+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 35695 2023-10-21T04:25:13.867+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:25:13.867+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:25:13.867+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:25:13.867+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 35695 2023-10-21T04:25:13.867+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 35695 STREAM_ACTIVE 2023-10-21T04:25:13.910+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4992762551574016313:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:25:13.911+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [3674078063353481907] belonging to collectionId: 9, ttid: ShardToken34:c1:80:2c:eb:79:82:c5. Initiating build 2023-10-21T04:25:13.929+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [4992762551574016313] &{2} 2023-10-21T04:25:13.929+05:30 [Info] Indexer::handleBuildRecoveredIndexes [4992762551574016313] 2023-10-21T04:25:13.929+05:30 [Info] bulkUpdateRState: Index instance 4992762551574016313 rstate moved to PENDING 2023-10-21T04:25:13.929+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [4992762551574016313] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:25:13.929+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:13.929+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:13.929+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:13.953+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:25:13.958+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:13.958+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:13.958+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.959+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:13.959+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.960+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:13.960+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:13.960+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:25:13.961+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:25:13.961+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:13.961+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:13.962+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_system:_query:{0xc00bfc3380 }] 2023-10-21T04:25:13.964+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.963+05:30 [Info] switched currmeta from 156 -> 157 force true 2023-10-21T04:25:13.966+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:13.966+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.968+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:13.968+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_system:_query reset bucket state to NIL 2023-10-21T04:25:13.969+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:25:13.969+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId 21321 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} 2023-10-21T04:25:13.970+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.971+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:13.971+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:13.971+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:13.972+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:25:13.973+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:25:13.973+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4992762551574016313 stream INIT_STREAM keyspaceId bucket_%2:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:25:13.973+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:25:13.973+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=56,1=66,2=52,3=47,4=60,5=57,6=63,7=54,8=58,9=66,10=52,11=58,12=57,13=56,14=65,15=69,16=58,17=48,18=48,19=48,20=54,21=56,22=57,23=55,24=54,25=63,26=61,27=57,28=56,29=51,30=48,31=62,32=69,33=58,34=63,35=53,36=54,37=61,38=55,39=52,40=72,41=57,42=57,43=68,44=60,45=66,46=58,47=57,48=58,49=57,50=68,51=53,52=56,53=63,54=62,55=69,56=63,57=65,58=48,59=65,60=48,61=41,62=60,63=75] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 21321 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} NumVBuckets: 64 2023-10-21T04:25:13.976+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_system:_query:{0xc00b57fbc0 } bucket_1:_system:_query:{0xc00bfc3380 }] 2023-10-21T04:25:13.980+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_%2:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:25:13.977+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:13.980+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 51.821751ms 2023-10-21T04:25:13.981+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 51.867479ms 2023-10-21T04:25:13.981+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [4992762551574016313] 2023-10-21T04:25:13.981+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 69.620792ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:13.982+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4992762551574016313:true] 2023-10-21T04:25:13.985+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_system:_query Check pending stream merge. 2023-10-21T04:25:14.008+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.704085ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:14.028+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:14.028+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:14.034+05:30 [Info] switched currmeta from 157 -> 158 force true 2023-10-21T04:25:14.039+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:42380" doAuth() ... 2023-10-21T04:25:14.041+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:42380" auth successful 2023-10-21T04:25:14.041+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:42380" +1 2023-10-21T04:25:14.104+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9758361756242347, throttleDelayMs (new, old, change): (10, 7, 3) 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 7609950505007110692 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 3593050551190633737 RealInstId: 3593050551190633737 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 10590902705185414921 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 2428478379951969204 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 12882995014846270330 RealInstId: 12882995014846270330 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 2281164589865190206 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 3539475206979968695 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.427+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 620420137039842024 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_partitioned Progress: 100 InstId: 11460826864060385936 RealInstId: 11460826864060385936 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:idx_secondary Progress: 100 InstId: 10979931881244616620 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:_default:#primary Progress: 100 InstId: 2296526085582506469 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 4992762551574016313 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_secondary Progress: 100 InstId: 12004664282277897596 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:idx_partitioned Progress: 100 InstId: 9912003609070511414 RealInstId: 9912003609070511414 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:_default:#primary Progress: 100 InstId: 15667952562388529486 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 5004576769623010783 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:25:14.428+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:25:14.550+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:25:14.667+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 0 263268055970459 56 21321 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:14.670+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 1 176639880977228 66 21321 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:14.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 3 215749515705641 47 21321 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:25:14.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 2 35403626585217 52 21321 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:14.678+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 4 159113185399719 60 21321 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:14.678+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 5 3874519919278 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.684+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 8 264975169139761 58 21321 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:14.685+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 7 170288557775498 54 21321 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:14.685+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 6 261493429196201 63 21321 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:14.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 10 206533515052028 52 21321 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:14.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 9 236398693961229 66 21321 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:14.691+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 11 234758907181738 58 21321 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:14.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 13 102350060311343 56 21321 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:14.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 14 140236896721046 65 21321 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:14.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 16 200752780087280 58 21321 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:14.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 15 162158720181435 69 21321 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:14.706+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 12 116104661063478 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 22 192215088742243 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 18 207134287529256 48 21321 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:14.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 17 98820809009123 48 21321 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:14.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 19 208836052671833 48 21321 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:14.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 20 92956357968546 54 21321 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:14.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 21 27979933591532 56 21321 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:14.740+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 23 154934968850942 55 21321 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:14.757+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 24 271667630317209 54 21321 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:14.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 25 168740244208559 63 21321 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:14.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 28 236953356346174 56 21321 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:14.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 26 120989102157727 61 21321 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:25:14.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 27 5715487545729 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.763+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 30 60545336618717 48 21321 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:14.763+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 29 130231725986856 51 21321 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:14.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 32 67362620867705 69 21321 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:14.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 31 73432771436458 62 21321 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:14.773+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 34 51776324008509 63 21321 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:14.773+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 33 278426329668882 58 21321 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:14.778+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 35 24368176313355 53 21321 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:14.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 38 181267443431019 55 21321 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:14.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 36 43594649874772 54 21321 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:14.792+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 39 100659545510465 52 21321 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:14.792+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 37 83647976044922 61 21321 172.31.5.112:9000. HWT [61-61,61]. 2023-10-21T04:25:14.794+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 40 153478150002150 72 21321 172.31.5.112:9000. HWT [72-72,72]. 2023-10-21T04:25:14.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 43 249919763471169 68 21321 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:25:14.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 42 45479338375128 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.797+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 41 116785907755771 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.799+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 44 166675095997741 60 21321 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:14.800+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:14.800+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:14.804+05:30 [Info] switched currmeta from 158 -> 159 force true 2023-10-21T04:25:14.804+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:25:14.806+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 47 149022976193937 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.806+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 45 49796766757346 66 21321 172.31.5.112:9000. HWT [66-66,66]. 2023-10-21T04:25:14.806+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 46 124194574848641 58 21321 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:14.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 50 199728167534433 68 21321 172.31.5.112:9000. HWT [68-68,68]. 2023-10-21T04:25:14.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 48 19274425379955 58 21321 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:14.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 49 184487117398647 57 21321 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:14.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 51 196777743741606 53 21321 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:14.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 52 158076890855634 56 21321 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:14.814+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 53 138935694816848 63 21321 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:14.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 54 24856128127173 62 21321 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:14.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 55 61798873988101 69 21321 172.31.5.112:9000. HWT [69-69,69]. 2023-10-21T04:25:14.821+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 56 29071415952852 63 21321 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:14.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 57 191719959781498 65 21321 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:14.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 58 103336286678362 48 21321 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:14.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 59 140942236111856 65 21321 172.31.5.112:9000. HWT [65-65,65]. 2023-10-21T04:25:14.836+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 61 119871837460216 41 21321 172.31.5.112:9000. HWT [41-41,41]. 2023-10-21T04:25:14.836+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 60 56626109228904 48 21321 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:14.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 62 177105814270799 60 21321 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:14.846+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_%2:_system:_query InstanceIds [4992762551574016313] 2023-10-21T04:25:14.846+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query Cid 9 SessionId 21321 2023-10-21T04:25:14.846+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 21321 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 4992762551574016313 INDEX_STATE_CATCHUP 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 4992762551574016313 true 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 4992762551574016313 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query Session: 21321 BuildTS: [0=56,1=66,2=52,3=47,4=60,5=57,6=63,7=54,8=58,9=66,10=52,11=58,12=57,13=56,14=65,15=69,16=58,17=48,18=48,19=48,20=54,21=56,22=57,23=55,24=54,25=63,26=61,27=57,28=56,29=51,30=48,31=62,32=69,33=58,34=63,35=53,36=54,37=61,38=55,39=52,40=72,41=57,42=57,43=68,44=60,45=66,46=58,47=57,48=58,49=57,50=68,51=53,52=56,53=63,54=62,55=69,56=63,57=65,58=48,59=65,60=48,61=41,62=60,63=75] 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: MAINT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:25:14.846+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:25:14.846+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 21321 STREAM_ACTIVE 2023-10-21T04:25:14.847+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 21321 2023-10-21T04:25:14.847+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_system:_query 21321 2023-10-21T04:25:14.847+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:25:14.847+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:14.848+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:14.848+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:14.849+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:14.850+05:30 [Info] send progress stats to clients 2023-10-21T04:25:14.850+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:25:14.850+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:25:14.850+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:3] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:25:14.850+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 21321 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:25:14.850+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=54,1=64,2=4,3=45,4=4,5=55,6=61,7=4,8=4,9=64,10=4,11=4,12=4,13=54,14=63,15=4,16=56,17=46,18=4,19=46,20=52,21=4,22=4,23=53,24=16,25=61,26=4,27=55,28=4,29=4,30=4,31=4,32=4,33=56,34=61,35=51,36=34,37=4,38=26,39=50,40=70,41=4,42=4,43=66,44=58,45=64,46=56,47=4,48=56,49=55,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=4,59=4,60=4,61=15,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:14.853+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 63 252537041863196 75 21321 172.31.5.112:9000. HWT [75-75,75]. 2023-10-21T04:25:14.855+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:25:14.860+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId bucket_%2 2023-10-21T04:25:14.860+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 21321 2023-10-21T04:25:14.860+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 21321 2023-10-21T04:25:14.860+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:25:14.860+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:25:14.860+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 4992762551574016313 false 2023-10-21T04:25:14.860+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_system:_query 4992762551574016313 2023-10-21T04:25:14.860+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: MAINT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:25:14.860+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 21321 2023-10-21T04:25:14.863+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.284252ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:14.876+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:25:14.877+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:25:14.897+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:14.897+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:14.901+05:30 [Info] switched currmeta from 159 -> 160 force true 2023-10-21T04:25:15.073+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.968883718629522 2023-10-21T04:25:15.384+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 0 186517626268899 56 20581 172.31.5.112:9000. HWT [55-56,56]. 2023-10-21T04:25:15.384+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 2 148075163102199 54 20581 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:15.384+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 1 178840478062605 60 20581 172.31.5.112:9000. HWT [59-60,60]. 2023-10-21T04:25:15.385+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 3 215374859564337 35 20581 172.31.5.112:9000. HWT [35-35,35]. 2023-10-21T04:25:15.387+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 5 39552632604610 53 20581 172.31.5.112:9000. HWT [52-53,53]. 2023-10-21T04:25:15.387+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 4 28605411869539 58 20581 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:15.390+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 6 242327631330054 62 20581 172.31.5.112:9000. HWT [62-62,62]. 2023-10-21T04:25:15.390+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 7 191903564840858 63 20581 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:15.392+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 9 225280996447309 55 20581 172.31.5.112:9000. HWT [53-55,55]. 2023-10-21T04:25:15.392+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 8 50961953886801 45 20581 172.31.5.112:9000. HWT [45-45,45]. 2023-10-21T04:25:15.397+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 12 4429827396609 42 20581 172.31.5.112:9000. HWT [42-42,42]. 2023-10-21T04:25:15.397+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 10 274410200699189 56 20581 172.31.5.112:9000. HWT [54-56,56]. 2023-10-21T04:25:15.397+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 11 66881023717063 51 20581 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:15.402+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 13 183148377680315 47 20581 172.31.5.112:9000. HWT [46-47,47]. 2023-10-21T04:25:15.402+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 14 226814108379280 75 20581 172.31.5.112:9000. HWT [75-75,75]. 2023-10-21T04:25:15.407+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 15 127185710837163 51 20581 172.31.5.112:9000. HWT [50-51,51]. 2023-10-21T04:25:15.407+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 16 257031483891250 60 20581 172.31.5.112:9000. HWT [55-60,60]. 2023-10-21T04:25:15.407+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 17 55290066231400 51 20581 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:15.409+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 18 242578104468179 63 20581 172.31.5.112:9000. HWT [63-63,63]. 2023-10-21T04:25:15.412+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 19 210306088158507 48 20581 172.31.5.112:9000. HWT [48-48,48]. 2023-10-21T04:25:15.415+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 21 209771427122473 48 20581 172.31.5.112:9000. HWT [47-48,48]. 2023-10-21T04:25:15.415+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 20 230239359684242 50 20581 172.31.5.112:9000. HWT [50-50,50]. 2023-10-21T04:25:15.416+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 23 216462437971699 59 20581 172.31.5.112:9000. HWT [57-59,59]. 2023-10-21T04:25:15.416+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 22 260036488774725 51 20581 172.31.5.112:9000. HWT [51-51,51]. 2023-10-21T04:25:15.418+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 24 160961818008256 59 20581 172.31.5.112:9000. HWT [59-59,59]. 2023-10-21T04:25:15.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 25 280874278672777 72 20581 172.31.5.112:9000. HWT [71-72,72]. 2023-10-21T04:25:15.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 26 219132834127088 52 20581 172.31.5.112:9000. HWT [52-52,52]. 2023-10-21T04:25:15.425+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 28 102578907139247 58 20581 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:15.428+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 29 45957426777107 57 20581 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:15.431+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 31 217381278987664 67 20581 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:25:15.432+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 30 99433130332778 58 20581 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:15.432+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 27 184056281482149 50 20581 172.31.5.112:9000. HWT [49-50,50]. 2023-10-21T04:25:15.433+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 33 136958029391999 67 20581 172.31.5.112:9000. HWT [65-67,67]. 2023-10-21T04:25:15.433+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 32 82469662608686 60 20581 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:15.435+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 35 70213628416850 47 20581 172.31.5.112:9000. HWT [47-47,47]. 2023-10-21T04:25:15.435+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 34 28417669987133 60 20581 172.31.5.112:9000. HWT [58-60,60]. 2023-10-21T04:25:15.438+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 36 253845164956592 58 20581 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:15.438+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 37 253494845579900 54 20581 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:15.443+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 38 188107826286720 60 20581 172.31.5.112:9000. HWT [60-60,60]. 2023-10-21T04:25:15.443+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 39 222555662236724 68 20581 172.31.5.112:9000. HWT [67-68,68]. 2023-10-21T04:25:15.444+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 41 259381390638966 58 20581 172.31.5.112:9000. HWT [58-58,58]. 2023-10-21T04:25:15.444+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 40 5915864305121 62 20581 172.31.5.112:9000. HWT [60-62,62]. 2023-10-21T04:25:15.447+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 42 252931432434966 46 20581 172.31.5.112:9000. HWT [46-46,46]. 2023-10-21T04:25:15.450+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 45 12301831552869 69 20581 172.31.5.112:9000. HWT [68-69,69]. 2023-10-21T04:25:15.450+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 43 146267938201125 55 20581 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:15.450+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 44 158545118983293 67 20581 172.31.5.112:9000. HWT [67-67,67]. 2023-10-21T04:25:15.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 46 242994859770444 54 20581 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:15.458+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 47 271074273345339 53 20581 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:15.458+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 48 227378777358533 60 20581 172.31.5.112:9000. HWT [57-60,60]. 2023-10-21T04:25:15.458+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 50 207367350302991 64 20581 172.31.5.112:9000. HWT [64-64,64]. 2023-10-21T04:25:15.463+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 51 200436537417141 54 20581 172.31.5.112:9000. HWT [54-54,54]. 2023-10-21T04:25:15.463+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 53 204867593700407 56 20581 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:15.467+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 52 188718367400378 56 20581 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:15.469+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 49 215467668600630 34 20581 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:25:15.469+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 54 279362078165491 55 20581 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:15.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 56 264835102027079 54 20581 172.31.5.112:9000. HWT [52-54,54]. 2023-10-21T04:25:15.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 55 170056388266420 72 20581 172.31.5.112:9000. HWT [70-72,72]. 2023-10-21T04:25:15.476+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 61 106932309485550 53 20581 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:15.476+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 57 129411666769864 55 20581 172.31.5.112:9000. HWT [55-55,55]. 2023-10-21T04:25:15.476+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 58 16044440367455 54 20581 172.31.5.112:9000. HWT [53-54,54]. 2023-10-21T04:25:15.476+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 60 75566139929862 58 20581 172.31.5.112:9000. HWT [56-58,58]. 2023-10-21T04:25:15.476+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 59 125009072873642 53 20581 172.31.5.112:9000. HWT [53-53,53]. 2023-10-21T04:25:15.480+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 62 10527297481171 57 20581 172.31.5.112:9000. HWT [57-57,57]. 2023-10-21T04:25:15.484+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 bucket_1:_system:_query InstanceIds [4992762551574016313 5004576769623010783] 2023-10-21T04:25:15.484+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 20581 2023-10-21T04:25:15.484+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 20581 2023-10-21T04:25:15.484+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:25:15.484+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:25:15.484+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 5004576769623010783 INDEX_STATE_CATCHUP 2023-10-21T04:25:15.484+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 5004576769623010783 true 2023-10-21T04:25:15.484+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 5004576769623010783 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query Session: 20581 BuildTS: [0=56,1=60,2=54,3=35,4=58,5=53,6=62,7=63,8=45,9=55,10=56,11=51,12=42,13=47,14=75,15=51,16=60,17=51,18=63,19=48,20=50,21=48,22=51,23=59,24=59,25=72,26=52,27=50,28=58,29=57,30=58,31=67,32=60,33=67,34=60,35=47,36=58,37=54,38=60,39=68,40=62,41=58,42=46,43=55,44=67,45=69,46=54,47=53,48=60,49=34,50=64,51=54,52=56,53=56,54=55,55=72,56=54,57=55,58=54,59=53,60=58,61=53,62=57,63=56] 2023-10-21T04:25:15.484+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_system:_query, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} , fetchKVSeq false 2023-10-21T04:25:15.485+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:25:15.485+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 20581 STREAM_ACTIVE 2023-10-21T04:25:15.485+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 20581 2023-10-21T04:25:15.485+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_system:_query 20581 2023-10-21T04:25:15.485+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:15.486+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:15.486+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:15.486+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:15.486+05:30 [Info] send progress stats to clients 2023-10-21T04:25:15.486+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:25:15.486+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:25:15.486+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:4] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:25:15.486+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 20581 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:25:15.490+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=56,1=60,2=4,3=4,4=4,5=53,6=4,7=4,8=4,9=55,10=56,11=4,12=4,13=47,14=75,15=51,16=60,17=4,18=63,19=4,20=4,21=48,22=4,23=59,24=37,25=72,26=4,27=50,28=4,29=57,30=4,31=4,32=4,33=67,34=60,35=4,36=4,37=4,38=4,39=68,40=62,41=4,42=4,43=35,44=4,45=69,46=24,47=4,48=60,49=4,50=4,51=4,52=4,53=4,54=4,55=72,56=54,57=4,58=54,59=4,60=58,61=21,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:15.490+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 63 133748675726909 56 20581 172.31.5.112:9000. HWT [56-56,56]. 2023-10-21T04:25:15.491+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:25:15.495+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId bucket_1 2023-10-21T04:25:15.495+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 20581 2023-10-21T04:25:15.495+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 20581 2023-10-21T04:25:15.495+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:25:15.495+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:25:15.495+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 5004576769623010783 false 2023-10-21T04:25:15.495+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_system:_query 5004576769623010783 2023-10-21T04:25:15.495+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_system:_query. 2023-10-21T04:25:15.495+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 20581 2023-10-21T04:25:15.499+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.36037ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:15.512+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:15.512+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:15.516+05:30 [Info] switched currmeta from 160 -> 162 force true 2023-10-21T04:25:15.529+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:15.529+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:15.534+05:30 [Info] switched currmeta from 162 -> 162 force true 2023-10-21T04:25:15.848+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:25:15.848+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_system:_query 2023-10-21T04:25:15.848+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 4992762551574016313 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId: 21321, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 18394135563695078153 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 ef70def4f29b 5 56 56 56} { 1 a0a72d5bdb4c 5 66 66 66} { 2 20330c961c81 5 52 52 52} { 3 c439193e7d29 5 47 47 47} { 4 90b66ce513a7 5 60 60 60} { 5 3861b622aae 5 57 57 57} { 6 edd3aef555a9 5 63 63 63} { 7 9ae064feda8a 5 54 54 54} { 8 f0fe56d4b431 5 58 58 58} { 9 d700dc2dce0d 5 66 66 66} { 10 bbd754cc83fc 5 52 52 52} { 11 d58311453aaa 5 58 58 58} { 12 6998b9145736 5 57 57 57} { 13 5d163afcd32f 5 56 56 56} { 14 7f8b7269d896 5 65 65 65} { 15 937b84e9e4bb 5 69 69 69} { 16 b69566250ff0 5 58 58 58} { 17 59e08337a7e3 5 48 48 48} { 18 bc6335a10528 5 48 48 48} { 19 bdef6ebc5d59 5 48 48 48} { 20 548b16b242a2 5 54 54 54} { 21 19729600a7ec 5 56 56 56} { 22 aed19015bf63 5 57 57 57} { 23 8ce99bc819fe 5 55 55 55} { 24 f7148c9a1e99 5 54 54 54} { 25 9977e650bbaf 5 63 63 63} { 26 6e09f874af9f 5 61 61 61} { 27 532bd9ad981 5 57 57 57} { 28 d78200a2233e 5 56 56 56} { 29 7671efc41428 5 51 51 51} { 30 3710cf482edd 5 48 48 48} { 31 42c966419faa 5 62 62 62} { 32 3d4415168479 5 69 69 69} { 33 fd3a2e79dd12 5 58 58 58} { 34 2f171d667a3d 5 63 63 63} { 35 1629a87e340b 5 53 53 53} { 36 27a62b9eb154 5 54 54 54} { 37 4c13cfe5f97a 5 61 61 61} { 38 a4dc9d93366b 5 55 55 55} { 39 5b8ca0741241 5 52 52 52} { 40 8b966a9c95e6 5 72 72 72} { 41 6a37568beafb 5 57 57 57} { 42 295cfbcef7d8 5 57 57 57} { 43 e34cfab95b41 5 68 68 68} { 44 979711e3cd2d 5 60 60 60} { 45 2d4a3697f1e2 5 66 66 66} { 46 70f44d845281 5 58 58 58} { 47 87891d8f3591 5 57 57 57} { 48 1187ad472c73 5 58 58 58} { 49 a7ca41192277 5 57 57 57} { 50 b5a6d67a3f61 5 68 68 68} { 51 b2f7e383f2a6 5 53 53 53} { 52 8fc524ed38d2 5 56 56 56} { 53 7e5c7cbc7650 5 63 63 63} { 54 169b44b000c5 5 62 62 62} { 55 3834abee7c05 5 69 69 69} { 56 1a70b76bc5d4 5 63 63 63} { 57 ae5e4819607a 5 65 65 65} { 58 5dfbdaa8ed5a 5 48 48 48} { 59 802fabead3f0 5 65 65 65} { 60 33804b202768 5 48 48 48} { 61 6d05d64a56f8 5 41 41 41} { 62 a113a925b34f 5 60 60 60} { 63 e5ae5cac421c 5 75 75 75} 2023-10-21T04:25:15.848+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 4992762551574016313 INDEX_STATE_ACTIVE 2023-10-21T04:25:15.848+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query State Changed to INACTIVE 2023-10-21T04:25:15.848+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:25:15.848+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:25:15.848+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 0 New 0 2023-10-21T04:25:15.848+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 21321 2023-10-21T04:25:15.851+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:15.852+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:15.852+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:15.852+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 21321 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:15.852+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4992762551574016313 from stream INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:25:15.852+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:25:15.852+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:25:15.852+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:15.852+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 21321 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:15.852+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_system:_query:{0xc00bfc3380 }] 2023-10-21T04:25:15.852+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:15.853+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:15.853+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:15.853+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:15.853+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:15.853+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:25:15.853+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:15.853+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:15.853+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:25:15.853+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:15.853+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId 21321 2023-10-21T04:25:15.853+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:25:15.856+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 0 263268055970459 56 2023-10-21T04:25:15.856+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.856+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 1 176639880977228 66 2023-10-21T04:25:15.856+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.856+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 2 35403626585217 52 2023-10-21T04:25:15.856+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 3 215749515705641 47 2023-10-21T04:25:15.862+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 4 159113185399719 60 2023-10-21T04:25:15.862+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 5 3874519919278 57 2023-10-21T04:25:15.862+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 6 261493429196201 63 2023-10-21T04:25:15.862+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 7 170288557775498 54 2023-10-21T04:25:15.862+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 8 264975169139761 58 2023-10-21T04:25:15.862+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 9 236398693961229 66 2023-10-21T04:25:15.862+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.862+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 10 206533515052028 52 2023-10-21T04:25:15.863+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.865+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 27 5715487545729 57 2023-10-21T04:25:15.865+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.865+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 32 67362620867705 69 2023-10-21T04:25:15.865+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 14 140236896721046 65 2023-10-21T04:25:15.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 46 124194574848641 58 2023-10-21T04:25:15.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 20 92956357968546 54 2023-10-21T04:25:15.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 52 158076890855634 56 2023-10-21T04:25:15.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 22 192215088742243 57 2023-10-21T04:25:15.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 23 154934968850942 55 2023-10-21T04:25:15.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 24 271667630317209 54 2023-10-21T04:25:15.866+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.866+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 26 120989102157727 61 2023-10-21T04:25:15.867+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.867+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 29 130231725986856 51 2023-10-21T04:25:15.867+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.867+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 33 278426329668882 58 2023-10-21T04:25:15.867+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.867+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 16 200752780087280 58 2023-10-21T04:25:15.867+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.867+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 48 19274425379955 58 2023-10-21T04:25:15.867+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.867+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:25:15.867+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 21321 2023-10-21T04:25:15.867+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 18 207134287529256 48 2023-10-21T04:25:15.867+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 50 199728167534433 68 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 30 60545336618717 48 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 35 24368176313355 53 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 42 45479338375128 57 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 53 138935694816848 63 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 21 27979933591532 56 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 31 73432771436458 62 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 34 51776324008509 63 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 40 153478150002150 72 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 36 43594649874772 54 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 11 234758907181738 58 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 43 249919763471169 68 2023-10-21T04:25:15.870+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.870+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 12 116104661063478 57 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 44 166675095997741 60 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 13 102350060311343 56 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 45 49796766757346 66 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 17 98820809009123 48 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 49 184487117398647 57 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 25 168740244208559 63 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 28 236953356346174 56 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 38 181267443431019 55 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 15 162158720181435 69 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 47 149022976193937 57 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 19 208836052671833 48 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 51 196777743741606 53 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 37 83647976044922 61 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 39 100659545510465 52 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.871+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 41 116785907755771 57 2023-10-21T04:25:15.871+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 21321 2023-10-21T04:25:15.871+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.872+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:25:15.872+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 61 119871837460216 41 2023-10-21T04:25:15.872+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.872+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 62 177105814270799 60 2023-10-21T04:25:15.872+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.872+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 56 29071415952852 63 2023-10-21T04:25:15.872+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.872+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 57 191719959781498 65 2023-10-21T04:25:15.876+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.877+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]} {Rev:[0 0 0 0 0 0 0 3] ID:rebalance/1972139801924f1fdf1188af4e7137b9 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:15.877+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:15.878+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 59 140942236111856 65 2023-10-21T04:25:15.878+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.878+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 60 56626109228904 48 2023-10-21T04:25:15.878+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.879+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 63 252537041863196 75 2023-10-21T04:25:15.879+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.880+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 54 24856128127173 62 2023-10-21T04:25:15.880+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.880+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 55 61798873988101 69 2023-10-21T04:25:15.880+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.880+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_system:_query 58 103336286678362 48 2023-10-21T04:25:15.880+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:25:15.886+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 33.002454ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:15.990+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 6] 2023-10-21T04:25:16.002+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:16.002+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:16.003+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 6] 2023-10-21T04:25:16.022+05:30 [Info] switched currmeta from 162 -> 164 force true 2023-10-21T04:25:16.027+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009b215c0 2023-10-21T04:25:16.028+05:30 [Info] handleUpdateIndexRState: Index instance 4992762551574016313 rstate moved to ACTIVE 2023-10-21T04:25:16.041+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:16.041+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:16.044+05:30 [Info] switched currmeta from 164 -> 164 force true 2023-10-21T04:25:16.047+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[4992762551574016313:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:25:16.047+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken34:c1:80:2c:eb:79:82:c5, elapsed(sec): 13.019733019 2023-10-21T04:25:16.049+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.051844ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:16.063+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:16.063+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:16.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9541276153426994 2023-10-21T04:25:16.072+05:30 [Info] switched currmeta from 164 -> 165 force true 2023-10-21T04:25:16.089+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:16.091+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken34:c1:80:2c:eb:79:82:c5 2023-10-21T04:25:16.490+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:25:16.490+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_system:_query 2023-10-21T04:25:16.491+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 5004576769623010783 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId: 20581, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 2946622789168712077 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 a9a304dbace3 3 56 55 56} { 1 a2a78b2a980d 3 60 59 60} { 2 86ac6f7f2bf7 3 54 54 54} { 3 c3e1ddff7531 3 35 35 35} { 4 1a043769c363 3 58 58 58} { 5 23f9109dc7c2 3 53 52 53} { 6 dc654c627b06 3 62 62 62} { 7 ae8907d03f9a 3 63 63 63} { 8 2e5981289251 3 45 45 45} { 9 cce451c4944d 3 55 53 55} { 10 f9931a891135 3 56 54 56} { 11 3cd3f3a92ec7 3 51 51 51} { 12 4076649d401 3 42 42 42} { 13 a6928dfe29bb 3 47 46 47} { 14 ce49465c2890 3 75 75 75} { 15 73acbb1d6dab 3 51 50 51} { 16 e9c4ce4b1e32 3 60 55 60} { 17 324938c1e068 3 51 51 51} { 18 dc9f9dbf42d3 3 63 63 63} { 19 bf45b3adc52b 3 48 48 48} { 20 d166c73f4e92 3 50 50 50} { 21 bec93765cd29 3 48 47 48} { 22 ec80768ac445 3 51 51 51} { 23 c4df16b8f6f3 3 59 57 59} { 24 9264d7fc1ac0 3 59 59 59} { 25 ff74239b5989 3 72 71 72} { 26 c74cd6bc9cf0 3 52 52 52} { 27 a765f146c3a5 3 50 49 50} { 28 5d4b83524caf 3 58 58 58} { 29 29cc4c190c13 3 57 57 57} { 30 5a6f1469926a 3 58 58 58} { 31 c5b505ea3590 3 67 67 67} { 32 4b0176f1cd2e 3 60 60 60} { 33 7c9006b0e87f 3 67 65 67} { 34 19d8811ffb3d 3 60 58 60} { 35 3fdbe2615352 3 47 47 47} { 36 e6deeee29bb0 3 58 58 58} { 37 e68d5e39227c 3 54 54 54} { 38 ab15442a5080 3 60 60 60} { 39 ca69c730bc34 3 68 67 68} { 40 56164fd8de1 3 62 60 62} { 41 ebe7efa67f76 3 58 58 58} { 42 e60a302ecd16 3 46 46 46} { 43 8507a87f9e25 3 55 55 55} { 44 9032297f747d 3 67 67 67} { 45 b303e4cbf65 3 69 68 69} { 46 dd00a64c824c 3 54 54 54} { 47 f68a65c53f3b 3 53 53 53} { 48 ceccbf4100c5 3 60 57 60} { 49 c3f779d8f736 3 34 34 34} { 50 bc997940b90f 3 64 64 64} { 51 b64bc49cddb5 3 54 54 54} { 52 aba36b4061ba 3 56 56 56} { 53 ba5374260437 3 56 56 56} { 54 fe140d6ec9f3 3 55 55 55} { 55 9aaa569d49b4 3 72 70 72} { 56 f0ddba2e2d47 3 54 52 54} { 57 75b3006d77c8 3 55 55 55} { 58 e97a32b315f 3 54 53 54} { 59 71b1f161e0aa 3 53 53 53} { 60 44ba1ced7506 3 58 56 58} { 61 61411e5267ee 3 53 53 53} { 62 99313d3a9d3 3 57 57 57} { 63 79a4ca4e4e3d 3 56 56 56} 2023-10-21T04:25:16.491+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 5004576769623010783 INDEX_STATE_ACTIVE 2023-10-21T04:25:16.492+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_system:_query State Changed to INACTIVE 2023-10-21T04:25:16.492+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:25:16.492+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:25:16.492+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:16.492+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 20581 2023-10-21T04:25:16.492+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:16.492+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:16.493+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 6 ReplicaId: 1 2023-10-21T04:25:16.496+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 20581 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:16.496+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 5004576769623010783 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:25:16.496+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:25:16.497+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:25:16.497+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:16.497+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 20581 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:16.498+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:25:16.498+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:25:16.519+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:25:16.520+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:25:16.520+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:42380: use of closed network connection 2023-10-21T04:25:16.520+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:42380" exit: read tcp4 127.0.0.1:9121->127.0.0.1:42380: use of closed network connection 2023-10-21T04:25:16.520+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:42380" closed ! 2023-10-21T04:25:16.521+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:25:16.521+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:16.522+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:16.522+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:16.523+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:16.524+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:25:16.524+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:25:16.546+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId 20581 2023-10-21T04:25:16.546+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:25:16.570+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:16.570+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:16.576+05:30 [Info] switched currmeta from 165 -> 166 force true 2023-10-21T04:25:16.584+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 38.661035ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:16.607+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:16.607+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:16.611+05:30 [Info] switched currmeta from 166 -> 167 force true 2023-10-21T04:25:16.625+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00a7dc720 2023-10-21T04:25:16.625+05:30 [Info] handleUpdateIndexRState: Index instance 5004576769623010783 rstate moved to ACTIVE 2023-10-21T04:25:16.634+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.74976ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:16.634+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[5004576769623010783:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:25:16.634+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken12:9b:ba:a8:e5:6a:47:bd, elapsed(sec): 13.874411161 2023-10-21T04:25:16.643+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken12:9b:ba:a8:e5:6a:47:bd 2023-10-21T04:25:16.649+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:16.646+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_1:_system:_query]. Err genServer.closed 2023-10-21T04:25:16.720+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:25:16.720+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Treating genServer.closed As Success 2023-10-21T04:25:16.720+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 20581 2023-10-21T04:25:16.720+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 20581 2023-10-21T04:25:16.720+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:25:16.756+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:16.797+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:16.797+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:16.799+05:30 [Info] switched currmeta from 167 -> 169 force true 2023-10-21T04:25:16.799+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:16.799+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:16.799+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4992762551574016313:[0]]. New rollbacktime: 1697842503623358421, Old rollbacktime: 0 2023-10-21T04:25:16.802+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken8:4b:39:d1:66:4f:1c:93 MasterId: SourceId: DestId: RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:16.802+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10979931881244616620 PartitionId 0 Started Billing WUs 2023-10-21T04:25:16.848+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:25:17.069+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8624184587705906, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:25:17.538+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:17.544+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:17.544+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:17.555+05:30 [Info] switched currmeta from 169 -> 170 force true 2023-10-21T04:25:17.556+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:17.557+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:17.558+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:17.559+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:17.559+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:17.560+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3539475206979968695:[0]]. New rollbacktime: 1697842503263938379, Old rollbacktime: 0 2023-10-21T04:25:17.591+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken6a:a7:ac:ec:fd:3f:d4:ed MasterId: SourceId: DestId: RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:17.591+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 17394655409336449770 PartitionId 0 Started Billing WUs 2023-10-21T04:25:17.591+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 14271070195512901425 PartitionId 0 Started Billing WUs 2023-10-21T04:25:17.612+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:17.614+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:17.626+05:30 [Info] switched currmeta from 170 -> 170 force true 2023-10-21T04:25:17.644+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.645+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.646+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.647+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.648+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.649+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.650+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[5004576769623010783:[0]]. New rollbacktime: 1697842504289265542, Old rollbacktime: 0 2023-10-21T04:25:17.684+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:17.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:17.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:17.700+05:30 [Info] switched currmeta from 170 -> 170 force true 2023-10-21T04:25:17.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 0 2023-10-21T04:25:17.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 0 2023-10-21T04:25:17.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 0 2023-10-21T04:25:17.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 0 2023-10-21T04:25:17.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2428478379951969204:[0]]. New rollbacktime: 1697842503737063565, Old rollbacktime: 0 2023-10-21T04:25:17.910+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:17.910+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:17.919+05:30 [Info] switched currmeta from 170 -> 171 force true 2023-10-21T04:25:17.924+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:17.924+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:17.929+05:30 [Info] switched currmeta from 171 -> 171 force true 2023-10-21T04:25:17.977+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:17.977+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:17.979+05:30 [Info] switched currmeta from 171 -> 176 force true 2023-10-21T04:25:18.029+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.030+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.035+05:30 [Info] switched currmeta from 176 -> 178 force true 2023-10-21T04:25:18.052+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.052+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.062+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:25:18.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8370863623684113, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:25:18.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.074+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:25:18.102+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.103+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.106+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:25:18.128+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.128+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.138+05:30 [Info] switched currmeta from 178 -> 179 force true 2023-10-21T04:25:18.177+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.179+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.183+05:30 [Info] switched currmeta from 179 -> 180 force true 2023-10-21T04:25:18.227+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.227+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.238+05:30 [Info] switched currmeta from 180 -> 181 force true 2023-10-21T04:25:18.326+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.326+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.329+05:30 [Info] switched currmeta from 181 -> 182 force true 2023-10-21T04:25:18.369+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.369+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.377+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:25:18.421+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.421+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.439+05:30 [Info] switched currmeta from 182 -> 183 force true 2023-10-21T04:25:18.472+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.472+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.475+05:30 [Info] switched currmeta from 183 -> 184 force true 2023-10-21T04:25:18.483+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.484+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.493+05:30 [Info] switched currmeta from 184 -> 185 force true 2023-10-21T04:25:18.502+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.503+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.508+05:30 [Info] switched currmeta from 185 -> 185 force true 2023-10-21T04:25:18.520+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.520+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.526+05:30 [Info] switched currmeta from 185 -> 185 force true 2023-10-21T04:25:18.584+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.584+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.593+05:30 [Info] switched currmeta from 185 -> 187 force true 2023-10-21T04:25:18.605+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.609+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.631+05:30 [Info] switched currmeta from 187 -> 188 force true 2023-10-21T04:25:18.726+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.727+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.749+05:30 [Info] switched currmeta from 188 -> 191 force true 2023-10-21T04:25:18.804+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.806+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.822+05:30 [Info] switched currmeta from 191 -> 193 force true 2023-10-21T04:25:18.830+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.830+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.833+05:30 [Info] switched currmeta from 193 -> 193 force true 2023-10-21T04:25:18.916+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:18.916+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:18.940+05:30 [Info] switched currmeta from 193 -> 194 force true 2023-10-21T04:25:18.989+05:30 [Info] ShardRebalancer::computeProgress 1 2023-10-21T04:25:19.015+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.015+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.030+05:30 [Info] switched currmeta from 194 -> 196 force true 2023-10-21T04:25:19.113+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.113+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.151+05:30 [Info] switched currmeta from 196 -> 197 force true 2023-10-21T04:25:19.248+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.248+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.253+05:30 [Info] switched currmeta from 197 -> 198 force true 2023-10-21T04:25:19.336+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.336+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.360+05:30 [Info] switched currmeta from 198 -> 200 force true 2023-10-21T04:25:19.410+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.410+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.424+05:30 [Info] switched currmeta from 200 -> 201 force true 2023-10-21T04:25:19.449+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.449+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.455+05:30 [Info] switched currmeta from 201 -> 202 force true 2023-10-21T04:25:19.477+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.477+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.479+05:30 [Info] switched currmeta from 202 -> 204 force true 2023-10-21T04:25:19.486+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.486+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.488+05:30 [Info] switched currmeta from 204 -> 204 force true 2023-10-21T04:25:19.518+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.518+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.528+05:30 [Info] switched currmeta from 204 -> 206 force true 2023-10-21T04:25:19.544+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.544+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.550+05:30 [Info] switched currmeta from 206 -> 206 force true 2023-10-21T04:25:19.572+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.572+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.576+05:30 [Info] switched currmeta from 206 -> 206 force true 2023-10-21T04:25:19.634+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.634+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.640+05:30 [Info] switched currmeta from 206 -> 207 force true 2023-10-21T04:25:19.645+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.645+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.647+05:30 [Info] switched currmeta from 207 -> 207 force true 2023-10-21T04:25:19.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.691+05:30 [Info] switched currmeta from 207 -> 208 force true 2023-10-21T04:25:19.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.766+05:30 [Info] switched currmeta from 208 -> 211 force true 2023-10-21T04:25:19.772+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.772+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.774+05:30 [Info] switched currmeta from 211 -> 213 force true 2023-10-21T04:25:19.780+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.780+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.783+05:30 [Info] switched currmeta from 213 -> 213 force true 2023-10-21T04:25:19.941+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.941+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.954+05:30 [Info] switched currmeta from 213 -> 221 force true 2023-10-21T04:25:19.982+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:19.982+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:19.993+05:30 [Info] switched currmeta from 221 -> 222 force true 2023-10-21T04:25:20.030+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:20.030+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:20.045+05:30 [Info] switched currmeta from 222 -> 223 force true 2023-10-21T04:25:20.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 1, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:25:20.077+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:20.077+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:20.086+05:30 [Info] switched currmeta from 223 -> 225 force true 2023-10-21T04:25:20.105+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:20.105+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:20.116+05:30 [Info] switched currmeta from 225 -> 225 force true 2023-10-21T04:25:20.163+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:20.164+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:20.167+05:30 [Info] switched currmeta from 225 -> 226 force true 2023-10-21T04:25:20.206+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:20.206+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:20.210+05:30 [Info] switched currmeta from 226 -> 226 force true 2023-10-21T04:25:20.764+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:20.786+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokened:42:e0:2a:27:fe:e0:80 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7609950505007110692 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 2281269425900993179 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17516054303786468982 RealInstId: 14472287716825196064 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 6637393687348361258 RealInstId: 3593050551190633737 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10590902705185414921 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 2428478379951969204 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 16581200427024765500 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:20.797+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:0 bada28845870208460ab46b431032b01:0 e28a1725ce62fe1e923bede04f38724f:-1 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:20.798+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardToken36:37:b8:9d:2e:c3:90:4a is in state: ShardTokenReady 2023-10-21T04:25:20.798+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardTokened:42:e0:2a:27:fe:e0:80 2023-10-21T04:25:20.875+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 6] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]} {Rev:[0 0 0 0 0 0 0 4] ID:rebalance/1972139801924f1fdf1188af4e7137b9 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.99 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:20.876+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:20.900+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 7] 2023-10-21T04:25:20.902+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 7] 2023-10-21T04:25:20.941+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:20.976+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:20.976+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [2864853750198994630 12177293282243154215] 2023-10-21T04:25:20.976+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : all deamons started 2023-10-21T04:25:20.976+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : all deamons started 2023-10-21T04:25:20.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:25:20.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:25:20.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : RestoreShardDone completed 2023-10-21T04:25:20.976+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : all deamons started 2023-10-21T04:25:20.976+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : all deamons started 2023-10-21T04:25:20.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:25:20.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:25:20.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : RestoreShardDone completed 2023-10-21T04:25:20.977+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [2864853750198994630 12177293282243154215], elapsed: 395.844µs 2023-10-21T04:25:20.977+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[2864853750198994630:true 12177293282243154215:true] 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 4 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 5 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 6 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 7 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 8 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 1 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 2 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 7315818257769233877, partnId: 3 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 14271070195512901425, partnId: 0 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 6 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 7 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 8 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 1 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 2 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 3 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 4 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9912003609070511414, partnId: 5 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15667952562388529486, partnId: 0 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12004664282277897596, partnId: 0 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17394655409336449770, partnId: 0 2023-10-21T04:25:20.977+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 5004576769623010783, partnId: 0 2023-10-21T04:25:20.977+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.42µs 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data(shard2864853750198994630) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630/data/recovery(shard2864853750198994630) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data(shard12177293282243154215) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215/data/recovery(shard12177293282243154215) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:20.977+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [2864853750198994630 12177293282243154215] 2023-10-21T04:25:20.977+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [2864853750198994630 12177293282243154215], errMap: map[2864853750198994630: 12177293282243154215:], elapsed: 9.342µs 2023-10-21T04:25:20.978+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:25:20.978+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:25:20.978+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_DONE] 2023-10-21T04:25:20.978+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 28.361µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:21.017+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken12:9b:ba:a8:e5:6a:47:bd MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 Shards: [2864853750198994630 12177293282243154215] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9885812108833241320 RealInstId: 7315818257769233877 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 12004664282277897596 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 17394655409336449770 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 6611915605599465689 RealInstId: 9912003609070511414 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 15667952562388529486 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 14271070195512901425 RealInstId: 0 Name: #primary_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 5004576769623010783 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:21.048+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:-1 bada28845870208460ab46b431032b01:0 e28a1725ce62fe1e923bede04f38724f:-1 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:21.048+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardToken36:37:b8:9d:2e:c3:90:4a is in state: ShardTokenCommit 2023-10-21T04:25:21.048+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken12:9b:ba:a8:e5:6a:47:bd 2023-10-21T04:25:21.072+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9906816080753567, throttleDelayMs (new, old, change): (9, 5, 4) 2023-10-21T04:25:21.111+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:21.111+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [10890719122371003479 7659339677706884510] 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : all deamons started 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : all deamons started 2023-10-21T04:25:21.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:25:21.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:25:21.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : RestoreShardDone completed 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : all deamons started 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : all deamons started 2023-10-21T04:25:21.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:25:21.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:25:21.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : RestoreShardDone completed 2023-10-21T04:25:21.111+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [10890719122371003479 7659339677706884510], elapsed: 336.46µs 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data(shard10890719122371003479) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479/data/recovery(shard10890719122371003479) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data(shard7659339677706884510) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510/data/recovery(shard7659339677706884510) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:25:21.111+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[7659339677706884510:true 10890719122371003479:true] 2023-10-21T04:25:21.111+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 6 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 7 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 8 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 1 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 2 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 3 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 4 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 13987057765505463258, partnId: 5 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16733302726553098964, partnId: 0 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10979931881244616620, partnId: 0 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 1094343481594206610, partnId: 0 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 4 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 5 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 6 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 7 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 8 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 1 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 2 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 11460826864060385936, partnId: 3 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4992762551574016313, partnId: 0 2023-10-21T04:25:21.112+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2296526085582506469, partnId: 0 2023-10-21T04:25:21.112+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.957µs 2023-10-21T04:25:21.113+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [10890719122371003479 7659339677706884510] 2023-10-21T04:25:21.113+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [10890719122371003479 7659339677706884510], errMap: map[7659339677706884510: 10890719122371003479:], elapsed: 9.828µs 2023-10-21T04:25:21.113+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:25:21.113+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:25:21.113+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_DONE] 2023-10-21T04:25:21.113+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 31.95µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:21.167+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken34:c1:80:2c:eb:79:82:c5 MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: f22df9bc00cfefa0906def8153acae8a (127.0.0.1:9002) DestId: 2298f6cccd108c9b6151cc01f5a848f9 (127.0.0.1:9003) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1094343481594206610 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 13895180128994009539 RealInstId: 11460826864060385936 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 10979931881244616620 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 12820935061706293349 RealInstId: 13987057765505463258 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2296526085582506469 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 16733302726553098964 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 4992762551574016313 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:21.206+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:-2 bada28845870208460ab46b431032b01:0 e28a1725ce62fe1e923bede04f38724f:-1 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:21.206+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardToken36:37:b8:9d:2e:c3:90:4a is in state: ShardTokenCommit 2023-10-21T04:25:21.206+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken34:c1:80:2c:eb:79:82:c5 2023-10-21T04:25:21.230+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken36:37:b8:9d:2e:c3:90:4a MasterId: 2298f6cccd108c9b6151cc01f5a848f9 SourceId: bada28845870208460ab46b431032b01 (127.0.0.1:9001) DestId: e28a1725ce62fe1e923bede04f38724f (127.0.0.1:9004) RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [10890719122371003479 7659339677706884510] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14296137856158897341 RealInstId: 0 Name: idx_secondary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 9963702112820124680 RealInstId: 15633219898206385003 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 791854418175666346 RealInstId: 12882995014846270330 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [1 2 3 4 5 6 7 8] Versions: [6 6 6 6 6 6 6 6] InstId: 2281164589865190206 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [1] InstId: 1757261333586968909 RealInstId: 0 Name: #primary_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] InstId: 3539475206979968695 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [6] InstId: 620420137039842024 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: _default/0 Partitions: [0] Versions: [6] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:21.239+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[2298f6cccd108c9b6151cc01f5a848f9:-2 bada28845870208460ab46b431032b01:0 e28a1725ce62fe1e923bede04f38724f:-2 f22df9bc00cfefa0906def8153acae8a:0], batchSize: 2 2023-10-21T04:25:21.239+05:30 [Info] ShardRebalancer::processShardTransferTokenAsMaster No Tokens Found. Mark Done. 2023-10-21T04:25:21.239+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 7] call. taskList: &{Rev:[0 0 0 0 0 0 0 8] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1972139801924f1fdf1188af4e7137b9 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]} {Rev:[0 0 0 0 0 0 0 5] ID:rebalance/1972139801924f1fdf1188af4e7137b9 Type:task-rebalance Status:task-running IsCancelable:true Progress:1 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1972139801924f1fdf1188af4e7137b9]}]} 2023-10-21T04:25:21.239+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 8] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:21.240+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:25:21.276+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 8] 2023-10-21T04:25:21.281+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 8] 2023-10-21T04:25:21.313+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:25:21.313+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:25:21.313+05:30 [Info] ShardRebalancer::updateProgress Done Received 2023-10-21T04:25:21.313+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:25:21.313+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:25:21.313+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress done received. progress collector stopped 2023-10-21T04:25:21.326+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {2298f6cccd108c9b6151cc01f5a848f9 1972139801924f1fdf1188af4e7137b9 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:25:21.356+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:25:21.356+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken6a:a7:ac:ec:fd:3f:d4:ed MasterId: SourceId: DestId: RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokened:42:e0:2a:27:fe:e0:80 SiblingTokenId: ShardToken12:9b:ba:a8:e5:6a:47:bd Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:21.356+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken8:4b:39:d1:66:4f:1c:93 MasterId: SourceId: DestId: RebalId: 1972139801924f1fdf1188af4e7137b9 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken36:37:b8:9d:2e:c3:90:4a SiblingTokenId: ShardToken34:c1:80:2c:eb:79:82:c5 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:25:21.387+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:25:21.387+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:25:21.387+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.631µs 2023-10-21T04:25:21.387+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:25:21.387+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:25:21.387+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:25:21.391+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:25:21.391+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:25:21.391+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:25:21.391+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:25:21.391+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 28.939µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:21.394+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:25:21.394+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:25:21.394+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster true, forceUnbalanced false, err: 2023-10-21T04:25:21.394+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 8] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:25:21.394+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:21.395+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:25:21.423+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:25:21.428+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 9] 2023-10-21T04:25:21.451+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:25:21.451+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:25:21.466+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:25:21.466+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:21.618+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:25:21.625+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 9] 2023-10-21T04:25:21.960+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:25:21.960+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:25:21.969+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:25:22.087+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9906382437638278, throttleDelayMs (new, old, change): (10, 9, 1) 2023-10-21T04:25:22.480+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9002. 2023-10-21T04:25:22.480+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 5 != len(c.nodesvs): 4 2023-10-21T04:25:22.480+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:25:22.572+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:42072 2023-10-21T04:25:22.572+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:25:22.625+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:25:22.625+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:22.625+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:22.625+05:30 [Info] UnwatchMetadata(): indexer f22df9bc00cfefa0906def8153acae8a 2023-10-21T04:25:22.625+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:25:22.625+05:30 [Info] UnwatchMetadata(): finish for indexer f22df9bc00cfefa0906def8153acae8a 2023-10-21T04:25:22.625+05:30 [Info] UnwatchMetadata(): indexer bada28845870208460ab46b431032b01 2023-10-21T04:25:22.625+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:25:22.626+05:30 [Info] UnwatchMetadata(): finish for indexer bada28845870208460ab46b431032b01 2023-10-21T04:25:22.626+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:60496->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:25:22.626+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:60030->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:25:22.726+05:30 [Info] switched currmeta from 226 -> 228 force true 2023-10-21T04:25:22.726+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:25:22.726+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] ... stopped 2023-10-21T04:25:22.726+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:25:22.726+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] ... stopped 2023-10-21T04:25:22.762+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Stopping releaseConnsRoutine 2023-10-21T04:25:22.768+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Stopping releaseConnsRoutine 2023-10-21T04:25:22.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:52146 closed 2023-10-21T04:25:22.943+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:40516. Error = read tcp4 127.0.0.1:9118->127.0.0.1:40516: read: connection reset by peer. Kill Pipe. 2023-10-21T04:25:22.943+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4c:d8:28:e5:1e:6b:c5:36 as follower. 2023-10-21T04:25:23.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9999530767629122 2023-10-21T04:25:23.286+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 704.787067ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:23.286+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:25:23.286+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:25:23.286+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:25:23.295+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:25:23.297+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:25:23.397+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:23.397+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:23.400+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:25:23.480+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:23.598+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40548. Reason = EOF. Cleaning Pipe. 2023-10-21T04:25:23.599+05:30 [Info] messageListener.start(): message channel closed. Remove peer 57:e3:77:c6:cc:fe:6f:46 as follower. 2023-10-21T04:25:23.604+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:52180 closed 2023-10-21T04:25:23.608+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9001. 2023-10-21T04:25:23.618+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9001. 2023-10-21T04:25:23.699+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.699+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(1) 2023-10-21T04:25:23.775+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.775+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:25:23.787+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:25:23.796+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.796+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(2) 2023-10-21T04:25:23.800+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 143] updated cluster rebalancing status to false 2023-10-21T04:25:23.861+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.861+05:30 [Info] Validation Failed for cluster info.. Retrying(2) 2023-10-21T04:25:23.880+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.880+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:25:23.896+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.896+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(3) 2023-10-21T04:25:23.906+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.906+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(4) 2023-10-21T04:25:23.918+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.918+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(5) 2023-10-21T04:25:23.925+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.925+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(6) 2023-10-21T04:25:23.945+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.945+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(7) 2023-10-21T04:25:23.950+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.950+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(8) 2023-10-21T04:25:23.954+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.954+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(9) 2023-10-21T04:25:23.964+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.964+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(10) 2023-10-21T04:25:23.966+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.966+05:30 [Info] Validation Failed for cluster info.. Retrying(3) 2023-10-21T04:25:23.970+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.970+05:30 [Info] Validation Failed for cluster info.. Retrying(2) 2023-10-21T04:25:23.991+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:25:23.991+05:30 [Error] Validation Failed while updating nodes and nodesvs.. &{{{1 0} 0 0 {{} -1073741824} {{} 0}} http://%40index:0fc409ca0877ca88271088dd5d65e3c1@127.0.0.1:9003 default updateIndexerList 300 2000000000 1 false map[] {0xc00c5a82d0 0xc004615760 {map[ale:0.0.0 asn1:5.0.21 chronicle:0.0.1 crypto:5.1.4 esaml:4.4.0 inets:8.3.1 kernel:8.5.4 lhttpc:1.3.0 ns_server:7.6.0-1559-rel-EE-enterprise os_mon:2.8.2 public_key:1.13.3 sasl:4.2 ssl:10.9.1 stdlib:4.3.1.1 xmerl:1.3.31.1] 7.6.0-1559-rel-EE-enterprise true 73d3202e218e8e8b3a6d8b83ef2f6375 [{default /poolsStreaming/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375 /pools/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375}] false} updateIndexerList} {map[] [{458758 active 172.31.5.112:9000 map[cmd_get:0 couch_docs_actual_disk_size:1.913216e+06 couch_docs_data_size:1.540447e+06 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:2048 curr_items_tot:2048 ep_bg_fetched:0 get_hits:0 mem_used:8.79496e+06 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 9.6191488e+09 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12000 distTCP:21403 distTLS:21453 httpsMgmt:19000] healthy 1464 7.6.0-1559-rel-EE-enterprise false [kv n1ql] e3997f8ca787b6ec4f8b06cf33f6aa3e inet 20013814 Group 1} {458758 active 127.0.0.1:9003 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.25485e+06 index_disk_size:4.4598183e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 9.62019328e+09 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12006 distTCP:21403 distTLS:21453 httpsMgmt:19003] healthy 1458 7.6.0-1559-rel-EE-enterprise true [index] 2298f6cccd108c9b6151cc01f5a848f9 inet 134147881 Group 2} {458758 active 127.0.0.1:9004 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.259544e+06 index_disk_size:4.4575109e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 9.60899072e+09 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12008 distTCP:21403 distTLS:21453 httpsMgmt:19004] healthy 1453 7.6.0-1559-rel-EE-enterprise false [index] e28a1725ce62fe1e923bede04f38724f inet 30040301 Group 1}] map[] map[terseBucketsBase:/pools/default/b/ terseStreamingBucketsBase:/pools/default/bs/ uri:/pools/default/buckets?v=45061250&uuid=73d3202e218e8e8b3a6d8b83ef2f6375] /pools/default/nodeServices?v=8773313 /pools/default/serverGroups?v=76905227 [{bucket_%2 7007f0ab442ac00e52d057a30cd22644} {bucket_1 db19c9050b4ab1232e6bc6654011618f}] {0xc00c5a82d0 0xc004615760 {map[ale:0.0.0 asn1:5.0.21 chronicle:0.0.1 crypto:5.1.4 esaml:4.4.0 inets:8.3.1 kernel:8.5.4 lhttpc:1.3.0 ns_server:7.6.0-1559-rel-EE-enterprise os_mon:2.8.2 public_key:1.13.3 sasl:4.2 ssl:10.9.1 stdlib:4.3.1.1 xmerl:1.3.31.1] 7.6.0-1559-rel-EE-enterprise true 73d3202e218e8e8b3a6d8b83ef2f6375 [{default /poolsStreaming/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375 /pools/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375}] false} updateIndexerList}} [{458758 active 172.31.5.112:9000 map[cmd_get:0 couch_docs_actual_disk_size:1.913216e+06 couch_docs_data_size:1.540447e+06 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:2048 curr_items_tot:2048 ep_bg_fetched:0 get_hits:0 mem_used:8.79496e+06 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 9.6191488e+09 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12000 distTCP:21403 distTLS:21453 httpsMgmt:19000] healthy 1464 7.6.0-1559-rel-EE-enterprise false [kv n1ql] e3997f8ca787b6ec4f8b06cf33f6aa3e inet 20013814 Group 1} {458758 active 127.0.0.1:9003 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.25485e+06 index_disk_size:4.4598183e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 9.62019328e+09 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12006 distTCP:21403 distTLS:21453 httpsMgmt:19003] healthy 1458 7.6.0-1559-rel-EE-enterprise true [index] 2298f6cccd108c9b6151cc01f5a848f9 inet 134147881 Group 2} {458758 active 127.0.0.1:9004 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.259544e+06 index_disk_size:4.4575109e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 9.60899072e+09 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12008 distTCP:21403 distTLS:21453 httpsMgmt:19004] healthy 1453 7.6.0-1559-rel-EE-enterprise false [index] e28a1725ce62fe1e923bede04f38724f inet 30040301 Group 1}] [{map[kv:12000 kvSSL:11998 mgmt:9000 mgmtSSL:19000 n1ql:9499 n1qlSSL:19499 projector:10000] 172.31.5.112 false} {map[mgmt:9001 mgmtSSL:19001] 127.0.0.1 false} {map[indexAdmin:9118 indexHttp:9120 indexHttps:19120 indexScan:9119 indexStreamCatchup:9122 indexStreamInit:9121 indexStreamMaint:9123 mgmt:9003 mgmtSSL:19003] 127.0.0.1 true} {map[indexAdmin:9124 indexHttp:9126 indexHttps:19126 indexScan:9125 indexStreamCatchup:9128 indexStreamInit:9127 indexStreamMaint:9129 mgmt:9004 mgmtSSL:19004] 127.0.0.1 false}] map[] [] [] 7 6 map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999]} 2023-10-21T04:25:24.069+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9989976896923961 2023-10-21T04:25:25.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7911808691704301, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:25:25.770+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:25:25.770+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:25:25.770+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:25:25.772+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:25:25.772+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:25:25.772+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:25:25.772+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:25:25.772+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:25:25.772+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:42072. Reason = EOF. Cleaning Pipe. 2023-10-21T04:25:25.772+05:30 [Info] messageListener.start(): message channel closed. Remove peer ac:82:f0:aa:51:f6:b0:d5 as follower. 2023-10-21T04:25:25.773+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:42072->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:25:25.773+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:38494->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:25:25.992+05:30 [Info] Error occurred during nodes and nodesvs update (ClusterInfo Validation Failed) .. Retrying(1) 2023-10-21T04:25:26.008+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:25:26.008+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:26.008+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:26.010+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:26.010+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:26.010+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:25:26.020+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:26.020+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:26.024+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:26.036+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:26.038+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:26.044+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:26.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6758887600347906, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:25:26.091+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:25:26.091+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:26.099+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:26.099+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:26.106+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:26.118+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:26.118+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:26.123+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:26.155+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:26.160+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:26.160+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:26.163+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:27.490+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53890" doAuth() ... 2023-10-21T04:25:27.490+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53890" doReceive() ... 2023-10-21T04:25:27.505+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53894" doAuth() ... 2023-10-21T04:25:27.505+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53894" doReceive() ... 2023-10-21T04:25:27.530+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53898" doAuth() ... 2023-10-21T04:25:27.530+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53898" doReceive() ... 2023-10-21T04:25:27.555+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53922" doAuth() ... 2023-10-21T04:25:27.557+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53922" doReceive() ... 2023-10-21T04:25:27.569+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53926" doAuth() ... 2023-10-21T04:25:27.570+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53926" doReceive() ... 2023-10-21T04:25:27.593+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53928" doAuth() ... 2023-10-21T04:25:27.598+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53928" doReceive() ... 2023-10-21T04:25:27.609+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53936" doAuth() ... 2023-10-21T04:25:27.617+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53936" doReceive() ... 2023-10-21T04:25:27.629+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53938" doAuth() ... 2023-10-21T04:25:27.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53938" doReceive() ... 2023-10-21T04:25:27.639+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53944" doAuth() ... 2023-10-21T04:25:27.643+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53944" doReceive() ... 2023-10-21T04:25:27.658+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53950" doAuth() ... 2023-10-21T04:25:27.660+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53950" doReceive() ... 2023-10-21T04:25:27.664+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53952" doAuth() ... 2023-10-21T04:25:27.666+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53952" doReceive() ... 2023-10-21T04:25:27.774+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53958" doAuth() ... 2023-10-21T04:25:27.778+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53958" doReceive() ... 2023-10-21T04:25:27.806+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is enabled 2023-10-21T04:25:27.814+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:27.819+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53964" doAuth() ... 2023-10-21T04:25:27.820+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53964" doReceive() ... 2023-10-21T04:25:27.830+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:27.830+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:27.837+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:25:27.860+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53968" doAuth() ... 2023-10-21T04:25:27.861+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53968" doReceive() ... 2023-10-21T04:25:27.881+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53972" doAuth() ... 2023-10-21T04:25:27.883+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53972" doReceive() ... 2023-10-21T04:25:27.907+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53978" doAuth() ... 2023-10-21T04:25:27.910+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53978" doReceive() ... 2023-10-21T04:25:27.914+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53980" doAuth() ... 2023-10-21T04:25:27.914+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53980" doReceive() ... 2023-10-21T04:25:27.930+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53982" doAuth() ... 2023-10-21T04:25:27.931+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53982" doReceive() ... 2023-10-21T04:25:27.952+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53984" doAuth() ... 2023-10-21T04:25:27.953+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53984" doReceive() ... 2023-10-21T04:25:27.978+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53990" doAuth() ... 2023-10-21T04:25:27.978+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53990" doReceive() ... 2023-10-21T04:25:27.987+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53994" doAuth() ... 2023-10-21T04:25:27.988+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:53994" doReceive() ... 2023-10-21T04:25:28.076+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54006" doAuth() ... 2023-10-21T04:25:28.078+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54006" doReceive() ... 2023-10-21T04:25:28.200+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54018" doAuth() ... 2023-10-21T04:25:28.201+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54018" doReceive() ... 2023-10-21T04:25:28.227+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54026" doAuth() ... 2023-10-21T04:25:28.228+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54026" doReceive() ... 2023-10-21T04:25:28.262+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54030" doAuth() ... 2023-10-21T04:25:28.269+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54030" doReceive() ... 2023-10-21T04:25:28.283+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54034" doAuth() ... 2023-10-21T04:25:28.284+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54034" doReceive() ... 2023-10-21T04:25:28.286+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54036" doAuth() ... 2023-10-21T04:25:28.288+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54036" doReceive() ... 2023-10-21T04:25:28.289+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54038" doAuth() ... 2023-10-21T04:25:28.290+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54038" doReceive() ... 2023-10-21T04:25:28.293+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54040" doAuth() ... 2023-10-21T04:25:28.297+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54040" doReceive() ... 2023-10-21T04:25:28.309+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54042" doAuth() ... 2023-10-21T04:25:28.310+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54042" doReceive() ... 2023-10-21T04:25:28.324+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54044" doAuth() ... 2023-10-21T04:25:28.325+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54044" doReceive() ... 2023-10-21T04:25:28.397+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54054" doAuth() ... 2023-10-21T04:25:28.398+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54054" doReceive() ... 2023-10-21T04:25:28.456+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54068" doAuth() ... 2023-10-21T04:25:28.456+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54068" doReceive() ... 2023-10-21T04:25:28.459+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54070" doAuth() ... 2023-10-21T04:25:28.459+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54070" doReceive() ... 2023-10-21T04:25:28.462+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54074" doAuth() ... 2023-10-21T04:25:28.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54074" doReceive() ... 2023-10-21T04:25:28.503+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54080" doAuth() ... 2023-10-21T04:25:28.503+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54080" doReceive() ... 2023-10-21T04:25:28.550+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54092" doAuth() ... 2023-10-21T04:25:28.551+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54092" doReceive() ... 2023-10-21T04:25:28.575+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54098" doAuth() ... 2023-10-21T04:25:28.576+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54098" doReceive() ... 2023-10-21T04:25:28.588+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54100" doAuth() ... 2023-10-21T04:25:28.590+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54100" doReceive() ... 2023-10-21T04:25:28.602+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54104" doAuth() ... 2023-10-21T04:25:28.603+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54104" doReceive() ... 2023-10-21T04:25:28.606+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54106" doAuth() ... 2023-10-21T04:25:28.607+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54106" doReceive() ... 2023-10-21T04:25:28.628+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54112" doAuth() ... 2023-10-21T04:25:28.629+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54112" doReceive() ... 2023-10-21T04:25:28.632+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54114" doAuth() ... 2023-10-21T04:25:28.633+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54114" doReceive() ... 2023-10-21T04:25:28.644+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54116" doAuth() ... 2023-10-21T04:25:28.644+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54116" doReceive() ... 2023-10-21T04:25:28.664+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54118" doAuth() ... 2023-10-21T04:25:28.664+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54118" doReceive() ... 2023-10-21T04:25:28.682+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54124" doAuth() ... 2023-10-21T04:25:28.683+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54124" doReceive() ... 2023-10-21T04:25:28.748+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54128" doAuth() ... 2023-10-21T04:25:28.751+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54128" doReceive() ... 2023-10-21T04:25:28.757+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54130" doAuth() ... 2023-10-21T04:25:28.759+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54130" doReceive() ... 2023-10-21T04:25:28.764+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54132" doAuth() ... 2023-10-21T04:25:28.769+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54132" doReceive() ... 2023-10-21T04:25:28.784+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54136" doAuth() ... 2023-10-21T04:25:28.787+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54136" doReceive() ... 2023-10-21T04:25:28.802+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54138" doAuth() ... 2023-10-21T04:25:28.803+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54138" doReceive() ... 2023-10-21T04:25:28.809+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54144" doAuth() ... 2023-10-21T04:25:28.809+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54144" doReceive() ... 2023-10-21T04:25:28.812+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54146" doAuth() ... 2023-10-21T04:25:28.813+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54146" doReceive() ... 2023-10-21T04:25:28.841+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54150" doAuth() ... 2023-10-21T04:25:28.841+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54150" doReceive() ... 2023-10-21T04:25:28.917+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54160" doAuth() ... 2023-10-21T04:25:28.917+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54160" doReceive() ... 2023-10-21T04:25:28.960+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54166" doAuth() ... 2023-10-21T04:25:28.961+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54166" doReceive() ... 2023-10-21T04:25:28.989+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54182" doAuth() ... 2023-10-21T04:25:28.990+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54182" doReceive() ... 2023-10-21T04:25:29.006+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54188" doAuth() ... 2023-10-21T04:25:29.007+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54188" doReceive() ... 2023-10-21T04:25:29.051+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:25:29.072+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9599533442839256, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:25:29.110+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54200" doAuth() ... 2023-10-21T04:25:29.112+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54200" doReceive() ... 2023-10-21T04:25:29.187+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54208" doAuth() ... 2023-10-21T04:25:29.188+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54208" doReceive() ... 2023-10-21T04:25:29.245+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54218" doAuth() ... 2023-10-21T04:25:29.248+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54218" doReceive() ... 2023-10-21T04:25:29.263+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54222" doAuth() ... 2023-10-21T04:25:29.265+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54222" doReceive() ... 2023-10-21T04:25:29.410+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54246" doAuth() ... 2023-10-21T04:25:29.413+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54246" doReceive() ... 2023-10-21T04:25:29.417+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54248" doAuth() ... 2023-10-21T04:25:29.425+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54248" doReceive() ... 2023-10-21T04:25:29.453+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54252" doAuth() ... 2023-10-21T04:25:29.458+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54252" doReceive() ... 2023-10-21T04:25:29.891+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54272" doAuth() ... 2023-10-21T04:25:29.893+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54272" doReceive() ... 2023-10-21T04:25:29.919+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54274" doAuth() ... 2023-10-21T04:25:29.928+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54274" doReceive() ... 2023-10-21T04:25:29.930+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54276" doAuth() ... 2023-10-21T04:25:29.932+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54276" doReceive() ... 2023-10-21T04:25:29.935+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54278" doAuth() ... 2023-10-21T04:25:29.937+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54278" doReceive() ... 2023-10-21T04:25:29.944+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54282" doAuth() ... 2023-10-21T04:25:29.948+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54282" doReceive() ... 2023-10-21T04:25:29.951+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54284" doAuth() ... 2023-10-21T04:25:29.953+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54284" doReceive() ... 2023-10-21T04:25:29.957+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54286" doAuth() ... 2023-10-21T04:25:29.960+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54286" doReceive() ... 2023-10-21T04:25:29.963+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54288" doAuth() ... 2023-10-21T04:25:29.966+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54288" doReceive() ... 2023-10-21T04:25:29.981+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54296" doAuth() ... 2023-10-21T04:25:29.981+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54296" doReceive() ... 2023-10-21T04:25:29.990+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54298" doAuth() ... 2023-10-21T04:25:29.991+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54298" doReceive() ... 2023-10-21T04:25:30.013+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54306" doAuth() ... 2023-10-21T04:25:30.016+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54306" doReceive() ... 2023-10-21T04:25:30.051+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54312" doAuth() ... 2023-10-21T04:25:30.051+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54312" doReceive() ... 2023-10-21T04:25:30.055+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54314" doAuth() ... 2023-10-21T04:25:30.060+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54314" doReceive() ... 2023-10-21T04:25:30.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9434188507250529, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:25:30.090+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54316" doAuth() ... 2023-10-21T04:25:30.092+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54316" doReceive() ... 2023-10-21T04:25:30.097+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54318" doAuth() ... 2023-10-21T04:25:30.097+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54318" doReceive() ... 2023-10-21T04:25:30.127+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54328" doAuth() ... 2023-10-21T04:25:30.135+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54328" doReceive() ... 2023-10-21T04:25:30.196+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54332" doAuth() ... 2023-10-21T04:25:30.197+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54332" doReceive() ... 2023-10-21T04:25:30.200+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54334" doAuth() ... 2023-10-21T04:25:30.201+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54334" doReceive() ... 2023-10-21T04:25:30.289+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54340" doAuth() ... 2023-10-21T04:25:30.289+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54340" doReceive() ... 2023-10-21T04:25:30.295+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54342" doAuth() ... 2023-10-21T04:25:30.300+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54342" doReceive() ... 2023-10-21T04:25:30.394+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54360" doAuth() ... 2023-10-21T04:25:30.396+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54360" doReceive() ... 2023-10-21T04:25:30.514+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54380" doAuth() ... 2023-10-21T04:25:30.514+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54380" doReceive() ... 2023-10-21T04:25:30.519+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54382" doAuth() ... 2023-10-21T04:25:30.519+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54382" doReceive() ... 2023-10-21T04:25:30.539+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54386" doAuth() ... 2023-10-21T04:25:30.539+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54386" doReceive() ... 2023-10-21T04:25:30.563+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54396" doAuth() ... 2023-10-21T04:25:30.567+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54396" doReceive() ... 2023-10-21T04:25:30.596+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54404" doAuth() ... 2023-10-21T04:25:30.600+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54404" doReceive() ... 2023-10-21T04:25:30.644+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54412" doAuth() ... 2023-10-21T04:25:30.644+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54412" doReceive() ... 2023-10-21T04:25:30.688+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54414" doAuth() ... 2023-10-21T04:25:30.692+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54414" doReceive() ... 2023-10-21T04:25:30.736+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54418" doAuth() ... 2023-10-21T04:25:30.739+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54418" doReceive() ... 2023-10-21T04:25:30.761+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54424" doAuth() ... 2023-10-21T04:25:30.763+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54424" doReceive() ... 2023-10-21T04:25:30.924+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54434" doAuth() ... 2023-10-21T04:25:30.928+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54434" doReceive() ... 2023-10-21T04:25:30.940+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54436" doAuth() ... 2023-10-21T04:25:30.943+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54436" doReceive() ... 2023-10-21T04:25:30.949+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54438" doAuth() ... 2023-10-21T04:25:30.949+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54438" doReceive() ... 2023-10-21T04:25:30.953+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54440" doAuth() ... 2023-10-21T04:25:30.957+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54440" doReceive() ... 2023-10-21T04:25:30.962+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54442" doAuth() ... 2023-10-21T04:25:30.963+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54442" doReceive() ... 2023-10-21T04:25:30.997+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54450" doAuth() ... 2023-10-21T04:25:30.997+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54450" doReceive() ... 2023-10-21T04:25:31.000+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54452" doAuth() ... 2023-10-21T04:25:31.003+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54452" doReceive() ... 2023-10-21T04:25:31.006+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54454" doAuth() ... 2023-10-21T04:25:31.010+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54454" doReceive() ... 2023-10-21T04:25:31.041+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54458" doAuth() ... 2023-10-21T04:25:31.044+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54458" doReceive() ... 2023-10-21T04:25:31.169+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54468" doAuth() ... 2023-10-21T04:25:31.173+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54468" doReceive() ... 2023-10-21T04:25:31.190+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54474" doAuth() ... 2023-10-21T04:25:31.192+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54474" doReceive() ... 2023-10-21T04:25:31.195+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54476" doAuth() ... 2023-10-21T04:25:31.201+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54476" doReceive() ... 2023-10-21T04:25:31.228+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54480" doAuth() ... 2023-10-21T04:25:31.233+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54480" doReceive() ... 2023-10-21T04:25:31.350+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54492" doAuth() ... 2023-10-21T04:25:31.351+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54492" doReceive() ... 2023-10-21T04:25:31.393+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54500" doAuth() ... 2023-10-21T04:25:31.395+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54500" doReceive() ... 2023-10-21T04:25:31.398+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54502" doAuth() ... 2023-10-21T04:25:31.404+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54502" doReceive() ... 2023-10-21T04:25:31.416+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54506" doAuth() ... 2023-10-21T04:25:31.418+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54506" doReceive() ... 2023-10-21T04:25:31.436+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54508" doAuth() ... 2023-10-21T04:25:31.438+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54508" doReceive() ... 2023-10-21T04:25:31.460+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54518" doAuth() ... 2023-10-21T04:25:31.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54518" doReceive() ... 2023-10-21T04:25:31.478+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54520" doAuth() ... 2023-10-21T04:25:31.481+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54520" doReceive() ... 2023-10-21T04:25:31.485+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54522" doAuth() ... 2023-10-21T04:25:31.493+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54522" doReceive() ... 2023-10-21T04:25:31.552+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54528" doAuth() ... 2023-10-21T04:25:31.555+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54528" doReceive() ... 2023-10-21T04:25:31.558+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54530" doAuth() ... 2023-10-21T04:25:31.562+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54530" doReceive() ... 2023-10-21T04:25:31.627+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54542" doAuth() ... 2023-10-21T04:25:31.630+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54542" doReceive() ... 2023-10-21T04:25:31.673+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54546" doAuth() ... 2023-10-21T04:25:31.674+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54546" doReceive() ... 2023-10-21T04:25:31.677+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54548" doAuth() ... 2023-10-21T04:25:31.682+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54548" doReceive() ... 2023-10-21T04:25:34.529+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54696" doAuth() ... 2023-10-21T04:25:34.529+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54696" doReceive() ... 2023-10-21T04:25:34.546+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54702" doAuth() ... 2023-10-21T04:25:34.546+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54702" doReceive() ... 2023-10-21T04:25:34.552+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54704" doAuth() ... 2023-10-21T04:25:34.552+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54704" doReceive() ... 2023-10-21T04:25:34.581+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54710" doAuth() ... 2023-10-21T04:25:34.582+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54710" doReceive() ... 2023-10-21T04:25:34.647+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54720" doAuth() ... 2023-10-21T04:25:34.649+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54720" doReceive() ... 2023-10-21T04:25:34.683+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54722" doAuth() ... 2023-10-21T04:25:34.684+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54722" doReceive() ... 2023-10-21T04:25:34.719+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54730" doAuth() ... 2023-10-21T04:25:34.721+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54730" doReceive() ... 2023-10-21T04:25:34.755+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54738" doAuth() ... 2023-10-21T04:25:34.756+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54738" doReceive() ... 2023-10-21T04:25:34.857+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is enabled 2023-10-21T04:25:34.961+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54752" doAuth() ... 2023-10-21T04:25:34.962+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54752" doReceive() ... 2023-10-21T04:25:35.057+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54762" doAuth() ... 2023-10-21T04:25:35.059+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54762" doReceive() ... 2023-10-21T04:25:35.069+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9646410250382979, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:25:35.131+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54768" doAuth() ... 2023-10-21T04:25:35.132+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54768" doReceive() ... 2023-10-21T04:25:35.136+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54770" doAuth() ... 2023-10-21T04:25:35.139+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54770" doReceive() ... 2023-10-21T04:25:35.142+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54772" doAuth() ... 2023-10-21T04:25:35.143+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54772" doReceive() ... 2023-10-21T04:25:35.332+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54790" doAuth() ... 2023-10-21T04:25:35.335+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54790" doReceive() ... 2023-10-21T04:25:35.510+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54816" doAuth() ... 2023-10-21T04:25:35.510+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54816" doReceive() ... 2023-10-21T04:25:35.529+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54818" doAuth() ... 2023-10-21T04:25:35.530+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54818" doReceive() ... 2023-10-21T04:25:35.551+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54820" doAuth() ... 2023-10-21T04:25:35.552+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54820" doReceive() ... 2023-10-21T04:25:35.555+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54822" doAuth() ... 2023-10-21T04:25:35.556+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54822" doReceive() ... 2023-10-21T04:25:35.609+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54830" doAuth() ... 2023-10-21T04:25:35.610+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54830" doReceive() ... 2023-10-21T04:25:35.618+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54832" doAuth() ... 2023-10-21T04:25:35.625+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54832" doReceive() ... 2023-10-21T04:25:35.633+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54836" doAuth() ... 2023-10-21T04:25:35.643+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54836" doReceive() ... 2023-10-21T04:25:35.646+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54838" doAuth() ... 2023-10-21T04:25:35.647+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54838" doReceive() ... 2023-10-21T04:25:35.649+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54840" doAuth() ... 2023-10-21T04:25:35.652+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54840" doReceive() ... 2023-10-21T04:25:35.670+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54842" doAuth() ... 2023-10-21T04:25:35.673+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54842" doReceive() ... 2023-10-21T04:25:35.727+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54854" doAuth() ... 2023-10-21T04:25:35.728+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54854" doReceive() ... 2023-10-21T04:25:35.744+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54862" doAuth() ... 2023-10-21T04:25:35.745+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54862" doReceive() ... 2023-10-21T04:25:35.753+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54866" doAuth() ... 2023-10-21T04:25:35.754+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54866" doReceive() ... 2023-10-21T04:25:35.809+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54876" doAuth() ... 2023-10-21T04:25:35.810+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54876" doReceive() ... 2023-10-21T04:25:35.920+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54888" doAuth() ... 2023-10-21T04:25:35.921+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54888" doReceive() ... 2023-10-21T04:25:35.924+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54890" doAuth() ... 2023-10-21T04:25:35.925+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54890" doReceive() ... 2023-10-21T04:25:36.071+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9672633874298643, throttleDelayMs (new, old, change): (3, 1, 2) 2023-10-21T04:25:36.136+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54910" doAuth() ... 2023-10-21T04:25:36.136+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54910" doReceive() ... 2023-10-21T04:25:36.155+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54912" doAuth() ... 2023-10-21T04:25:36.158+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54912" doReceive() ... 2023-10-21T04:25:36.335+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54938" doAuth() ... 2023-10-21T04:25:36.336+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54938" doReceive() ... 2023-10-21T04:25:36.340+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54940" doAuth() ... 2023-10-21T04:25:36.340+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54940" doReceive() ... 2023-10-21T04:25:36.343+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54942" doAuth() ... 2023-10-21T04:25:36.344+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54942" doReceive() ... 2023-10-21T04:25:36.549+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54958" doAuth() ... 2023-10-21T04:25:36.550+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54958" doReceive() ... 2023-10-21T04:25:37.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9635577326333227, throttleDelayMs (new, old, change): (4, 3, 1) 2023-10-21T04:25:37.165+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54998" doAuth() ... 2023-10-21T04:25:37.167+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:54998" doReceive() ... 2023-10-21T04:25:37.211+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55000" doAuth() ... 2023-10-21T04:25:37.213+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55000" doReceive() ... 2023-10-21T04:25:37.218+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55002" doAuth() ... 2023-10-21T04:25:37.219+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55002" doReceive() ... 2023-10-21T04:25:37.275+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55008" doAuth() ... 2023-10-21T04:25:37.276+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55008" doReceive() ... 2023-10-21T04:25:37.342+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55014" doAuth() ... 2023-10-21T04:25:37.345+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55014" doReceive() ... 2023-10-21T04:25:37.354+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55018" doAuth() ... 2023-10-21T04:25:37.355+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55018" doReceive() ... 2023-10-21T04:25:37.464+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55026" doAuth() ... 2023-10-21T04:25:37.470+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55026" doReceive() ... 2023-10-21T04:25:37.514+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55034" doAuth() ... 2023-10-21T04:25:37.516+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55034" doReceive() ... 2023-10-21T04:25:37.564+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55040" doAuth() ... 2023-10-21T04:25:37.566+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55040" doReceive() ... 2023-10-21T04:25:37.696+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55052" doAuth() ... 2023-10-21T04:25:37.697+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55052" doReceive() ... 2023-10-21T04:25:37.716+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55056" doAuth() ... 2023-10-21T04:25:37.717+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55056" doReceive() ... 2023-10-21T04:25:37.797+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55060" doAuth() ... 2023-10-21T04:25:37.797+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55060" doReceive() ... 2023-10-21T04:25:37.812+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55062" doAuth() ... 2023-10-21T04:25:37.813+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55062" doReceive() ... 2023-10-21T04:25:37.815+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55064" doAuth() ... 2023-10-21T04:25:37.819+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55064" doReceive() ... 2023-10-21T04:25:37.856+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55070" doAuth() ... 2023-10-21T04:25:37.857+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55070" doReceive() ... 2023-10-21T04:25:37.907+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55074" doAuth() ... 2023-10-21T04:25:37.908+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55074" doReceive() ... 2023-10-21T04:25:37.937+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55080" doAuth() ... 2023-10-21T04:25:37.937+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55080" doReceive() ... 2023-10-21T04:25:37.964+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55084" doAuth() ... 2023-10-21T04:25:37.965+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55084" doReceive() ... 2023-10-21T04:25:38.003+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55088" doAuth() ... 2023-10-21T04:25:38.009+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55088" doReceive() ... 2023-10-21T04:25:38.019+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55094" doAuth() ... 2023-10-21T04:25:38.019+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55094" doReceive() ... 2023-10-21T04:25:38.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9409177444775223, throttleDelayMs (new, old, change): (3, 4, -1) 2023-10-21T04:25:38.071+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55108" doAuth() ... 2023-10-21T04:25:38.071+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55108" doReceive() ... 2023-10-21T04:25:38.113+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55134" doAuth() ... 2023-10-21T04:25:38.113+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55134" doReceive() ... 2023-10-21T04:25:38.133+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55138" doAuth() ... 2023-10-21T04:25:38.133+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55138" doReceive() ... 2023-10-21T04:25:38.137+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55140" doAuth() ... 2023-10-21T04:25:38.138+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55140" doReceive() ... 2023-10-21T04:25:38.160+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55144" doAuth() ... 2023-10-21T04:25:38.160+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55144" doReceive() ... 2023-10-21T04:25:38.213+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55148" doAuth() ... 2023-10-21T04:25:38.214+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55148" doReceive() ... 2023-10-21T04:25:38.232+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55150" doAuth() ... 2023-10-21T04:25:38.233+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55150" doReceive() ... 2023-10-21T04:25:38.250+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55152" doAuth() ... 2023-10-21T04:25:38.251+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55152" doReceive() ... 2023-10-21T04:25:38.254+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55154" doAuth() ... 2023-10-21T04:25:38.254+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55154" doReceive() ... 2023-10-21T04:25:38.305+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55160" doAuth() ... 2023-10-21T04:25:38.309+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55160" doReceive() ... 2023-10-21T04:25:38.320+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55164" doAuth() ... 2023-10-21T04:25:38.325+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55164" doReceive() ... 2023-10-21T04:25:38.385+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55174" doAuth() ... 2023-10-21T04:25:38.388+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55174" doReceive() ... 2023-10-21T04:25:38.391+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55176" doAuth() ... 2023-10-21T04:25:38.393+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55176" doReceive() ... 2023-10-21T04:25:38.495+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55186" doAuth() ... 2023-10-21T04:25:38.495+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55186" doReceive() ... 2023-10-21T04:25:38.521+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55192" doAuth() ... 2023-10-21T04:25:38.522+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55192" doReceive() ... 2023-10-21T04:25:38.548+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55198" doAuth() ... 2023-10-21T04:25:38.551+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55198" doReceive() ... 2023-10-21T04:25:38.554+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55200" doAuth() ... 2023-10-21T04:25:38.556+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55200" doReceive() ... 2023-10-21T04:25:38.592+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55206" doAuth() ... 2023-10-21T04:25:38.597+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55206" doReceive() ... 2023-10-21T04:25:38.601+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55208" doAuth() ... 2023-10-21T04:25:38.605+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55208" doReceive() ... 2023-10-21T04:25:38.663+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55214" doAuth() ... 2023-10-21T04:25:38.667+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55214" doReceive() ... 2023-10-21T04:25:38.672+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55216" doAuth() ... 2023-10-21T04:25:38.674+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55216" doReceive() ... 2023-10-21T04:25:38.678+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55218" doAuth() ... 2023-10-21T04:25:38.680+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55218" doReceive() ... 2023-10-21T04:25:38.695+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55222" doAuth() ... 2023-10-21T04:25:38.697+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55222" doReceive() ... 2023-10-21T04:25:38.699+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55224" doAuth() ... 2023-10-21T04:25:38.708+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55224" doReceive() ... 2023-10-21T04:25:38.712+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55230" doAuth() ... 2023-10-21T04:25:38.712+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55230" doReceive() ... 2023-10-21T04:25:38.739+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55232" doAuth() ... 2023-10-21T04:25:38.741+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55232" doReceive() ... 2023-10-21T04:25:38.807+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55242" doAuth() ... 2023-10-21T04:25:38.809+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55242" doReceive() ... 2023-10-21T04:25:38.866+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55248" doAuth() ... 2023-10-21T04:25:38.868+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55248" doReceive() ... 2023-10-21T04:25:38.916+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55254" doAuth() ... 2023-10-21T04:25:38.919+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55254" doReceive() ... 2023-10-21T04:25:38.922+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55256" doAuth() ... 2023-10-21T04:25:38.931+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55256" doReceive() ... 2023-10-21T04:25:38.938+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55258" doAuth() ... 2023-10-21T04:25:38.938+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55258" doReceive() ... 2023-10-21T04:25:39.078+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55272" doAuth() ... 2023-10-21T04:25:39.082+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55272" doReceive() ... 2023-10-21T04:25:39.091+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55274" doAuth() ... 2023-10-21T04:25:39.094+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55274" doReceive() ... 2023-10-21T04:25:39.139+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55280" doAuth() ... 2023-10-21T04:25:39.141+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55280" doReceive() ... 2023-10-21T04:25:39.203+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55290" doAuth() ... 2023-10-21T04:25:39.206+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55290" doReceive() ... 2023-10-21T04:25:39.209+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55292" doAuth() ... 2023-10-21T04:25:39.211+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55292" doReceive() ... 2023-10-21T04:25:39.314+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55300" doAuth() ... 2023-10-21T04:25:39.317+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55300" doReceive() ... 2023-10-21T04:25:39.351+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55308" doAuth() ... 2023-10-21T04:25:39.354+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:55308" doReceive() ... 2023-10-21T04:25:41.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9311299071895, throttleDelayMs (new, old, change): (1, 3, -2) 2023-10-21T04:25:42.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9250306405933378, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:25:42.183+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 7315818257769233877 &{0} 2023-10-21T04:25:42.183+05:30 [Info] Indexer::handleDropIndex - IndexInstId 7315818257769233877 2023-10-21T04:25:42.184+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [7315818257769233877] 2023-10-21T04:25:42.184+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [7315818257769233877] 2023-10-21T04:25:42.184+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:42.185+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [7315818257769233877] 2023-10-21T04:25:42.186+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 4, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:42.186+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:42.186+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:42.186+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:42.186+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex 2023-10-21T04:25:42.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex stopped 2023-10-21T04:25:42.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.188+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 3, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:42.188+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 6, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.188+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.188+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex 2023-10-21T04:25:42.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex stopped 2023-10-21T04:25:42.188+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.188+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex 2023-10-21T04:25:42.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex stopped 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.188+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 5, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 7, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.188+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.188+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:42.188+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 8, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.188+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.189+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:42.189+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 7315818257769233877 Defn: DefnId: 17656144409930763141 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} 2023-10-21T04:25:42.189+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 5.803037ms 2023-10-21T04:25:42.189+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 5.845313ms 2023-10-21T04:25:42.189+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 1, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.189+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.189+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.189+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:42.189+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7315818257769233877, PartitionId 2, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.189+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.189+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.189+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:42.189+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 7315818257769233877 2023-10-21T04:25:42.188+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:42.192+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex 2023-10-21T04:25:42.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex stopped 2023-10-21T04:25:42.192+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.192+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.192+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.192+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:42.193+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:42.193+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:42.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex 2023-10-21T04:25:42.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex stopped 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:42.193+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:42.193+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:42.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex 2023-10-21T04:25:42.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex stopped 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:42.193+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:42.193+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:42.193+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex 2023-10-21T04:25:42.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex stopped 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.194+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.194+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:42.194+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:42.194+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:42.194+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex 2023-10-21T04:25:42.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex stopped 2023-10-21T04:25:42.194+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.271+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 124.719425ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:42.341+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.343+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex] 2023-10-21T04:25:42.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex closed 2023-10-21T04:25:42.378+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.378+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.378+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.378+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:42.378+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:42.378+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:42.379+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:42.379+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex 2023-10-21T04:25:42.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex stopped 2023-10-21T04:25:42.379+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.379+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.379+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.379+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.356+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex] 2023-10-21T04:25:42.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex closed 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:42.382+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:42.382+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:42.382+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:42.382+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex 2023-10-21T04:25:42.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex stopped 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.382+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:42.382+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex] 2023-10-21T04:25:42.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex closed 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:42.384+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:42.384+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:42.384+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:42.384+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex 2023-10-21T04:25:42.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex stopped 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.399+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex] 2023-10-21T04:25:42.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex closed 2023-10-21T04:25:42.399+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.399+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.399+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.399+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:42.399+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:42.399+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:42.399+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:42.399+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex 2023-10-21T04:25:42.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex stopped 2023-10-21T04:25:42.400+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.400+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.400+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.400+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.405+05:30 [Info] switched currmeta from 228 -> 232 force true 2023-10-21T04:25:42.414+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex] 2023-10-21T04:25:42.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex closed 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:42.415+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:42.415+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:42.415+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:42.415+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex 2023-10-21T04:25:42.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex stopped 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.415+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.419+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex] 2023-10-21T04:25:42.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex closed 2023-10-21T04:25:42.419+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.419+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.419+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.419+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:42.419+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:42.419+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:42.419+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:42.419+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex 2023-10-21T04:25:42.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex stopped 2023-10-21T04:25:42.420+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.420+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.420+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.420+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.450+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex] 2023-10-21T04:25:42.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex closed 2023-10-21T04:25:42.451+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.452+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.452+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.453+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex] 2023-10-21T04:25:42.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex closed 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:42.470+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:42.470+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:42.470+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:42.470+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex 2023-10-21T04:25:42.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex stopped 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:42.479+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:42.479+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:42.479+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:42.479+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex 2023-10-21T04:25:42.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex stopped 2023-10-21T04:25:42.479+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.479+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.479+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.479+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex] 2023-10-21T04:25:42.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex closed 2023-10-21T04:25:42.497+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.497+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:25:42.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 199 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex ... 2023-10-21T04:25:42.506+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:25:42.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 221 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex ... 2023-10-21T04:25:42.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex] 2023-10-21T04:25:42.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex closed 2023-10-21T04:25:42.531+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.531+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:25:42.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 198 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex ... 2023-10-21T04:25:42.534+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 17394655409336449770 &{0} 2023-10-21T04:25:42.534+05:30 [Info] Indexer::handleDropIndex - IndexInstId 17394655409336449770 2023-10-21T04:25:42.534+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [17394655409336449770] 2023-10-21T04:25:42.535+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [17394655409336449770] 2023-10-21T04:25:42.535+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:42.536+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [17394655409336449770] 2023-10-21T04:25:42.537+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 17394655409336449770 Defn: DefnId: 8378417662949994532 Name: idx_secondary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:42.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.539+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17394655409336449770, PartitionId 0, IndexDefnId 8378417662949994532. 2023-10-21T04:25:42.541+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.541+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.549+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.550+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:42.551+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:42.550+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 16.198519ms 2023-10-21T04:25:42.550+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 16.240289ms 2023-10-21T04:25:42.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.550+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.550+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 17394655409336449770 2023-10-21T04:25:42.551+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_4.index destroyed 2023-10-21T04:25:42.552+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 4, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.553+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 4 Destroy Done 2023-10-21T04:25:42.567+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex] 2023-10-21T04:25:42.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex closed 2023-10-21T04:25:42.567+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.567+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:25:42.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 197 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex ... 2023-10-21T04:25:42.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:25:42.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 219 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex ... 2023-10-21T04:25:42.573+05:30 [Info] switched currmeta from 232 -> 233 force true 2023-10-21T04:25:42.575+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.575+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_2.index destroyed 2023-10-21T04:25:42.575+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 2, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.576+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 2 Destroy Done 2023-10-21T04:25:42.577+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.578+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:42.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex] 2023-10-21T04:25:42.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex closed 2023-10-21T04:25:42.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:25:42.580+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 220 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex ... 2023-10-21T04:25:42.581+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.581+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex] 2023-10-21T04:25:42.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex closed 2023-10-21T04:25:42.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex] 2023-10-21T04:25:42.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex closed 2023-10-21T04:25:42.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex] 2023-10-21T04:25:42.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex closed 2023-10-21T04:25:42.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex] 2023-10-21T04:25:42.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex closed 2023-10-21T04:25:42.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:25:42.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 201 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex ... 2023-10-21T04:25:42.583+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.587+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.582+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.587+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_3.index destroyed 2023-10-21T04:25:42.587+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 3, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.587+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 3 Destroy Done 2023-10-21T04:25:42.587+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:42.587+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:42.588+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:42.588+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex 2023-10-21T04:25:42.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex stopped 2023-10-21T04:25:42.588+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.588+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.588+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.588+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.585+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.589+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.585+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.590+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.611+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 179.352804ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:42.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:25:42.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 223 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex ... 2023-10-21T04:25:42.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:25:42.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 203 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex ... 2023-10-21T04:25:42.620+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:25:42.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 200 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex ... 2023-10-21T04:25:42.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:25:42.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 202 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex ... 2023-10-21T04:25:42.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:25:42.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 204 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex ... 2023-10-21T04:25:42.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index/mainIndex sucessfully destroyed 2023-10-21T04:25:42.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... 2023-10-21T04:25:42.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 226 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_6.index destroyed 2023-10-21T04:25:42.654+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 6, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.655+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 6 Destroy Done 2023-10-21T04:25:42.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index/mainIndex sucessfully destroyed 2023-10-21T04:25:42.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex] 2023-10-21T04:25:42.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex closed 2023-10-21T04:25:42.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... 2023-10-21T04:25:42.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 225 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_1.index destroyed 2023-10-21T04:25:42.660+05:30 [Info] bucket_1/idx_secondary_defer/Mainstore#17394655409336449770:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.661+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.661+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.661+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:42.661+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:42.661+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:42.661+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:42.661+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex 2023-10-21T04:25:42.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex stopped 2023-10-21T04:25:42.661+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.661+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.661+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.661+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.660+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 1, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.661+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 1 Destroy Done 2023-10-21T04:25:42.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_8.index destroyed 2023-10-21T04:25:42.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:25:42.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 222 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex ... 2023-10-21T04:25:42.663+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 8, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.663+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 8 Destroy Done 2023-10-21T04:25:42.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_5.index destroyed 2023-10-21T04:25:42.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:25:42.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 224 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex ... 2023-10-21T04:25:42.667+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex] 2023-10-21T04:25:42.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex closed 2023-10-21T04:25:42.667+05:30 [Info] bucket_1/idx_secondary_defer/Backstore#17394655409336449770:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.667+05:30 [Info] Indexer::cleanupIndexData IndexInst 17394655409336449770 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex ... 2023-10-21T04:25:42.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 205 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex ... 2023-10-21T04:25:42.667+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 5, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.668+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 5 Destroy Done 2023-10-21T04:25:42.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_7315818257769233877_7.index destroyed 2023-10-21T04:25:42.672+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7315818257769233877, PartitionId 7, IndexDefnId 17656144409930763141. 2023-10-21T04:25:42.672+05:30 [Info] Indexer::cleanupIndexData IndexInst 7315818257769233877 Partition 7 Destroy Done 2023-10-21T04:25:42.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex ... 2023-10-21T04:25:42.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 227 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex ... 2023-10-21T04:25:42.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_defer_17394655409336449770_0.index destroyed 2023-10-21T04:25:42.675+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17394655409336449770, PartitionId 0, IndexDefnId 8378417662949994532. 2023-10-21T04:25:42.676+05:30 [Info] Indexer::cleanupIndexData IndexInst 17394655409336449770 Partition 0 Destroy Done 2023-10-21T04:25:42.714+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:42.714+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:42.719+05:30 [Info] switched currmeta from 233 -> 236 force true 2023-10-21T04:25:42.753+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10979931881244616620 &{0} 2023-10-21T04:25:42.753+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10979931881244616620 2023-10-21T04:25:42.754+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:25:42.754+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:42.754+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10979931881244616620 Name: idx_secondary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:25:42.754+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10979931881244616620] 2023-10-21T04:25:42.754+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10979931881244616620] 2023-10-21T04:25:42.755+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:42.755+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10979931881244616620] 2023-10-21T04:25:42.755+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:42.755+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 10979931881244616620 Defn: DefnId: 7525085113233084253 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:42.756+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.153982ms 2023-10-21T04:25:42.756+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10979931881244616620, PartitionId 0, IndexDefnId 7525085113233084253. 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex] 2023-10-21T04:25:42.756+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex] 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex] 2023-10-21T04:25:42.756+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex 2023-10-21T04:25:42.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex stopped 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.756+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.756+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10979931881244616620 2023-10-21T04:25:42.760+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId Instances [10979931881244616620] 2023-10-21T04:25:42.766+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId 2023-10-21T04:25:42.766+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 35695 2023-10-21T04:25:42.769+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex] 2023-10-21T04:25:42.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex closed 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:42.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex] 2023-10-21T04:25:42.771+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex] 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex] 2023-10-21T04:25:42.771+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex 2023-10-21T04:25:42.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex stopped 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:42.771+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:42.771+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 15.108667ms 2023-10-21T04:25:42.780+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 26.09871ms 2023-10-21T04:25:42.782+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:42.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex] 2023-10-21T04:25:42.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex closed 2023-10-21T04:25:42.782+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:42.782+05:30 [Info] Indexer::cleanupIndexData IndexInst 10979931881244616620 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:42.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex ... 2023-10-21T04:25:42.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 191 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex ... 2023-10-21T04:25:42.788+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 40.18848ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:42.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/docIndex sucessfully destroyed 2023-10-21T04:25:42.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex ... 2023-10-21T04:25:42.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 213 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex ... 2023-10-21T04:25:42.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:42.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10979931881244616620_0.index destroyed 2023-10-21T04:25:42.799+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10979931881244616620, PartitionId 0, IndexDefnId 7525085113233084253. 2023-10-21T04:25:42.799+05:30 [Info] Indexer::cleanupIndexData IndexInst 10979931881244616620 Partition 0 Destroy Done 2023-10-21T04:25:42.834+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:42.834+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:42.847+05:30 [Info] switched currmeta from 236 -> 240 force true 2023-10-21T04:25:42.907+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:42.907+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:42.932+05:30 [Info] switched currmeta from 240 -> 240 force true 2023-10-21T04:25:43.025+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4992762551574016313 &{0} 2023-10-21T04:25:43.025+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4992762551574016313 2023-10-21T04:25:43.025+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:43.026+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:43.026+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4992762551574016313 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:43.026+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4992762551574016313] 2023-10-21T04:25:43.026+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4992762551574016313] 2023-10-21T04:25:43.026+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:43.026+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4992762551574016313] 2023-10-21T04:25:43.026+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:43.027+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4992762551574016313 Defn: DefnId: 3674078063353481907 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:43.027+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.44781ms 2023-10-21T04:25:43.027+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0, IndexDefnId 3674078063353481907. 2023-10-21T04:25:43.027+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:25:43.028+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:25:43.028+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex 2023-10-21T04:25:43.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex stopped 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.028+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.028+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId Instances [4992762551574016313] 2023-10-21T04:25:43.028+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4992762551574016313 2023-10-21T04:25:43.033+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId 2023-10-21T04:25:43.033+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 35695 2023-10-21T04:25:43.033+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.219506ms 2023-10-21T04:25:43.033+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 7.730487ms 2023-10-21T04:25:43.033+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex] 2023-10-21T04:25:43.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex closed 2023-10-21T04:25:43.033+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.033+05:30 [Info] Indexer::cleanupIndexData IndexInst 4992762551574016313 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:43.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex ... 2023-10-21T04:25:43.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex ... 2023-10-21T04:25:43.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_4992762551574016313_0.index destroyed 2023-10-21T04:25:43.037+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4992762551574016313, PartitionId 0, IndexDefnId 3674078063353481907. 2023-10-21T04:25:43.037+05:30 [Info] Indexer::cleanupIndexData IndexInst 4992762551574016313 Partition 0 Destroy Done 2023-10-21T04:25:43.050+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 32.193317ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:43.069+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9596761368069183, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:25:43.200+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 16733302726553098964 &{0} 2023-10-21T04:25:43.200+05:30 [Info] Indexer::handleDropIndex - IndexInstId 16733302726553098964 2023-10-21T04:25:43.201+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [16733302726553098964] 2023-10-21T04:25:43.201+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [16733302726553098964] 2023-10-21T04:25:43.201+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:43.201+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [16733302726553098964] 2023-10-21T04:25:43.201+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 16733302726553098964 Defn: DefnId: 13856704970496601958 Name: #primary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:43.201+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 802.368µs 2023-10-21T04:25:43.201+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 840.164µs 2023-10-21T04:25:43.201+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16733302726553098964, PartitionId 0, IndexDefnId 13856704970496601958. 2023-10-21T04:25:43.201+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.201+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.201+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:43.201+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:43.201+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:43.201+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:43.201+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex 2023-10-21T04:25:43.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex stopped 2023-10-21T04:25:43.202+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.202+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.202+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.202+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.202+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 16733302726553098964 2023-10-21T04:25:43.226+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex] 2023-10-21T04:25:43.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex closed 2023-10-21T04:25:43.227+05:30 [Info] bucket_%2/#primary_defer/Mainstore#16733302726553098964:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.227+05:30 [Info] Indexer::cleanupIndexData IndexInst 16733302726553098964 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:43.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:25:43.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 228 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex ... 2023-10-21T04:25:43.231+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 38.441407ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:43.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_defer_16733302726553098964_0.index destroyed 2023-10-21T04:25:43.235+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16733302726553098964, PartitionId 0, IndexDefnId 13856704970496601958. 2023-10-21T04:25:43.235+05:30 [Info] Indexer::cleanupIndexData IndexInst 16733302726553098964 Partition 0 Destroy Done 2023-10-21T04:25:43.283+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.283+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.295+05:30 [Info] switched currmeta from 240 -> 248 force true 2023-10-21T04:25:43.308+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.308+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.321+05:30 [Info] switched currmeta from 248 -> 248 force true 2023-10-21T04:25:43.329+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:43.343+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.343+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.355+05:30 [Info] switched currmeta from 248 -> 248 force true 2023-10-21T04:25:43.381+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 1094343481594206610 &{0} 2023-10-21T04:25:43.381+05:30 [Info] Indexer::handleDropIndex - IndexInstId 1094343481594206610 2023-10-21T04:25:43.381+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [1094343481594206610] 2023-10-21T04:25:43.381+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [1094343481594206610] 2023-10-21T04:25:43.381+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:43.381+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [1094343481594206610] 2023-10-21T04:25:43.381+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 1094343481594206610 Defn: DefnId: 18026660730857541767 Name: idx_secondary_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:43.382+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 929.299µs 2023-10-21T04:25:43.382+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 977.15µs 2023-10-21T04:25:43.382+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 1094343481594206610, PartitionId 0, IndexDefnId 18026660730857541767. 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:43.382+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:43.382+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex 2023-10-21T04:25:43.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex stopped 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.382+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.382+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 1094343481594206610 2023-10-21T04:25:43.396+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex] 2023-10-21T04:25:43.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex closed 2023-10-21T04:25:43.399+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.400+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.407+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.410+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:43.410+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:43.410+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:43.410+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:43.410+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex 2023-10-21T04:25:43.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex stopped 2023-10-21T04:25:43.410+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.410+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.410+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.410+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.401+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 29.479346ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:43.436+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex] 2023-10-21T04:25:43.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex closed 2023-10-21T04:25:43.438+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.438+05:30 [Info] Indexer::cleanupIndexData IndexInst 1094343481594206610 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:43.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex ... 2023-10-21T04:25:43.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 197 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex ... 2023-10-21T04:25:43.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/docIndex sucessfully destroyed 2023-10-21T04:25:43.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex ... 2023-10-21T04:25:43.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 219 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex ... 2023-10-21T04:25:43.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_defer_1094343481594206610_0.index destroyed 2023-10-21T04:25:43.460+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 1094343481594206610, PartitionId 0, IndexDefnId 18026660730857541767. 2023-10-21T04:25:43.460+05:30 [Info] Indexer::cleanupIndexData IndexInst 1094343481594206610 Partition 0 Destroy Done 2023-10-21T04:25:43.507+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.507+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.510+05:30 [Info] switched currmeta from 248 -> 252 force true 2023-10-21T04:25:43.554+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.554+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.557+05:30 [Info] switched currmeta from 252 -> 252 force true 2023-10-21T04:25:43.630+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 14271070195512901425 &{0} 2023-10-21T04:25:43.632+05:30 [Info] Indexer::handleDropIndex - IndexInstId 14271070195512901425 2023-10-21T04:25:43.632+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [14271070195512901425] 2023-10-21T04:25:43.632+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [14271070195512901425] 2023-10-21T04:25:43.632+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:43.632+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [14271070195512901425] 2023-10-21T04:25:43.632+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 14271070195512901425 Defn: DefnId: 11619165464560280276 Name: #primary_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:25:43.632+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 14271070195512901425 2023-10-21T04:25:43.633+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 2.591866ms 2023-10-21T04:25:43.633+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.531662ms 2023-10-21T04:25:43.633+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14271070195512901425, PartitionId 0, IndexDefnId 11619165464560280276. 2023-10-21T04:25:43.633+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.633+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.633+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:43.633+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:43.633+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:43.633+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:43.633+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex 2023-10-21T04:25:43.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex stopped 2023-10-21T04:25:43.633+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.633+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.633+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.633+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.640+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex] 2023-10-21T04:25:43.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex closed 2023-10-21T04:25:43.641+05:30 [Info] bucket_1/#primary_defer/Mainstore#14271070195512901425:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.641+05:30 [Info] Indexer::cleanupIndexData IndexInst 14271070195512901425 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:43.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:25:43.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 228 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex ... 2023-10-21T04:25:43.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_defer_14271070195512901425_0.index destroyed 2023-10-21T04:25:43.643+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14271070195512901425, PartitionId 0, IndexDefnId 11619165464560280276. 2023-10-21T04:25:43.643+05:30 [Info] Indexer::cleanupIndexData IndexInst 14271070195512901425 Partition 0 Destroy Done 2023-10-21T04:25:43.671+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 51.427245ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:43.699+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.700+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.704+05:30 [Info] switched currmeta from 252 -> 255 force true 2023-10-21T04:25:43.743+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.743+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.747+05:30 [Info] switched currmeta from 255 -> 256 force true 2023-10-21T04:25:43.761+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:43.761+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:43.765+05:30 [Info] switched currmeta from 256 -> 256 force true 2023-10-21T04:25:43.897+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 11460826864060385936 &{0} 2023-10-21T04:25:43.897+05:30 [Info] Indexer::handleDropIndex - IndexInstId 11460826864060385936 2023-10-21T04:25:43.897+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:43.898+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:43.898+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 11460826864060385936 Name: idx_partitioned Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:43.898+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [11460826864060385936] 2023-10-21T04:25:43.898+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [11460826864060385936] 2023-10-21T04:25:43.898+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:43.898+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [11460826864060385936] 2023-10-21T04:25:43.898+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.898+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.899+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex stopped 2023-10-21T04:25:43.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex stopped 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.899+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:25:43.899+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex 2023-10-21T04:25:43.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex stopped 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.899+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.899+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0}] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:43.900+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 11460826864060385936 Defn: DefnId: 16162012140118277065 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0}] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:43.900+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.722639ms 2023-10-21T04:25:43.900+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:25:43.900+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:25:43.900+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex 2023-10-21T04:25:43.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex stopped 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.900+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.900+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:25:43.901+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:25:43.901+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex 2023-10-21T04:25:43.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex stopped 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.901+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.901+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 11460826864060385936 2023-10-21T04:25:43.906+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex] 2023-10-21T04:25:43.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex closed 2023-10-21T04:25:43.906+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.906+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:25:43.907+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:25:43.907+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex 2023-10-21T04:25:43.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex stopped 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.907+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.910+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex] 2023-10-21T04:25:43.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex closed 2023-10-21T04:25:43.910+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.910+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:43.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex ... 2023-10-21T04:25:43.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex ... 2023-10-21T04:25:43.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/docIndex sucessfully destroyed 2023-10-21T04:25:43.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex ... 2023-10-21T04:25:43.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex ... 2023-10-21T04:25:43.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_4.index destroyed 2023-10-21T04:25:43.915+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 4, IndexDefnId 16162012140118277065. 2023-10-21T04:25:43.915+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 4 Destroy Done 2023-10-21T04:25:43.915+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:25:43.915+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex 2023-10-21T04:25:43.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex stopped 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.915+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:25:43.915+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex 2023-10-21T04:25:43.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex stopped 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.915+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:25:43.915+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:25:43.916+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:25:43.916+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex 2023-10-21T04:25:43.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex stopped 2023-10-21T04:25:43.916+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.916+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.916+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.916+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.916+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId Instances [11460826864060385936] 2023-10-21T04:25:43.928+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId 2023-10-21T04:25:43.928+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 35695 2023-10-21T04:25:43.928+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 28.107779ms 2023-10-21T04:25:43.934+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 36.252214ms 2023-10-21T04:25:43.942+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex] 2023-10-21T04:25:43.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex closed 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:25:43.943+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:25:43.943+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex 2023-10-21T04:25:43.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex stopped 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.943+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.958+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex] 2023-10-21T04:25:43.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex closed 2023-10-21T04:25:43.958+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.959+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:43.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex ... 2023-10-21T04:25:43.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex ... 2023-10-21T04:25:43.963+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex] 2023-10-21T04:25:43.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex closed 2023-10-21T04:25:43.963+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.983+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.972+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.979+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:43.983+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 95.339164ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:43.985+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.985+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:43.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex] 2023-10-21T04:25:43.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex closed 2023-10-21T04:25:43.998+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:43.998+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:43.998+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:43.998+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:25:43.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/docIndex sucessfully destroyed 2023-10-21T04:25:43.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex ... 2023-10-21T04:25:43.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex ... 2023-10-21T04:25:43.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:43.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:25:43.999+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:25:43.999+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:25:43.999+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:25:43.999+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex 2023-10-21T04:25:43.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex stopped 2023-10-21T04:25:43.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:43.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:43.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:43.999+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:43.999+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:25:44.000+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:25:44.000+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:25:44.016+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.000+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex 2023-10-21T04:25:44.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex stopped 2023-10-21T04:25:44.017+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.017+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.017+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.017+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.017+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:44.017+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:44.033+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_8.index destroyed 2023-10-21T04:25:44.040+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 8, IndexDefnId 16162012140118277065. 2023-10-21T04:25:44.040+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 8 Destroy Done 2023-10-21T04:25:44.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex] 2023-10-21T04:25:44.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex closed 2023-10-21T04:25:44.041+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.041+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.041+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.044+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:25:44.044+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:25:44.044+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:25:44.044+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:25:44.044+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex 2023-10-21T04:25:44.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex stopped 2023-10-21T04:25:44.044+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.044+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.044+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.044+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex] 2023-10-21T04:25:44.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex closed 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:25:44.045+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:25:44.045+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex 2023-10-21T04:25:44.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex stopped 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex] 2023-10-21T04:25:44.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex closed 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:25:44.045+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:25:44.045+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex 2023-10-21T04:25:44.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex stopped 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.045+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex] 2023-10-21T04:25:44.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex closed 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:25:44.046+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:25:44.046+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex 2023-10-21T04:25:44.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex stopped 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.046+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.069+05:30 [Info] switched currmeta from 256 -> 259 force true 2023-10-21T04:25:44.079+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex] 2023-10-21T04:25:44.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex closed 2023-10-21T04:25:44.082+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.085+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex ... 2023-10-21T04:25:44.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex ... 2023-10-21T04:25:44.093+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex ... 2023-10-21T04:25:44.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex ... 2023-10-21T04:25:44.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex] 2023-10-21T04:25:44.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex closed 2023-10-21T04:25:44.095+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.095+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex ... 2023-10-21T04:25:44.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex ... 2023-10-21T04:25:44.096+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.096+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex] 2023-10-21T04:25:44.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex closed 2023-10-21T04:25:44.098+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.099+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.099+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex] 2023-10-21T04:25:44.099+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex closed 2023-10-21T04:25:44.099+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.100+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.099+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9950395984257248, throttleDelayMs (new, old, change): (6, 1, 5) 2023-10-21T04:25:44.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex ... 2023-10-21T04:25:44.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex ... 2023-10-21T04:25:44.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex ... 2023-10-21T04:25:44.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex ... 2023-10-21T04:25:44.111+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_1.index destroyed 2023-10-21T04:25:44.112+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 1, IndexDefnId 16162012140118277065. 2023-10-21T04:25:44.113+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 1 Destroy Done 2023-10-21T04:25:44.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex ... 2023-10-21T04:25:44.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex ... 2023-10-21T04:25:44.116+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex] 2023-10-21T04:25:44.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex closed 2023-10-21T04:25:44.118+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.118+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex ... 2023-10-21T04:25:44.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex ... 2023-10-21T04:25:44.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index/mainIndex sucessfully destroyed 2023-10-21T04:25:44.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex ... 2023-10-21T04:25:44.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_3.index destroyed 2023-10-21T04:25:44.120+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 3, IndexDefnId 16162012140118277065. 2023-10-21T04:25:44.120+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 3 Destroy Done 2023-10-21T04:25:44.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex] 2023-10-21T04:25:44.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex closed 2023-10-21T04:25:44.122+05:30 [Info] bucket_%2/idx_secondary_defer/Backstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.122+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex ... 2023-10-21T04:25:44.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex ... 2023-10-21T04:25:44.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_2.index destroyed 2023-10-21T04:25:44.128+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 2, IndexDefnId 16162012140118277065. 2023-10-21T04:25:44.129+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 2 Destroy Done 2023-10-21T04:25:44.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex ... 2023-10-21T04:25:44.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex ... 2023-10-21T04:25:44.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_5.index destroyed 2023-10-21T04:25:44.141+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 5, IndexDefnId 16162012140118277065. 2023-10-21T04:25:44.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex ... 2023-10-21T04:25:44.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex ... 2023-10-21T04:25:44.141+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 5 Destroy Done 2023-10-21T04:25:44.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index/mainIndex sucessfully destroyed 2023-10-21T04:25:44.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex ... 2023-10-21T04:25:44.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_7.index destroyed 2023-10-21T04:25:44.145+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 7, IndexDefnId 16162012140118277065. 2023-10-21T04:25:44.145+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 7 Destroy Done 2023-10-21T04:25:44.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.162+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_11460826864060385936_6.index destroyed 2023-10-21T04:25:44.162+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 11460826864060385936, PartitionId 6, IndexDefnId 16162012140118277065. 2023-10-21T04:25:44.163+05:30 [Info] Indexer::cleanupIndexData IndexInst 11460826864060385936 Partition 6 Destroy Done 2023-10-21T04:25:44.189+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 5004576769623010783 &{0} 2023-10-21T04:25:44.189+05:30 [Info] Indexer::handleDropIndex - IndexInstId 5004576769623010783 2023-10-21T04:25:44.189+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:44.189+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:44.189+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 5004576769623010783 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:44.189+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [5004576769623010783] 2023-10-21T04:25:44.189+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [5004576769623010783] 2023-10-21T04:25:44.189+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:44.190+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [5004576769623010783] 2023-10-21T04:25:44.190+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:44.190+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 5004576769623010783 Defn: DefnId: 12833670934079048728 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:44.190+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.090023ms 2023-10-21T04:25:44.190+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0, IndexDefnId 12833670934079048728. 2023-10-21T04:25:44.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:25:44.190+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:25:44.190+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:25:44.190+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:25:44.190+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex 2023-10-21T04:25:44.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex stopped 2023-10-21T04:25:44.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.191+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId Instances [5004576769623010783] 2023-10-21T04:25:44.193+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 5004576769623010783 2023-10-21T04:25:44.198+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex] 2023-10-21T04:25:44.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex closed 2023-10-21T04:25:44.199+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.199+05:30 [Info] Indexer::cleanupIndexData IndexInst 5004576769623010783 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex ... 2023-10-21T04:25:44.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex ... 2023-10-21T04:25:44.201+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId 2023-10-21T04:25:44.201+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51229 2023-10-21T04:25:44.202+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 11.566258ms 2023-10-21T04:25:44.202+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 12.726206ms 2023-10-21T04:25:44.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_5004576769623010783_0.index destroyed 2023-10-21T04:25:44.206+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 5004576769623010783, PartitionId 0, IndexDefnId 12833670934079048728. 2023-10-21T04:25:44.206+05:30 [Info] Indexer::cleanupIndexData IndexInst 5004576769623010783 Partition 0 Destroy Done 2023-10-21T04:25:44.217+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 40.100947ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:44.235+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:44.235+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:44.243+05:30 [Info] switched currmeta from 259 -> 262 force true 2023-10-21T04:25:44.345+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:44.346+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:44.360+05:30 [Info] switched currmeta from 262 -> 264 force true 2023-10-21T04:25:44.380+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:44.380+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:44.390+05:30 [Info] switched currmeta from 264 -> 264 force true 2023-10-21T04:25:44.441+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 13987057765505463258 &{0} 2023-10-21T04:25:44.441+05:30 [Info] Indexer::handleDropIndex - IndexInstId 13987057765505463258 2023-10-21T04:25:44.441+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [13987057765505463258] 2023-10-21T04:25:44.441+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [13987057765505463258] 2023-10-21T04:25:44.441+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:44.441+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [13987057765505463258] 2023-10-21T04:25:44.442+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 6, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:44.442+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:44.442+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex 2023-10-21T04:25:44.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex stopped 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.442+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 7, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:44.442+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:44.442+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex 2023-10-21T04:25:44.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex stopped 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.443+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 1, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex 2023-10-21T04:25:44.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex stopped 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.443+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 2, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex 2023-10-21T04:25:44.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex stopped 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.443+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 3, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:44.443+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex 2023-10-21T04:25:44.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex stopped 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.444+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 13987057765505463258 Defn: DefnId: 15431181238515168303 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 6 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 6 ReplicaId: 0 PartitionContainer: &{map[1:{1 6 [:9123] []} 2:{2 6 [:9123] []} 3:{3 6 [:9123] []} 4:{4 6 [:9123] []} 5:{5 6 [:9123] []} 6:{6 6 [:9123] []} 7:{7 6 [:9123] []} 8:{8 6 [:9123] []}] 8 KEY 0} 2023-10-21T04:25:44.444+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.915571ms 2023-10-21T04:25:44.444+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 2.955221ms 2023-10-21T04:25:44.444+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 4, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:44.444+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:44.444+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex 2023-10-21T04:25:44.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex stopped 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.444+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 5, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:44.444+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:44.444+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:44.444+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex 2023-10-21T04:25:44.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex stopped 2023-10-21T04:25:44.445+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.445+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.445+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.445+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.445+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 13987057765505463258 2023-10-21T04:25:44.449+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13987057765505463258, PartitionId 8, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.449+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.449+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.449+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:44.450+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:44.450+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:44.450+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:44.450+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex 2023-10-21T04:25:44.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex stopped 2023-10-21T04:25:44.450+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.450+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.450+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.450+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.504+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 70.850695ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:44.511+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex] 2023-10-21T04:25:44.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex closed 2023-10-21T04:25:44.513+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.554+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.555+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.555+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:44.556+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:44.556+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:44.556+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:44.556+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex 2023-10-21T04:25:44.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex stopped 2023-10-21T04:25:44.556+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.556+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.556+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.556+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.569+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:44.569+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:44.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex] 2023-10-21T04:25:44.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex closed 2023-10-21T04:25:44.590+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.590+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.590+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.590+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:44.590+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:44.590+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:44.590+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:44.591+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex 2023-10-21T04:25:44.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex stopped 2023-10-21T04:25:44.591+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.591+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.591+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.591+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.592+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex] 2023-10-21T04:25:44.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex closed 2023-10-21T04:25:44.592+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.592+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex] 2023-10-21T04:25:44.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex closed 2023-10-21T04:25:44.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:44.594+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:44.594+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:44.594+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:44.594+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex 2023-10-21T04:25:44.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex stopped 2023-10-21T04:25:44.594+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.594+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.594+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.594+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.597+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex] 2023-10-21T04:25:44.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex closed 2023-10-21T04:25:44.597+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.597+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.597+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.597+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:44.597+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:44.597+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:44.598+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:44.598+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex 2023-10-21T04:25:44.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex stopped 2023-10-21T04:25:44.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:44.599+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:44.599+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex 2023-10-21T04:25:44.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex stopped 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.602+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex] 2023-10-21T04:25:44.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex closed 2023-10-21T04:25:44.602+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:44.603+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:44.603+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex 2023-10-21T04:25:44.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex stopped 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex] 2023-10-21T04:25:44.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex closed 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:44.605+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:44.605+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex 2023-10-21T04:25:44.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex stopped 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.606+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex] 2023-10-21T04:25:44.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex closed 2023-10-21T04:25:44.606+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.607+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:25:44.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 202 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex ... 2023-10-21T04:25:44.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:25:44.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 224 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex ... 2023-10-21T04:25:44.641+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_6.index destroyed 2023-10-21T04:25:44.641+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 6, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.641+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 6 Destroy Done 2023-10-21T04:25:44.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex] 2023-10-21T04:25:44.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex closed 2023-10-21T04:25:44.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#13987057765505463258:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.642+05:30 [Info] switched currmeta from 264 -> 268 force true 2023-10-21T04:25:44.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:44.642+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:44.643+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:44.643+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:44.643+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex 2023-10-21T04:25:44.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex stopped 2023-10-21T04:25:44.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex] 2023-10-21T04:25:44.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex closed 2023-10-21T04:25:44.644+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.644+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:25:44.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 203 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex ... 2023-10-21T04:25:44.647+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:25:44.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 225 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex ... 2023-10-21T04:25:44.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex] 2023-10-21T04:25:44.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex closed 2023-10-21T04:25:44.673+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.673+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:25:44.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 198 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex ... 2023-10-21T04:25:44.675+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_7.index destroyed 2023-10-21T04:25:44.681+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 7, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.681+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 7 Destroy Done 2023-10-21T04:25:44.682+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex] 2023-10-21T04:25:44.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex closed 2023-10-21T04:25:44.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex] 2023-10-21T04:25:44.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:25:44.693+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.693+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 220 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex ... 2023-10-21T04:25:44.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex closed 2023-10-21T04:25:44.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex] 2023-10-21T04:25:44.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex closed 2023-10-21T04:25:44.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:25:44.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 201 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex ... 2023-10-21T04:25:44.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.696+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.696+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.698+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_2.index destroyed 2023-10-21T04:25:44.701+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 2, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.701+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 2 Destroy Done 2023-10-21T04:25:44.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:25:44.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 223 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex ... 2023-10-21T04:25:44.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:25:44.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 199 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex ... 2023-10-21T04:25:44.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_5.index destroyed 2023-10-21T04:25:44.712+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 5, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex] 2023-10-21T04:25:44.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex closed 2023-10-21T04:25:44.713+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.713+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:25:44.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 204 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex ... 2023-10-21T04:25:44.713+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 5 Destroy Done 2023-10-21T04:25:44.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:25:44.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 221 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex ... 2023-10-21T04:25:44.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_3.index destroyed 2023-10-21T04:25:44.720+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 3, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.720+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 3 Destroy Done 2023-10-21T04:25:44.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:25:44.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 200 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex ... 2023-10-21T04:25:44.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:25:44.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 226 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex ... 2023-10-21T04:25:44.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_8.index destroyed 2023-10-21T04:25:44.751+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 8, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.751+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 8 Destroy Done 2023-10-21T04:25:44.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:25:44.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 222 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex ... 2023-10-21T04:25:44.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_4.index destroyed 2023-10-21T04:25:44.760+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 4, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.760+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 4 Destroy Done 2023-10-21T04:25:44.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex] 2023-10-21T04:25:44.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex closed 2023-10-21T04:25:44.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#13987057765505463258:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.776+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:25:44.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : removed plasmaId 205 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex ... 2023-10-21T04:25:44.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard7659339677706884510(7659339677706884510)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/docIndex sucessfully destroyed 2023-10-21T04:25:44.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:25:44.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 227 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex ... 2023-10-21T04:25:44.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_13987057765505463258_1.index destroyed 2023-10-21T04:25:44.796+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13987057765505463258, PartitionId 1, IndexDefnId 15431181238515168303. 2023-10-21T04:25:44.798+05:30 [Info] Indexer::cleanupIndexData IndexInst 13987057765505463258 Partition 1 Destroy Done 2023-10-21T04:25:44.801+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15667952562388529486 &{0} 2023-10-21T04:25:44.801+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15667952562388529486 2023-10-21T04:25:44.801+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:44.801+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:44.801+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15667952562388529486 Name: #primary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:44.802+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15667952562388529486] 2023-10-21T04:25:44.802+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15667952562388529486] 2023-10-21T04:25:44.802+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:44.802+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15667952562388529486] 2023-10-21T04:25:44.802+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:44.802+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15667952562388529486 Defn: DefnId: 12338036531800130039 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: true NumReplica: 1 InstVersion: 6 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 6 ReplicaId: 1 PartitionContainer: &{map[0:{0 6 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:44.802+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.095515ms 2023-10-21T04:25:44.802+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15667952562388529486 2023-10-21T04:25:44.802+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0, IndexDefnId 12338036531800130039. 2023-10-21T04:25:44.802+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:44.802+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:44.802+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:44.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:25:44.802+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:25:44.802+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:25:44.802+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:25:44.802+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex 2023-10-21T04:25:44.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex stopped 2023-10-21T04:25:44.803+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:44.803+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:44.803+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:44.803+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:44.803+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId Instances [15667952562388529486] 2023-10-21T04:25:44.806+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId 2023-10-21T04:25:44.806+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51229 2023-10-21T04:25:44.806+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 3.79141ms 2023-10-21T04:25:44.806+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 4.957547ms 2023-10-21T04:25:44.809+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:44.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex] 2023-10-21T04:25:44.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex closed 2023-10-21T04:25:44.810+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:44.810+05:30 [Info] Indexer::cleanupIndexData IndexInst 15667952562388529486 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:44.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex ... 2023-10-21T04:25:44.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex ... 2023-10-21T04:25:44.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:44.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_15667952562388529486_0.index destroyed 2023-10-21T04:25:44.817+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15667952562388529486, PartitionId 0, IndexDefnId 12338036531800130039. 2023-10-21T04:25:44.817+05:30 [Info] Indexer::cleanupIndexData IndexInst 15667952562388529486 Partition 0 Destroy Done 2023-10-21T04:25:44.827+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 36.97058ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:45.071+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996116784732731, throttleDelayMs (new, old, change): (10, 6, 4) 2023-10-21T04:25:45.128+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 2296526085582506469 &{0} 2023-10-21T04:25:45.130+05:30 [Info] Indexer::handleDropIndex - IndexInstId 2296526085582506469 2023-10-21T04:25:45.131+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:45.131+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:45.131+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:45.131+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:45.131+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2296526085582506469 Name: #primary Keyspace: bucket_%2:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:45.131+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [2296526085582506469] 2023-10-21T04:25:45.131+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [2296526085582506469] 2023-10-21T04:25:45.131+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:45.132+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [2296526085582506469] 2023-10-21T04:25:45.132+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:45.132+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0, IndexDefnId 8857269605176042330. 2023-10-21T04:25:45.132+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:45.132+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.132+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.132+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:25:45.132+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:25:45.132+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:45.132+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:25:45.132+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc009280000 }] 2023-10-21T04:25:45.132+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 35695 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:45.132+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:25:45.132+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:25:45.132+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:45.132+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:45.132+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:45.132+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.26559ms 2023-10-21T04:25:45.132+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 2296526085582506469 2023-10-21T04:25:45.133+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_%2] 2023-10-21T04:25:45.135+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 2 35403626585217 52 2023-10-21T04:25:45.135+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.135+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 0 263268055970459 56 2023-10-21T04:25:45.135+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.135+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 1 176639880977228 66 2023-10-21T04:25:45.136+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 10 206533515052028 52 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 3 215749515705641 47 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 4 159113185399719 60 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 5 3874519919278 57 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 6 261493429196201 63 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 7 170288557775498 54 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 8 264975169139761 58 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.137+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 9 236398693961229 66 2023-10-21T04:25:45.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.139+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 16 200752780087280 58 2023-10-21T04:25:45.139+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.139+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 17 98820809009123 48 2023-10-21T04:25:45.139+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.139+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 14 140236896721046 65 2023-10-21T04:25:45.139+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.140+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 15 162158720181435 69 2023-10-21T04:25:45.140+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.140+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 13 102350060311343 56 2023-10-21T04:25:45.140+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.140+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 18 207134287529256 48 2023-10-21T04:25:45.140+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.140+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 19 208836052671833 48 2023-10-21T04:25:45.140+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.140+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 11 234758907181738 58 2023-10-21T04:25:45.140+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.140+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 12 116104661063478 57 2023-10-21T04:25:45.140+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.141+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': swap is disabled 2023-10-21T04:25:45.147+05:30 [Info] switched currmeta from 268 -> 275 force true 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 24 271667630317209 54 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 25 168740244208559 63 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 28 236953356346174 56 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 20 92956357968546 54 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 22 192215088742243 57 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 26 120989102157727 61 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 27 5715487545729 57 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 21 27979933591532 56 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.149+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 23 154934968850942 55 2023-10-21T04:25:45.149+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.151+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 55 61798873988101 69 2023-10-21T04:25:45.151+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.151+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 33 278426329668882 58 2023-10-21T04:25:45.151+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.151+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 39 100659545510465 52 2023-10-21T04:25:45.151+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.151+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 43 249919763471169 68 2023-10-21T04:25:45.151+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.151+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 47 149022976193937 57 2023-10-21T04:25:45.151+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.151+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 42 45479338375128 57 2023-10-21T04:25:45.151+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.151+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 45 49796766757346 66 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 46 124194574848641 58 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 56 29071415952852 63 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 30 60545336618717 48 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 36 43594649874772 54 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 37 83647976044922 61 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 49 184487117398647 57 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 50 199728167534433 68 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 52 158076890855634 56 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 29 130231725986856 51 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 40 153478150002150 72 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 44 166675095997741 60 2023-10-21T04:25:45.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 35 24368176313355 53 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 41 116785907755771 57 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 34 51776324008509 63 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 38 181267443431019 55 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 31 73432771436458 62 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 48 19274425379955 58 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 51 196777743741606 53 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 53 138935694816848 63 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 32 67362620867705 69 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 54 24856128127173 62 2023-10-21T04:25:45.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.159+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 60 56626109228904 48 2023-10-21T04:25:45.159+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.159+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 61 119871837460216 41 2023-10-21T04:25:45.159+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.160+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 62 177105814270799 60 2023-10-21T04:25:45.160+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.160+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 63 252537041863196 75 2023-10-21T04:25:45.160+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.161+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 57 191719959781498 65 2023-10-21T04:25:45.161+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.161+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 58 103336286678362 48 2023-10-21T04:25:45.161+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.161+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 59 140942236111856 65 2023-10-21T04:25:45.162+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:25:45.162+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_%2] 2023-10-21T04:25:45.162+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 35695 2023-10-21T04:25:45.164+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 31.573962ms 2023-10-21T04:25:45.164+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 34.15124ms 2023-10-21T04:25:45.163+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 44.355351ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:45.190+05:30 [Info] bucket_%2_7007f0ab442ac00e52d057a30cd22644: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:25:45.190+05:30 [Info] controller 'bucket_%2_7007f0ab442ac00e52d057a30cd22644/bucket_%2_7007f0ab442ac00e52d057a30cd22644': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex 2023-10-21T04:25:45.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex stopped 2023-10-21T04:25:45.190+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.190+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.190+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.190+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.192+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex] 2023-10-21T04:25:45.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex closed 2023-10-21T04:25:45.193+05:30 [Info] bucket_%2/idx_secondary_defer/Mainstore#1094343481594206610:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.193+05:30 [Info] Indexer::cleanupIndexData IndexInst 2296526085582506469 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex ... 2023-10-21T04:25:45.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex ... 2023-10-21T04:25:45.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10890719122371003479(10890719122371003479)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_2296526085582506469_0.index destroyed 2023-10-21T04:25:45.201+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 2296526085582506469, PartitionId 0, IndexDefnId 8857269605176042330. 2023-10-21T04:25:45.201+05:30 [Info] Indexer::cleanupIndexData IndexInst 2296526085582506469 Partition 0 Destroy Done 2023-10-21T04:25:45.283+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:45.283+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:45.289+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12004664282277897596 &{0} 2023-10-21T04:25:45.289+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12004664282277897596 2023-10-21T04:25:45.289+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:25:45.289+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:45.289+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12004664282277897596 Name: idx_secondary Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:25:45.289+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12004664282277897596] 2023-10-21T04:25:45.289+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12004664282277897596] 2023-10-21T04:25:45.289+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:45.290+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12004664282277897596] 2023-10-21T04:25:45.290+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:45.290+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 12004664282277897596 Defn: DefnId: 1252359852432998131 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: _default/0 IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:45.290+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.235878ms 2023-10-21T04:25:45.291+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12004664282277897596, PartitionId 0, IndexDefnId 1252359852432998131. 2023-10-21T04:25:45.291+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.291+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.291+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex] 2023-10-21T04:25:45.291+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex 2023-10-21T04:25:45.291+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex] 2023-10-21T04:25:45.291+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex] 2023-10-21T04:25:45.291+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex 2023-10-21T04:25:45.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex stopped 2023-10-21T04:25:45.291+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.291+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.291+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.291+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.291+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12004664282277897596 2023-10-21T04:25:45.291+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId Instances [12004664282277897596] 2023-10-21T04:25:45.294+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceId 2023-10-21T04:25:45.297+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51229 2023-10-21T04:25:45.297+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.562726ms 2023-10-21T04:25:45.297+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 8.0521ms 2023-10-21T04:25:45.297+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex] 2023-10-21T04:25:45.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex closed 2023-10-21T04:25:45.298+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.298+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.299+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.299+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex] 2023-10-21T04:25:45.300+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex 2023-10-21T04:25:45.301+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex] 2023-10-21T04:25:45.301+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex] 2023-10-21T04:25:45.301+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex 2023-10-21T04:25:45.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex stopped 2023-10-21T04:25:45.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.306+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex] 2023-10-21T04:25:45.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex closed 2023-10-21T04:25:45.307+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.307+05:30 [Info] Indexer::cleanupIndexData IndexInst 12004664282277897596 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex ... 2023-10-21T04:25:45.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 191 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex ... 2023-10-21T04:25:45.307+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 29.503021ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:45.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex ... 2023-10-21T04:25:45.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 213 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex ... 2023-10-21T04:25:45.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_12004664282277897596_0.index destroyed 2023-10-21T04:25:45.318+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12004664282277897596, PartitionId 0, IndexDefnId 1252359852432998131. 2023-10-21T04:25:45.318+05:30 [Info] Indexer::cleanupIndexData IndexInst 12004664282277897596 Partition 0 Destroy Done 2023-10-21T04:25:45.323+05:30 [Info] switched currmeta from 275 -> 279 force true 2023-10-21T04:25:45.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:45.378+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:45.387+05:30 [Info] switched currmeta from 279 -> 280 force true 2023-10-21T04:25:45.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:45.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:45.427+05:30 [Info] switched currmeta from 280 -> 280 force true 2023-10-21T04:25:45.437+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 9912003609070511414 &{0} 2023-10-21T04:25:45.437+05:30 [Info] Indexer::handleDropIndex - IndexInstId 9912003609070511414 2023-10-21T04:25:45.438+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:45.438+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:45.438+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9912003609070511414 Name: idx_partitioned Keyspace: bucket_1:_default:_default State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 6 ReplicaId: 1 2023-10-21T04:25:45.438+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [9912003609070511414] 2023-10-21T04:25:45.438+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [9912003609070511414] 2023-10-21T04:25:45.438+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842503623358421 bucket_1:1697842504289265542] 2023-10-21T04:25:45.438+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [9912003609070511414] 2023-10-21T04:25:45.438+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.438+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.438+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.438+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:25:45.438+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:25:45.438+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:25:45.438+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:25:45.439+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex 2023-10-21T04:25:45.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex stopped 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.439+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:25:45.439+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:25:45.439+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:25:45.439+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:25:45.439+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex 2023-10-21T04:25:45.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex stopped 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.439+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:25:45.439+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:25:45.439+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:25:45.439+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:25:45.439+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex 2023-10-21T04:25:45.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex stopped 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.439+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.440+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:25:45.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:25:45.440+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:25:45.440+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:25:45.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex 2023-10-21T04:25:45.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex stopped 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.440+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:25:45.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:25:45.440+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:25:45.440+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:25:45.440+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex 2023-10-21T04:25:45.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex stopped 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.440+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.440+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:25:45.440+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:25:45.440+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:45.441+05:30 [Info] MutationStreamReader:Shutdown StreamReader MAINT_STREAM 2023-10-21T04:25:45.441+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:25:45.441+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:25:45.441+05:30 [Error] DATP[->dataport ":9123"] Accept() Error: accept tcp4 0.0.0.0:9123: use of closed network connection 2023-10-21T04:25:45.441+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9123->127.0.0.1:37978: use of closed network connection 2023-10-21T04:25:45.441+05:30 [Error] DATP[->dataport ":9123"] worker "127.0.0.1:37978" exit: read tcp4 127.0.0.1:9123->127.0.0.1:37978: use of closed network connection 2023-10-21T04:25:45.441+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:37978" closed ! 2023-10-21T04:25:45.441+05:30 [Info] DATP[->dataport ":9123"] ... stopped 2023-10-21T04:25:45.441+05:30 [Error] DATP[->dataport ":9123"] fatal remote "127.0.0.1:37978" already gone 2023-10-21T04:25:45.441+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.441+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.441+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.441+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:25:45.441+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:25:45.441+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:25:45.441+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:25:45.441+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex 2023-10-21T04:25:45.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex stopped 2023-10-21T04:25:45.441+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.442+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:25:45.442+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:25:45.442+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:25:45.442+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:25:45.442+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex 2023-10-21T04:25:45.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex stopped 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.442+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.442+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 51229 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:25:45.442+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:25:45.442+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:25:45.442+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:25:45.442+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.816697ms 2023-10-21T04:25:45.442+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 9912003609070511414 2023-10-21T04:25:45.454+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:25:45.454+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:25:45.454+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:25:45.454+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:25:45.454+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex 2023-10-21T04:25:45.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex stopped 2023-10-21T04:25:45.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.456+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_1] 2023-10-21T04:25:45.470+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex] 2023-10-21T04:25:45.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex closed 2023-10-21T04:25:45.472+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': swap is disabled 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:25:45.472+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:25:45.472+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:25:45.472+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:25:45.472+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex 2023-10-21T04:25:45.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex stopped 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.474+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex] 2023-10-21T04:25:45.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex closed 2023-10-21T04:25:45.474+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.474+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.474+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.474+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:25:45.474+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:25:45.474+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:25:45.474+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:25:45.474+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex 2023-10-21T04:25:45.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex stopped 2023-10-21T04:25:45.474+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.475+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.475+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.475+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex] 2023-10-21T04:25:45.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex closed 2023-10-21T04:25:45.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.476+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.478+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex] 2023-10-21T04:25:45.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex closed 2023-10-21T04:25:45.478+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.478+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex ... 2023-10-21T04:25:45.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex ... 2023-10-21T04:25:45.478+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.478+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:25:45.482+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex ... 2023-10-21T04:25:45.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex ... 2023-10-21T04:25:45.485+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:25:45.485+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:25:45.485+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:25:45.485+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex 2023-10-21T04:25:45.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex stopped 2023-10-21T04:25:45.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.485+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.485+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_2298f6cccd108c9b6151cc01f5a848f9 KeyspaceIds [bucket_1]. Err genServer.closed 2023-10-21T04:25:45.485+05:30 [Error] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:25:45.485+05:30 [Info] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_1] Treating genServer.closed As Success 2023-10-21T04:25:45.485+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51229 2023-10-21T04:25:45.485+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 42.909222ms 2023-10-21T04:25:45.485+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 47.789504ms 2023-10-21T04:25:45.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex] 2023-10-21T04:25:45.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex closed 2023-10-21T04:25:45.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.487+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex ... 2023-10-21T04:25:45.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex ... 2023-10-21T04:25:45.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_4.index destroyed 2023-10-21T04:25:45.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex] 2023-10-21T04:25:45.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex closed 2023-10-21T04:25:45.494+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.494+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.494+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.494+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:25:45.495+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 4, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.495+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 4 Destroy Done 2023-10-21T04:25:45.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex ... 2023-10-21T04:25:45.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex ... 2023-10-21T04:25:45.495+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:25:45.495+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:25:45.495+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:25:45.495+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex 2023-10-21T04:25:45.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex stopped 2023-10-21T04:25:45.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.500+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex] 2023-10-21T04:25:45.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex closed 2023-10-21T04:25:45.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.508+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.509+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.511+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.511+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex ... 2023-10-21T04:25:45.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex ... 2023-10-21T04:25:45.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_5.index destroyed 2023-10-21T04:25:45.513+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 5, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.513+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 5 Destroy Done 2023-10-21T04:25:45.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex] 2023-10-21T04:25:45.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex closed 2023-10-21T04:25:45.513+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.513+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.513+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.513+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:25:45.535+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 109.28729ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:25:45.543+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex] 2023-10-21T04:25:45.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex closed 2023-10-21T04:25:45.544+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.544+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.544+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.544+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:25:45.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex] 2023-10-21T04:25:45.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex closed 2023-10-21T04:25:45.545+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:25:45.545+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:25:45.545+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:25:45.545+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex 2023-10-21T04:25:45.546+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:25:45.547+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:25:45.550+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:25:45.548+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.549+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.549+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex ... 2023-10-21T04:25:45.549+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex stopped 2023-10-21T04:25:45.551+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex 2023-10-21T04:25:45.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex stopped 2023-10-21T04:25:45.553+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.552+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 7 Close Done 2023-10-21T04:25:45.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex ... 2023-10-21T04:25:45.552+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.587+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.587+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.588+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex ... 2023-10-21T04:25:45.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex ... 2023-10-21T04:25:45.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.597+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.597+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.598+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_2.index destroyed 2023-10-21T04:25:45.601+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 2, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.603+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 2 Destroy Done 2023-10-21T04:25:45.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex] 2023-10-21T04:25:45.604+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex closed 2023-10-21T04:25:45.605+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.606+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.606+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.606+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:25:45.607+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:25:45.607+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:25:45.607+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:25:45.607+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex 2023-10-21T04:25:45.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex stopped 2023-10-21T04:25:45.608+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.608+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.608+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.608+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.604+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex] 2023-10-21T04:25:45.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex closed 2023-10-21T04:25:45.612+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.612+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex ... 2023-10-21T04:25:45.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex ... 2023-10-21T04:25:45.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex ... 2023-10-21T04:25:45.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex ... 2023-10-21T04:25:45.620+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.628+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index/mainIndex sucessfully destroyed 2023-10-21T04:25:45.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex] 2023-10-21T04:25:45.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex closed 2023-10-21T04:25:45.630+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.630+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:25:45.630+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:25:45.630+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:25:45.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_7.index destroyed 2023-10-21T04:25:45.631+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 7, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.632+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 7 Destroy Done 2023-10-21T04:25:45.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex ... 2023-10-21T04:25:45.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex ... 2023-10-21T04:25:45.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex] 2023-10-21T04:25:45.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex closed 2023-10-21T04:25:45.635+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.635+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex ... 2023-10-21T04:25:45.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex ... 2023-10-21T04:25:45.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex] 2023-10-21T04:25:45.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex closed 2023-10-21T04:25:45.643+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:25:45.643+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] 2023-10-21T04:25:45.643+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.643+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_3.index destroyed 2023-10-21T04:25:45.650+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 3, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.651+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 3 Destroy Done 2023-10-21T04:25:45.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex ... 2023-10-21T04:25:45.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex ... 2023-10-21T04:25:45.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_6.index destroyed 2023-10-21T04:25:45.665+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 6, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.665+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 6 Destroy Done 2023-10-21T04:25:45.698+05:30 [Info] bucket_1_db19c9050b4ab1232e6bc6654011618f: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] 2023-10-21T04:25:45.698+05:30 [Info] controller 'bucket_1_db19c9050b4ab1232e6bc6654011618f/bucket_1_db19c9050b4ab1232e6bc6654011618f': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex 2023-10-21T04:25:45.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex stopped 2023-10-21T04:25:45.698+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:25:45.698+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:25:45.699+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:25:45.699+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:25:45.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex ... 2023-10-21T04:25:45.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex ... 2023-10-21T04:25:45.701+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:25:45.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex ... 2023-10-21T04:25:45.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex] 2023-10-21T04:25:45.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex closed 2023-10-21T04:25:45.703+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#7315818257769233877:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:25:45.703+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:25:45.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex ... 2023-10-21T04:25:45.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex ... 2023-10-21T04:25:45.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex ... 2023-10-21T04:25:45.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard12177293282243154215(12177293282243154215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/docIndex sucessfully destroyed 2023-10-21T04:25:45.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_8.index destroyed 2023-10-21T04:25:45.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex ... 2023-10-21T04:25:45.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex ... 2023-10-21T04:25:45.715+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 8, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.715+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 8 Destroy Done 2023-10-21T04:25:45.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : metadata saved successfully 2023-10-21T04:25:45.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2864853750198994630(2864853750198994630)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_9912003609070511414_1.index destroyed 2023-10-21T04:25:45.717+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9912003609070511414, PartitionId 1, IndexDefnId 573521267164877540. 2023-10-21T04:25:45.717+05:30 [Info] Indexer::cleanupIndexData IndexInst 9912003609070511414 Partition 1 Destroy Done 2023-10-21T04:25:46.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:25:46.253+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:46.253+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:46.290+05:30 [Info] switched currmeta from 280 -> 284 force true 2023-10-21T04:25:46.304+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:46.304+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:46.306+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:25:47.006+05:30 [Warn] Client:runObserveStreamingEndpoint streaming endpoint for /pools/default/bs/bucket_1 returned err EOF 2023-10-21T04:25:47.006+05:30 [Warn] serviceChangeNotifier: Connection terminated for collection manifest notifier instance of http://%40index@127.0.0.1:9003-default, bucket: bucket_1, (EOF) 2023-10-21T04:25:47.006+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /pools/default/nodeServicesStreaming 2023-10-21T04:25:47.007+05:30 [Info] clusterInfoCacheLiteManager watchClusterChanges: restarting.. 2023-10-21T04:25:47.007+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /pools/default/bs/bucket_%2 2023-10-21T04:25:47.007+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /poolsStreaming/default 2023-10-21T04:25:47.007+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /pools/default/nodeServicesStreaming 2023-10-21T04:25:47.007+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /pools/default/bs/bucket_%2 2023-10-21T04:25:47.007+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /poolsStreaming/default 2023-10-21T04:25:47.031+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 143] bucketNames has been updated (was map[bucket_%2:7007f0ab442ac00e52d057a30cd22644 bucket_1:db19c9050b4ab1232e6bc6654011618f], now map[bucket_%2:7007f0ab442ac00e52d057a30cd22644]) 2023-10-21T04:25:47.032+05:30 INFO REGU.impl.(*meteringService).watchForBucketUpdates.func1() at metrics.go:129 [id 143] bucket bucket_1 has been dropped, will reset its metrics 2023-10-21T04:25:47.032+05:30 DEBU REGU.impl.dumpStats() at aggrecorder.go:364 [id 143] aggRecorder: [bucket_1] recorded 20518 aborted 0 refunded 0 2023-10-21T04:25:47.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9992995584292304 2023-10-21T04:25:48.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9994615737643943 2023-10-21T04:25:48.668+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:25:48.668+05:30 [Info] clusterInfoCacheLiteManager: watchClusterChanges sent notification on self restart 2023-10-21T04:25:48.668+05:30 [Info] handleBucketInfoChanges: stopped observing collection manifest for bucket bucket_1 2023-10-21T04:25:48.668+05:30 [Info] handleCollectionManifestChanges: stopped observing collection manifest for bucket bucket_1 2023-10-21T04:25:48.703+05:30 [Warn] AutofailoverServiceManager::HealthCheck: Slow heartbeat 2.853108795s. priorTime: 2023-10-21 04:25:45.850467058 +0530 IST m=+48.476523067, callTime: 2023-10-21 04:25:48.703576005 +0530 IST m=+51.329631862, healthInfo: {DiskFailures:0} 2023-10-21T04:25:49.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:25:49.561+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:25:49.705+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:49.707+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:49.710+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:25:49.710+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:49.716+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:25:49.730+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:49.730+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:49.733+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:25:50.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9980522593341841 2023-10-21T04:25:50.158+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:50.168+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:50.169+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:50.174+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:25:50.327+05:30 [Warn] Client:runObserveStreamingEndpoint streaming endpoint for /pools/default/bs/bucket_%2 returned err EOF 2023-10-21T04:25:50.327+05:30 [Warn] serviceChangeNotifier: Connection terminated for collection manifest notifier instance of http://%40index@127.0.0.1:9003-default, bucket: bucket_%2, (EOF) 2023-10-21T04:25:50.327+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /pools/default/nodeServicesStreaming 2023-10-21T04:25:50.327+05:30 [Info] clusterInfoCacheLiteManager watchClusterChanges: restarting.. 2023-10-21T04:25:50.328+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /poolsStreaming/default 2023-10-21T04:25:50.328+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /pools/default/nodeServicesStreaming 2023-10-21T04:25:50.328+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /poolsStreaming/default 2023-10-21T04:25:50.353+05:30 WARN REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:360 [id 144] [bucket_%2] unexpected error on bucket 127.0.0.1:9003 observe; will retry after 100ms: EOF 2023-10-21T04:25:50.453+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 144] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:25:50.455+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 143] bucketNames has been updated (was map[bucket_%2:7007f0ab442ac00e52d057a30cd22644], now map[]) 2023-10-21T04:25:50.456+05:30 INFO REGU.impl.(*meteringService).watchForBucketUpdates.func1() at metrics.go:129 [id 143] bucket bucket_%2 has been dropped, will reset its metrics 2023-10-21T04:25:50.456+05:30 DEBU REGU.impl.dumpStats() at aggrecorder.go:364 [id 143] aggRecorder: [bucket_%2] recorded 22284 aborted 0 refunded 0 2023-10-21T04:25:50.462+05:30 WARN REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:360 [id 144] [bucket_%2] unexpected error on bucket 127.0.0.1:9003 observe; will retry after 150ms: Requested resource not found. 2023-10-21T04:25:51.069+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9923821703142375 2023-10-21T04:25:51.342+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:25:51.342+05:30 [Info] clusterInfoCacheLiteManager: watchClusterChanges sent notification on self restart 2023-10-21T04:25:51.342+05:30 [Info] handleBucketInfoChanges: stopped observing collection manifest for bucket bucket_%2 2023-10-21T04:25:51.343+05:30 [Info] handleCollectionManifestChanges: stopped observing collection manifest for bucket bucket_%2 2023-10-21T04:25:51.348+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:25:51.356+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:25:51.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:51.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:51.361+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:25:51.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:25:51.378+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:25:51.380+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:25:51.424+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. err: operation_canceled 2023-10-21T04:25:51.424+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:25:51.424+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:25:51.429+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:25:51.429+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:51.618+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. err: operation_canceled 2023-10-21T04:25:51.618+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:25:51.618+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:25:51.621+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:25:51.626+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:25:51.626+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[2298f6cccd108c9b6151cc01f5a848f9 e28a1725ce62fe1e923bede04f38724f] IsBalanced:true Messages:[]} 2023-10-21T04:25:51.628+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 9] 2023-10-21T04:25:52.070+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9779953615755842 2023-10-21T04:25:53.070+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6648664334786883, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:25:54.069+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.29017133801326744, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:25:59.053+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:25:59.054+05:30 [Info] janitor: running cleanup. 2023-10-21T04:25:59.054+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[14741949230702299196 2048006213723315482 18026660730857541767 15945117906390072636 18323460706027642575 4138047918192025493 6985049432225870291 7392796230534034390 8857269605176042330 12027054384338714804 14292583894719865678 249887442666643345 16162012140118277065 12338036531800130039 12195715103909237684 14889401051926000176 8669649359784409226 7525085113233084253 15968368593314043432 4860549750398470299 8504686614261377811 13856704970496601958 15431181238515168303 5465835793930010953 3674078063353481907 11619165464560280276 17656144409930763141 8378417662949994532 1252359852432998131 12140458060295645238 18046371353728341033 4535197604682045916 12833670934079048728 573521267164877540]} 2023-10-21T04:25:59.054+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:25:59.065+05:30 [Info] memstats {"Alloc":98662880, "TotalAlloc":101372056, "Sys":117129496, "Lookups":0, "Mallocs":138432,"Frees":30006, "HeapAlloc":98662880, "HeapSys":102825984, "HeapIdle":1810432, "HeapInuse":101015552,"HeapReleased":1712128, "HeapObjects":108426,"MSpanInuse":321600, "MSpanSys": 326400, "StackInuse": 2031616,"GCSys":9776520, "LastGC":1697842498548517192,"PauseTotalNs":2167911, "PauseNs":[325444, 543320, 44319, 1254828], "NumGC":4} 2023-10-21T04:25:59.721+05:30 [Info] connected with 2 indexers 2023-10-21T04:25:59.721+05:30 [Info] client stats current counts: current: 0, not current: 0 2023-10-21T04:25:59.784+05:30 [Info] num concurrent scans {0} 2023-10-21T04:25:59.784+05:30 [Info] average scan response {0 ms} 2023-10-21T04:25:59.833+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:25:59.833+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:26:00.131+05:30 [Info] Indexer::ReadMemstats Time Taken 148.71µs 2023-10-21T04:26:03.028+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 259] aggregate storage info: map[] 2023-10-21T04:26:06.382+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:26:06.425+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:06.426+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:26:06.429+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:26:06.434+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:26:06.445+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 143] updated cluster rebalancing status to true 2023-10-21T04:26:06.445+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 143] updating rebalancing status to true 2023-10-21T04:26:06.461+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:06.461+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:26:06.463+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:26:06.501+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:26:06.540+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:06.540+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:26:06.553+05:30 [Info] switched currmeta from 284 -> 284 force true 2023-10-21T04:26:06.594+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {a78ef6a23736e254fe0ce0486fb81077 [] topology-change-rebalance [] [{2298f6cccd108c9b6151cc01f5a848f9 7060000 } {e28a1725ce62fe1e923bede04f38724f 7060000 }]} 2023-10-21T04:26:06.596+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 6.479µs 2023-10-21T04:26:06.596+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:26:06.596+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:26:06.596+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:26:06.602+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:26:06.602+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:26:06.602+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 6.341589ms 2023-10-21T04:26:06.602+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:26:06.602+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 8.505µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:26:06.602+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:26:06.602+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 10] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:26:06.602+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. taskList: &{Rev:[0 0 0 0 0 0 0 10] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a78ef6a23736e254fe0ce0486fb81077 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]}]} 2023-10-21T04:26:06.608+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 10] 2023-10-21T04:26:06.610+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 10] 2023-10-21T04:26:06.610+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {a78ef6a23736e254fe0ce0486fb81077 [] topology-change-rebalance [] [{2298f6cccd108c9b6151cc01f5a848f9 7060000 } {e28a1725ce62fe1e923bede04f38724f 7060000 }]} 2023-10-21T04:26:06.612+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:26:06.624+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"2298f6cccd108c9b6151cc01f5a848f9","RebalId":"a78ef6a23736e254fe0ce0486fb81077","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:26:06.628+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{2298f6cccd108c9b6151cc01f5a848f9 a78ef6a23736e254fe0ce0486fb81077 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:26:06.631+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{2298f6cccd108c9b6151cc01f5a848f9 a78ef6a23736e254fe0ce0486fb81077 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:26:06.631+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:26:07.643+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:26:07.643+05:30 [Info] NewShardRebalancer nodeId 2298f6cccd108c9b6151cc01f5a848f9 rebalToken &{2298f6cccd108c9b6151cc01f5a848f9 a78ef6a23736e254fe0ce0486fb81077 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:26:07.643+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:26:07.643+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 10] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a78ef6a23736e254fe0ce0486fb81077 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/a78ef6a23736e254fe0ce0486fb81077 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]}]} 2023-10-21T04:26:07.644+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:26:07.643+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.033070287s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:26:07.644+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{2298f6cccd108c9b6151cc01f5a848f9 a78ef6a23736e254fe0ce0486fb81077 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:26:07.644+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:26:07.647+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 11] 2023-10-21T04:26:07.647+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 11] 2023-10-21T04:26:07.647+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:26:07.647+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a78ef6a23736e254fe0ce0486fb81077 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/a78ef6a23736e254fe0ce0486fb81077 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]}]} 2023-10-21T04:26:07.648+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:26:07.648+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:26:07.649+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 11] 2023-10-21T04:26:07.649+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 11] 2023-10-21T04:26:07.669+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{2298f6cccd108c9b6151cc01f5a848f9 2298f6cccd108c9b6151cc01f5a848f9 plasma map[] [{39 bucket_1 _default _default []} {39 bucket_%2 _default _default []} {9 bucket_1 _system _query []} {9 bucket_%2 _system _query []}] [] 0 0 0 false} {e28a1725ce62fe1e923bede04f38724f e28a1725ce62fe1e923bede04f38724f plasma map[] [{39 bucket_%2 _default _default []} {39 bucket_1 _default _default []} {9 bucket_1 _system _query []} {9 bucket_%2 _system _query []}] [] 0 0 0 false}] map[]} 2023-10-21T04:26:07.696+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53754 2023-10-21T04:26:07.698+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:26:07.698+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:26:07.698+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53754 2023-10-21T04:26:07.713+05:30 [Info] Planner::processDeleteToken: processing 34 tokens for node 127.0.0.1:9004 2023-10-21T04:26:07.713+05:30 [Info] Planner::processDeleteToken: processing 34 tokens for node 127.0.0.1:9003 2023-10-21T04:26:07.713+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53754 2023-10-21T04:26:07.718+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:26:07.718+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:26:07.727+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:26:07.727+05:30 [Info] Planner::moveTenantsFromDeletedNodes No non-empty deleted nodes found. 2023-10-21T04:26:07.727+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG Group 1 Memory 0 Units 0 2023-10-21T04:26:07.727+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9003 SG Group 2 Memory 0 Units 0 2023-10-21T04:26:07.727+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [] 2023-10-21T04:26:07.727+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:26:07.727+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:26:07.727+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:26:07.727+05:30 [Info] ---------------------------------------- 2023-10-21T04:26:07.727+05:30 [Info] **************************************** 2023-10-21T04:26:07.727+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:26:07.727+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:26:07.727+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:26:07.727+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:26:07.727+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress done received. progress collector stopped 2023-10-21T04:26:07.727+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:26:07.727+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 11] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a78ef6a23736e254fe0ce0486fb81077 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/a78ef6a23736e254fe0ce0486fb81077 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]}]} 2023-10-21T04:26:07.727+05:30 [Info] ShardRebalancer::updateProgress Done Received 2023-10-21T04:26:07.727+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:26:07.727+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:26:07.728+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 11] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a78ef6a23736e254fe0ce0486fb81077 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/a78ef6a23736e254fe0ce0486fb81077 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a78ef6a23736e254fe0ce0486fb81077]}]} 2023-10-21T04:26:07.731+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {2298f6cccd108c9b6151cc01f5a848f9 a78ef6a23736e254fe0ce0486fb81077 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:26:07.731+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 12] 2023-10-21T04:26:07.731+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 12] 2023-10-21T04:26:07.736+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:26:07.736+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:26:07.736+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.583µs 2023-10-21T04:26:07.736+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:26:07.737+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:26:07.737+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:26:07.739+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:26:07.739+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:26:07.739+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:26:07.740+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:26:07.740+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 32.495µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:26:07.742+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:26:07.742+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:26:07.742+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster true, forceUnbalanced false, err: 2023-10-21T04:26:07.742+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 12] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:26:07.742+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:26:07.742+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:26:07.747+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:26:07.766+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:26:07.766+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[]} 2023-10-21T04:26:07.766+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:26:07.768+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:26:07.768+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[2298f6cccd108c9b6151cc01f5a848f9] IsBalanced:true Messages:[]} 2023-10-21T04:26:07.770+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:26:07.772+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:26:07.797+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:26:07.798+05:30 [Info] MetadataProvider: Updating cluster version to 18446744073709551615 2023-10-21T04:26:07.798+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:26:07.798+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:26:07.882+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:26:07.900+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:07.900+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:26:07.900+05:30 [Info] UnwatchMetadata(): indexer 2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:26:07.900+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:26:07.900+05:30 [Info] UnwatchMetadata(): finish for indexer 2298f6cccd108c9b6151cc01f5a848f9 2023-10-21T04:26:07.900+05:30 [Info] UnwatchMetadata(): indexer e28a1725ce62fe1e923bede04f38724f 2023-10-21T04:26:07.900+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:26:07.900+05:30 [Info] UnwatchMetadata(): finish for indexer e28a1725ce62fe1e923bede04f38724f 2023-10-21T04:26:07.901+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:40578->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:26:07.901+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:36972->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:26:07.901+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40578. Reason = EOF. Cleaning Pipe. 2023-10-21T04:26:07.901+05:30 [Info] messageListener.start(): message channel closed. Remove peer d8:ad:8b:82:6d:cc:9f:53 as follower. 2023-10-21T04:26:07.903+05:30 [Info] switched currmeta from 284 -> 286 force true 2023-10-21T04:26:07.903+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:26:07.903+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] ... stopped 2023-10-21T04:26:07.903+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:26:07.903+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] ... stopped 2023-10-21T04:26:07.903+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:52196 closed 2023-10-21T04:26:07.911+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:26:07.923+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:07.923+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:26:07.926+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40536. Reason = EOF. Cleaning Pipe. 2023-10-21T04:26:07.926+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2b:8c:e7:c7:10:2c:1b:19 as follower. 2023-10-21T04:26:07.928+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40538. Reason = EOF. Cleaning Pipe. 2023-10-21T04:26:07.928+05:30 [Info] messageListener.start(): message channel closed. Remove peer c6:5a:9d:0:dd:d4:1f:d4 as follower. 2023-10-21T04:26:07.929+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:52170 closed 2023-10-21T04:26:07.935+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55164 closed 2023-10-21T04:26:07.935+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55148 closed 2023-10-21T04:26:07.935+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54418 closed 2023-10-21T04:26:07.935+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54332 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54506 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55094 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55062 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54106 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55000 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55150 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55108 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54116 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54114 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53982 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53968 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54042 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54772 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54910 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55026 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54018 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55292 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54958 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54006 closed 2023-10-21T04:26:07.936+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54866 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54438 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54104 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54092 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55274 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54710 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55018 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55272 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54738 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54752 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53958 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54842 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54938 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55214 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55248 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54128 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:52182 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54386 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54200 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54296 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55060 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55280 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54832 closed 2023-10-21T04:26:07.937+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54704 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54454 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54328 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54414 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55230 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53984 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54912 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54840 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54480 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54548 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55174 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55154 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54500 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54382 closed 2023-10-21T04:26:07.938+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54286 closed 2023-10-21T04:26:07.939+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55308 closed 2023-10-21T04:26:07.939+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55222 closed 2023-10-21T04:26:07.939+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54502 closed 2023-10-21T04:26:07.939+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53950 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54722 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54034 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54404 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54334 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54026 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54520 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54208 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55008 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55134 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54040 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54546 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53898 closed 2023-10-21T04:26:07.945+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54942 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55206 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55052 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54282 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54720 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54182 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55218 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54044 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54318 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55200 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55140 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54762 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54222 closed 2023-10-21T04:26:07.946+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55208 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55084 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55040 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54272 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53890 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54822 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54068 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54124 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54836 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55138 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54252 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55256 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53994 closed 2023-10-21T04:26:07.947+05:30 [Info] switched currmeta from 286 -> 286 force true 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54036 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54696 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53952 closed 2023-10-21T04:26:07.947+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55074 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54136 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55192 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53936 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54862 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54118 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54342 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54100 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53990 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53980 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54312 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55290 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54074 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54144 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55056 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54038 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55232 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55088 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54138 closed 2023-10-21T04:26:07.948+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53928 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54424 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54820 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54314 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54542 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53894 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54288 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54518 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54476 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54360 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54166 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54790 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54396 closed 2023-10-21T04:26:07.949+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55144 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53938 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54274 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54442 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55064 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54434 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55014 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53922 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54150 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54248 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55080 closed 2023-10-21T04:26:07.950+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54830 closed 2023-10-21T04:26:07.956+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54278 closed 2023-10-21T04:26:07.963+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54940 closed 2023-10-21T04:26:07.964+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54316 closed 2023-10-21T04:26:07.964+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54530 closed 2023-10-21T04:26:07.964+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54160 closed 2023-10-21T04:26:07.964+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54218 closed 2023-10-21T04:26:07.964+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54702 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54284 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54132 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54890 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54130 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54888 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55002 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53944 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54246 closed 2023-10-21T04:26:07.965+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54468 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55254 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55152 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54452 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53926 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54450 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54998 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54818 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54146 closed 2023-10-21T04:26:07.966+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53978 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54474 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54276 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54458 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54080 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54508 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54070 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54340 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55242 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54380 closed 2023-10-21T04:26:07.967+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40532. Reason = EOF. Cleaning Pipe. 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55224 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55198 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54770 closed 2023-10-21T04:26:07.967+05:30 [Info] messageListener.start(): message channel closed. Remove peer 28:17:e4:a8:a6:18:3a:74 as follower. 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54522 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53972 closed 2023-10-21T04:26:07.967+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55186 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54030 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54492 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55160 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54412 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54730 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54306 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54098 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55300 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54876 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55216 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54440 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54838 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54298 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54188 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:53964 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55070 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55034 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54816 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54528 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55258 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54768 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54054 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54112 closed 2023-10-21T04:26:07.968+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54854 closed 2023-10-21T04:26:07.969+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:54436 closed 2023-10-21T04:26:07.969+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:55176 closed 2023-10-21T04:26:07.972+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40528. Reason = EOF. Cleaning Pipe. 2023-10-21T04:26:07.972+05:30 [Info] messageListener.start(): message channel closed. Remove peer 36:9f:f4:c5:7c:f2:2d:5c as follower. 2023-10-21T04:26:07.978+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:52188 closed 2023-10-21T04:26:07.979+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:52156 closed 2023-10-21T04:26:08.079+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:26:08.155+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 1 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:26:08.155+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:08.155+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:26:08.172+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:26:08.181+05:30 [Info] switched currmeta from 286 -> 286 force true 2023-10-21T04:26:08.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:08.195+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:26:08.198+05:30 [Info] switched currmeta from 286 -> 286 force true 2023-10-21T04:26:08.308+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:26:08.328+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:26:08.336+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:08.336+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:26:08.338+05:30 [Info] switched currmeta from 286 -> 286 force true 2023-10-21T04:26:08.363+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:08.364+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:26:08.365+05:30 [Info] switched currmeta from 286 -> 286 force true 2023-10-21T04:26:08.401+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:26:08.404+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 143] updated cluster rebalancing status to false 2023-10-21T04:26:08.406+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:26:08.406+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:26:08.408+05:30 [Info] switched currmeta from 286 -> 286 force true 2023-10-21T04:28:10.247+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=18194302e3f10df59bd076e1c4142d0b -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:28:10.248+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:28:10.248+05:30 [Info] Setting ipv6=false 2023-10-21T04:28:10.313+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:28:10.350+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:28:10.371+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:28:10.371+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:28:10.371+05:30 [Info] Setting log level to Info 2023-10-21T04:28:10.371+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:28:10.371+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:28:10.373+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:28:10.373+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:28:10.386+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:28:10 audit: created new audit service 2023-10-21T04:28:10.784+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:28:10.784+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:28:10.784+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:28:10.786+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:28:10.786+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:28:10.786+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:28:10.823+05:30 [Info] security context initialized 2023-10-21T04:28:10.824+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9120:19120 9499:19499 9999:9999] 2023-10-21T04:28:10.824+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:28:10.824+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:28:10.824+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:28:10.824+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:28:10.830+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:28:10.830+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:28:10.830+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:28:10.834+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:28:10.873+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:28:10.874+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:28:10.875+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:28:10.875+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:28:10.876+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:28:10.876+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:28:10.885+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:28:10.885+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:28:10.885+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:28:10.923+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:28:10.923+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:28:10.924+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:28:10.933+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:28:10.933+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:28:10.935+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:28:10.936+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:28:10.936+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:28:10.938+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:28:10.938+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:28:10.939+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 225 us 2023-10-21T04:28:10.940+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:28:10.945+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:28:10.958+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:28:10.965+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:28:10.965+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:28:10.965+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:28:10.965+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:28:10.965+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:28:10.972+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerId Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:28:10.974+05:30 [Info] Indexer Id 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:28:10.974+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:28:10.975+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:28:10.977+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:28:10.978+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:28:10.981+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 161] updating cluster rev epoch 0 -> 1 2023-10-21T04:28:10.981+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 161] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:28:10.997+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 172] bucketNames has been updated (was map[], now map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43]) 2023-10-21T04:28:10.998+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 206] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:28:10.998+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 207] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:28:11.020+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 172] updated cluster rebalancing status to true 2023-10-21T04:28:11.020+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:28:11.020+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:28:11.020+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:28:11.021+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 223] updating rebalancing status to true 2023-10-21T04:28:11.024+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:28:11.027+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:28:11.028+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:28:11.028+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:28:11.028+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:28:11.028+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning false RebalanceToken 2023-10-21T04:28:11.028+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:28:11.029+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:28:11.029+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:28:11.037+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:28:11.037+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:28:11.039+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:28:11.124+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap , elapsed: 94.821039ms 2023-10-21T04:28:11.124+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:28:11.124+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:28:11.124+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:28:11.124+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager false 2023-10-21T04:28:11.125+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:28:11.125+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:28:11.125+05:30 [Error] RebalanceServiceManager::getCachedIndexerNodeUUIDs: 127.0.0.1:9120/getCachedIndexerNodeUUIDs returned err: Unable to find given hostport in cbauth database: `127.0.0.1:9120' 2023-10-21T04:28:11.125+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:28:11.126+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:28:11.130+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:28:11.130+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:28:11.130+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:28:11.130+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:28:11.134+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:28:11.134+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:28:11.134+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:28:11.135+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:28:11.135+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:28:11.152+05:30 DEBU REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:323 [id 220] disregarding & deleting stale (age: 2m9.042405236s) storage report from node [index 2298f6cccd108c9b6151cc01f5a848f9] (delete err: ) 2023-10-21T04:28:11.242+05:30 DEBU REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:323 [id 220] disregarding & deleting stale (age: 2m9.152614492s) storage report from node [index e28a1725ce62fe1e923bede04f38724f] (delete err: ) 2023-10-21T04:28:11.242+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 220] /regulator/report/s/index/2298f6cccd108c9b6151cc01f5a848f9 has 0-length value 2023-10-21T04:28:11.242+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 220] /regulator/report/s/index/e28a1725ce62fe1e923bede04f38724f has 0-length value 2023-10-21T04:28:11.245+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 31] will report aggregate recorder stats every 5m0s 2023-10-21T04:28:11.254+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:28:11.254+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:28:11.254+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:28:11.254+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:28:11.257+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:28:11.275+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:28:11.275+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/stats: no such file or directory 2023-10-21T04:28:11.280+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:28:11.280+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:28:11.281+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:28:11.282+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:28:11.284+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:28:11.284+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:28:11.284+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{18194302e3f10df59bd076e1c4142d0b 7060000 } 2023-10-21T04:28:11.284+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:28:11.287+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:28:11.287+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:28:11.288+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:28:11.288+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:28:11.288+05:30 [Info] progress stats collection done. 2023-10-21T04:28:11.288+05:30 [Info] send progress stats to clients 2023-10-21T04:28:11.292+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:28:11.292+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:true Messages:[]} 2023-10-21T04:28:11.321+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:28:11.323+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:28:11.324+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:28:11.327+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:28:11.327+05:30 [Info] Setting log level to Info 2023-10-21T04:28:11.328+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:28:11.329+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:28:11.349+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:28:11.354+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:28:11.355+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:28:11.355+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:28:11.355+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:28:11.376+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:28:11.376+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:28:11.584+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:28:11.676+05:30 [Info] security context initialized 2023-10-21T04:28:11.676+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9499:19499 9999:9999] 2023-10-21T04:28:11.684+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:28:11.695+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:28:11.695+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:11.695+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:28:11.695+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:28:11.710+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:28:11.747+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:28:11.747+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:28:11.757+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:28:11.757+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:28:11.771+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:28:11.782+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:28:11.792+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:28:11.792+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:28:11.811+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:28:11.821+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:28:11.822+05:30 [Info] initialized currmeta 18 force true 2023-10-21T04:28:11.841+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:28:11.841+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:28:11.841+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:28:11.842+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:28:11.856+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:28:11.857+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:28:11.857+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:28:11.858+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:28:11.861+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:28:11.861+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:28:11.861+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:28:11.861+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:28:11.861+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:28:11.861+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:28:11.861+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:28:11.864+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:28:11.872+05:30 [Info] Indexer::ReadMemstats Time Taken 7.610759ms 2023-10-21T04:28:11.874+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:28:11.877+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 13.707629ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 1 error 2023-10-21T04:28:11.961+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {0a86361e1f5c834484c3516d42481e84 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:28:11.966+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 6.234µs 2023-10-21T04:28:11.966+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:28:11.966+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:28:11.967+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:28:11.971+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:28:11.971+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:28:11.971+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 4.491007ms 2023-10-21T04:28:11.971+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:28:11.971+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:28:11.971+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:28:11.971+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 7.645µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:11.971+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/0a86361e1f5c834484c3516d42481e84 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0a86361e1f5c834484c3516d42481e84]}]} 2023-10-21T04:28:11.973+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:28:11.981+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:11.983+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:11.981+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:28:11.984+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:28:11.991+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51064 2023-10-21T04:28:11.991+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:28:12.034+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:28:12.056+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51078 2023-10-21T04:28:12.061+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51080 2023-10-21T04:28:12.113+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 119.988521ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.114+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:28:12.114+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:28:12.114+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 20.024µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.117+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51086 2023-10-21T04:28:12.127+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 0 NodeUUIDs: [] 2023-10-21T04:28:12.128+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 1 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b] 2023-10-21T04:28:12.128+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 1 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b] 2023-10-21T04:28:12.128+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:28:12.128+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/0a86361e1f5c834484c3516d42481e84 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0a86361e1f5c834484c3516d42481e84]}]} 2023-10-21T04:28:12.132+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51092 2023-10-21T04:28:12.134+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51094 2023-10-21T04:28:12.143+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:28:12.162+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:28:12.188+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 126.565296ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.190+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.224µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.229+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:28:12.242+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 219] aggregate storage info: map[bucket_%2:map[kv:2279616 index:1413380] bucket_1:map[kv:2345152 index:1384628]] 2023-10-21T04:28:12.247+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34460" doAuth() ... 2023-10-21T04:28:12.249+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34460" doReceive() ... 2023-10-21T04:28:12.254+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51114 2023-10-21T04:28:12.256+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 134.754464ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.259+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.84µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.264+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 198.066033ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.265+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 11.135µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.282+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:28:12.294+05:30 [Info] switched currmeta from 18 -> 20 force true 2023-10-21T04:28:12.315+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 177.452255ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.316+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.865µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.323+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:12.323+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:12.324+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 188.456995ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.334+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34474" doAuth() ... 2023-10-21T04:28:12.334+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 15.486µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.334+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34474" doReceive() ... 2023-10-21T04:28:12.335+05:30 [Info] switched currmeta from 20 -> 20 force true 2023-10-21T04:28:12.335+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14283990141654320954:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:28:12.335+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6043349427976659583:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[14283990141654320954:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:28:12.335+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0] 14283990141654320954:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[6043349427976659583:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:28:12.335+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0] 14283990141654320954:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[6043349427976659583:[2 3 4 5 6 7 8 1] 14700121460508562886:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:28:12.335+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[5 6 7 8 1 2 3 4] 10601152754683910907:[0] 14283990141654320954:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[6043349427976659583:[2 3 4 5 6 7 8 1] 14700121460508562886:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:28:12.335+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[2 3 4 5 6 7 8 1] 14700121460508562886:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[5 6 7 8 1 2 3 4] 10601152754683910907:[0] 14283990141654320954:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:28:12.358+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:28:12.358+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:28:12.358+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:28:12.359+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:28:12.360+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34484" doAuth() ... 2023-10-21T04:28:12.362+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34484" doReceive() ... 2023-10-21T04:28:12.366+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 106.44613ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.368+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 11.572µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:12.377+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34488" doAuth() ... 2023-10-21T04:28:12.377+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34488" doReceive() ... 2023-10-21T04:28:12.385+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:28:12.386+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:28:12.387+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:28:12.389+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:28:12.438+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34498" doAuth() ... 2023-10-21T04:28:12.439+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34498" doReceive() ... 2023-10-21T04:28:12.445+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34500" doAuth() ... 2023-10-21T04:28:12.448+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34500" doReceive() ... 2023-10-21T04:28:12.563+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34508" doAuth() ... 2023-10-21T04:28:12.565+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34508" doReceive() ... 2023-10-21T04:28:12.609+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 2023-10-21T04:28:12.609+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842615494622315, Old rollbacktime: 0 2023-10-21T04:28:12.609+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4320881535426269172:[0] 8965923595543532942:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842615494622315, Old rollbacktime: 0 2023-10-21T04:28:12.609+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4320881535426269172:[0] 8965923595543532942:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0] 15864358084358521096:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842615494622315, Old rollbacktime: 0 2023-10-21T04:28:12.609+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4320881535426269172:[0] 8965923595543532942:[1 2 3 4 5 6 7 8] 17434304260397056806:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0] 15864358084358521096:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842615494622315, Old rollbacktime: 0 2023-10-21T04:28:12.609+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4320881535426269172:[0] 8965923595543532942:[1 2 3 4 5 6 7 8] 17434304260397056806:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0] 9142682180061755575:[0] 15864358084358521096:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842615494622315, Old rollbacktime: 0 2023-10-21T04:28:12.614+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:12.690+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:12.747+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {8806f5706747af4e635b8da5fcd8aeee 0a86361e1f5c834484c3516d42481e84 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:28:12.756+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:12.761+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {8806f5706747af4e635b8da5fcd8aeee 0a86361e1f5c834484c3516d42481e84 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:28:12.761+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"0a86361e1f5c834484c3516d42481e84","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:28:12.764+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:12.764+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{8806f5706747af4e635b8da5fcd8aeee 0a86361e1f5c834484c3516d42481e84 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:28:12.764+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{8806f5706747af4e635b8da5fcd8aeee 0a86361e1f5c834484c3516d42481e84 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:28:12.764+05:30 [Info] ShardRebalancer::observeRebalance &{8806f5706747af4e635b8da5fcd8aeee 0a86361e1f5c834484c3516d42481e84 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:28:12.781+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"0a86361e1f5c834484c3516d42481e84","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:28:12.781+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:28:12.817+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:12.825+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:12.867+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:12.972+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{8806f5706747af4e635b8da5fcd8aeee 0a86361e1f5c834484c3516d42481e84 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:28:13.040+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35918 2023-10-21T04:28:13.047+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35918 2023-10-21T04:28:13.075+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35918 2023-10-21T04:28:13.220+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken64:f5:b3:df:f7:af:55:66 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken16:85:61:79:1f:1d:d1:aa Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9142682180061755575 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11297114347036939030 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13684447062885654762 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.241+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken16:85:61:79:1f:1d:d1:aa MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken64:f5:b3:df:f7:af:55:66 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5722949411772971049 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14283990141654320954 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10872334221734875524 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.255+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken68:25:44:68:88:76:b:9c MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken74:e6:ce:b5:d1:f4:8c:55 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14700121460508562886 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17365236153427020369 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4804367503193957190 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.311+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken74:e6:ce:b5:d1:f4:8c:55 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken68:25:44:68:88:76:b:9c Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8835144498291080133 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4320881535426269172 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9659934401566352694 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.329+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken16:85:61:79:1f:1d:d1:aa MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken64:f5:b3:df:f7:af:55:66 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5722949411772971049 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14283990141654320954 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10872334221734875524 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.333+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken64:f5:b3:df:f7:af:55:66 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken16:85:61:79:1f:1d:d1:aa Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9142682180061755575 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11297114347036939030 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13684447062885654762 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.333+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:28:13.333+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:28:13.333+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:28:13.333+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 75.425µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:13.347+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken64:f5:b3:df:f7:af:55:66 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken16:85:61:79:1f:1d:d1:aa Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9142682180061755575 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11297114347036939030 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13684447062885654762 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.405+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken68:25:44:68:88:76:b:9c MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken74:e6:ce:b5:d1:f4:8c:55 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14700121460508562886 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17365236153427020369 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4804367503193957190 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.421+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken74:e6:ce:b5:d1:f4:8c:55 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken68:25:44:68:88:76:b:9c Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8835144498291080133 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4320881535426269172 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9659934401566352694 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.421+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:28:13.421+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:28:13.421+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:28:13.422+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.773µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:13.493+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken74:e6:ce:b5:d1:f4:8c:55 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken68:25:44:68:88:76:b:9c Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8835144498291080133 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4320881535426269172 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9659934401566352694 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.504+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34508 closed 2023-10-21T04:28:13.514+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = EOF. Cleaning Pipe. 2023-10-21T04:28:13.515+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly. 2023-10-21T04:28:13.515+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:51114. Error = read tcp4 127.0.0.1:9118->127.0.0.1:51114: read: connection reset by peer. Kill Pipe. 2023-10-21T04:28:13.515+05:30 [Info] messageListener.start(): message channel closed. Remove peer 93:7a:93:5:f9:ca:73:ae as follower. 2023-10-21T04:28:13.555+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken16:85:61:79:1f:1d:d1:aa MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken64:f5:b3:df:f7:af:55:66 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5722949411772971049 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14283990141654320954 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10872334221734875524 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.572+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:28:13.572+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/0a86361e1f5c834484c3516d42481e84 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0a86361e1f5c834484c3516d42481e84]}]} 2023-10-21T04:28:13.577+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:28:13.577+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:28:13.598+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/0a86361e1f5c834484c3516d42481e84, rev: [] 2023-10-21T04:28:13.598+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/0a86361e1f5c834484c3516d42481e84, rev: [] 2023-10-21T04:28:13.598+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:28:13.598+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:28:13.599+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:28:13.599+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:28:13.599+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:28:13.629+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:28:13.629+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken16:85:61:79:1f:1d:d1:aa MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken64:f5:b3:df:f7:af:55:66 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5722949411772971049 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14283990141654320954 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10872334221734875524 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.629+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken64:f5:b3:df:f7:af:55:66 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken16:85:61:79:1f:1d:d1:aa Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9142682180061755575 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11297114347036939030 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13684447062885654762 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.634+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken68:25:44:68:88:76:b:9c MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken74:e6:ce:b5:d1:f4:8c:55 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14700121460508562886 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17365236153427020369 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4804367503193957190 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.634+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken74:e6:ce:b5:d1:f4:8c:55 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0a86361e1f5c834484c3516d42481e84 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken68:25:44:68:88:76:b:9c Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8835144498291080133 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4320881535426269172 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9659934401566352694 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:28:13.681+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:28:13.681+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:28:13.681+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.7µs 2023-10-21T04:28:13.681+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:28:13.681+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:28:13.681+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:28:13.684+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:28:13.684+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:28:13.684+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:28:13.684+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:28:13.684+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 36.012µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:13.693+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:28:13.694+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:28:13.694+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:28:13.694+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:28:13.694+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:28:13.694+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:28:13.694+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:28:13.694+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 48.052µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:13.702+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:28:13.702+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:28:13.702+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/0a86361e1f5c834484c3516d42481e84, rev: [] call. err: 2023-10-21T04:28:13.702+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:28:13.702+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:28:13.712+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:28:13.712+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:28:13.736+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:28:13.741+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:28:13.741+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:28:13.742+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:28:13.743+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:28:14.011+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:14.011+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:14.045+05:30 [Info] switched currmeta from 20 -> 20 force true 2023-10-21T04:28:14.073+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 172] updated cluster rebalancing status to false 2023-10-21T04:28:14.105+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:28:14.146+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:14.146+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:14.161+05:30 [Info] switched currmeta from 20 -> 20 force true 2023-10-21T04:28:14.258+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9806387535763553, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:28:14.319+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:14.346+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:15.258+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6043367381394141, throttleDelayMs (new, old, change): (0, 3, -3) 2023-10-21T04:28:15.352+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:16.357+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:17.364+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:18.370+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:19.377+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:19.520+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:28:19.659+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51528 2023-10-21T04:28:19.678+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.622239ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:19.679+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 13.621µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:19.717+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34882" doAuth() ... 2023-10-21T04:28:19.717+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:34882" doReceive() ... 2023-10-21T04:28:20.179+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:28:20.382+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:28:39.315+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:39.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:39.323+05:30 [Info] switched currmeta from 20 -> 21 force true 2023-10-21T04:28:39.423+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:39.423+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:39.446+05:30 [Info] switched currmeta from 21 -> 24 force true 2023-10-21T04:28:39.556+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:39.556+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:39.570+05:30 [Info] switched currmeta from 24 -> 28 force true 2023-10-21T04:28:39.601+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:39.601+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:39.609+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:28:41.285+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:28:42.656+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:28:42.662+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:28:42.662+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:28:42.664+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:28:43.743+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:28:43.743+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:28:43.743+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:28:43.743+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:28:43.743+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:28:43.745+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:28:43.745+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:28:59.945+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 17797267771846799851 2023-10-21T04:28:59.945+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 329.875µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:28:59.995+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37092 2023-10-21T04:28:59.998+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37092 2023-10-21T04:29:00.022+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37092 2023-10-21T04:29:00.084+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 17797267771846799851 2023-10-21T04:29:00.084+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 305.754µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:00.134+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:00.134+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:00.140+05:30 [Info] switched currmeta from 28 -> 31 force true 2023-10-21T04:29:00.158+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:00.158+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:00.163+05:30 [Info] switched currmeta from 31 -> 33 force true 2023-10-21T04:29:00.168+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:00.168+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:00.171+05:30 [Info] switched currmeta from 33 -> 33 force true 2023-10-21T04:29:00.226+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:00.226+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:00.230+05:30 [Info] switched currmeta from 33 -> 35 force true 2023-10-21T04:29:00.333+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:00.333+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:00.335+05:30 [Info] switched currmeta from 35 -> 36 force true 2023-10-21T04:29:00.387+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:00.387+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:00.404+05:30 [Info] switched currmeta from 36 -> 37 force true 2023-10-21T04:29:00.419+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:00.419+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:00.433+05:30 [Info] switched currmeta from 37 -> 37 force true 2023-10-21T04:29:00.451+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52258 2023-10-21T04:29:00.467+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 13.240664ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:00.472+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52260 2023-10-21T04:29:00.500+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 22.269949ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:00.595+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 17797267771846799851. No update is performed. 2023-10-21T04:29:00.595+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 112.174µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:00.628+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52260. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:00.628+05:30 [Info] messageListener.start(): message channel closed. Remove peer 25:d:6d:f4:26:37:1a:d as follower. 2023-10-21T04:29:00.690+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 17797267771846799851. No update is performed. 2023-10-21T04:29:00.690+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 112.657µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:00.715+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52258. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:00.716+05:30 [Info] messageListener.start(): message channel closed. Remove peer d6:9e:a5:7f:43:be:a4:d9 as follower. 2023-10-21T04:29:00.968+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:01.000+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:01.168+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:29:01.168+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:29:01.177+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:29:01.201+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52296 2023-10-21T04:29:01.206+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:29:01.206+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:29:01.206+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:29:01.218+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:29:01.237+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 27.67476ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:01.256+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:29:01.256+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:29:01.257+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:29:01.266+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52300 2023-10-21T04:29:01.269+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:29:01.289+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.456738ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:01.289+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:29:01.289+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:29:01.289+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:29:01.307+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52296. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:01.308+05:30 [Info] messageListener.start(): message channel closed. Remove peer 8d:a8:3:9d:a2:59:e7:40 as follower. 2023-10-21T04:29:01.316+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:29:01.330+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:29:01.330+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:29:01.330+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:29:01.330+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:29:01.330+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:29:01.330+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:29:01.330+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:29:01.330+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:29:01.330+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:29:01.330+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:29:01.331+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:52300->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:01.331+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:48678->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:01.331+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:43546->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:01.332+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:44008->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:01.332+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52300. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:01.332+05:30 [Info] messageListener.start(): message channel closed. Remove peer f1:ee:56:63:a7:1c:f6:f6 as follower. 2023-10-21T04:29:01.489+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:01.490+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:01.495+05:30 [Info] switched currmeta from 37 -> 38 force true 2023-10-21T04:29:01.542+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:01.542+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:01.547+05:30 [Info] switched currmeta from 38 -> 39 force true 2023-10-21T04:29:01.737+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:01.790+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:02.051+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:02.051+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:02.059+05:30 [Info] switched currmeta from 39 -> 40 force true 2023-10-21T04:29:02.190+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:02.190+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:02.193+05:30 [Info] switched currmeta from 40 -> 41 force true 2023-10-21T04:29:02.723+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:29:02.731+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:02.731+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:02.734+05:30 [Info] switched currmeta from 41 -> 41 force true 2023-10-21T04:29:03.109+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842615494622315, Old rollbacktime: 0 2023-10-21T04:29:03.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:29:03.372+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:29:03.372+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:29:08.478+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 12824340686309573428 2023-10-21T04:29:08.478+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 179.157µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:08.547+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37092 2023-10-21T04:29:08.560+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37092 2023-10-21T04:29:08.603+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37092 2023-10-21T04:29:08.702+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 12824340686309573428 2023-10-21T04:29:08.702+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 234.102µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:08.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:08.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:08.772+05:30 [Info] switched currmeta from 41 -> 44 force true 2023-10-21T04:29:08.820+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:08.820+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:08.835+05:30 [Info] switched currmeta from 44 -> 46 force true 2023-10-21T04:29:08.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:08.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:08.883+05:30 [Info] switched currmeta from 46 -> 48 force true 2023-10-21T04:29:08.904+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:08.904+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:08.937+05:30 [Info] switched currmeta from 48 -> 49 force true 2023-10-21T04:29:09.090+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:09.090+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:09.095+05:30 [Info] switched currmeta from 49 -> 50 force true 2023-10-21T04:29:09.100+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:09.100+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:09.103+05:30 [Info] switched currmeta from 50 -> 50 force true 2023-10-21T04:29:09.142+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:29:09.142+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:29:09.152+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:29:09.176+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52506 2023-10-21T04:29:09.178+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:29:09.178+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:29:09.178+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:29:09.189+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:29:09.196+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.340441ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:09.213+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:29:09.213+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:29:09.213+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:29:09.225+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52518 2023-10-21T04:29:09.227+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:29:09.261+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 12824340686309573428. No update is performed. 2023-10-21T04:29:09.261+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 351.246µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:09.275+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 45.353095ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:09.275+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:29:09.275+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:29:09.275+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:29:09.297+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52506. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:09.297+05:30 [Info] messageListener.start(): message channel closed. Remove peer d3:23:e1:63:60:56:c1:e9 as follower. 2023-10-21T04:29:09.304+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:29:09.401+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:29:09.401+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:29:09.401+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:29:09.401+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:29:09.402+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:29:09.402+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:29:09.402+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:29:09.402+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:29:09.402+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:29:09.402+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:29:09.402+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:43756->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:09.402+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:44218->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:09.402+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:52518->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:09.403+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:48898->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:29:09.403+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52518. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:09.403+05:30 [Info] messageListener.start(): message channel closed. Remove peer f4:7d:78:f8:a9:cd:ec:84 as follower. 2023-10-21T04:29:09.414+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52530 2023-10-21T04:29:09.482+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 63.590249ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:09.533+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52542 2023-10-21T04:29:09.544+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52530. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:09.544+05:30 [Info] messageListener.start(): message channel closed. Remove peer 0:4a:8c:8d:bc:43:89:66 as follower. 2023-10-21T04:29:09.585+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 48.095109ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:09.644+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52542. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:09.644+05:30 [Info] messageListener.start(): message channel closed. Remove peer ee:db:92:81:7b:cc:63:25 as follower. 2023-10-21T04:29:09.698+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:09.776+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:09.983+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:10.086+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:10.240+05:30 [Info] fragAutoTuner: FragRatio at 100. MaxFragRatio 100, MaxBandwidth 0. BandwidthUsage 0. AvailDisk 0. TotalUsed 0. BandwidthRatio 1. UsedSpaceRatio 1. CleanerBandwidth 0. Duration 0. 2023-10-21T04:29:10.384+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:10.386+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:10.392+05:30 [Info] switched currmeta from 50 -> 52 force true 2023-10-21T04:29:10.406+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:10.406+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:10.413+05:30 [Info] switched currmeta from 52 -> 52 force true 2023-10-21T04:29:10.982+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:10.982+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:10.984+05:30 [Info] switched currmeta from 52 -> 53 force true 2023-10-21T04:29:11.033+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:11.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:11.045+05:30 [Info] switched currmeta from 53 -> 54 force true 2023-10-21T04:29:11.283+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:29:11.283+05:30 [Info] janitor: running cleanup. 2023-10-21T04:29:11.283+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[17656144409930763141 8669649359784409226 11365633301282701856 5465835793930010953 573521267164877540 12195715103909237684 1252359852432998131 2048006213723315482 4535197604682045916 4860549750398470299 8857269605176042330 14292583894719865678 15431181238515168303 18323460706027642575 249887442666643345 8504686614261377811 3674078063353481907 6985049432225870291 8378417662949994532 14889401051926000176 4138047918192025493 4532813151040661705 12338036531800130039 12833670934079048728 13856704970496601958 12140458060295645238 15945117906390072636 15968368593314043432 16162012140118277065 7392796230534034390 7525085113233084253 11619165464560280276 12027054384338714804 14741949230702299196 18026660730857541767 18046371353728341033]} 2023-10-21T04:29:11.287+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:29:11.291+05:30 [Info] memstats {"Alloc":86512168, "TotalAlloc":89322992, "Sys":104808728, "Lookups":0, "Mallocs":99623,"Frees":24271, "HeapAlloc":86512168, "HeapSys":90636288, "HeapIdle":2048000, "HeapInuse":88588288,"HeapReleased":1982464, "HeapObjects":75352,"MSpanInuse":239680, "MSpanSys": 244800, "StackInuse": 1638400,"GCSys":9832056, "LastGC":1697842690926061081,"PauseTotalNs":37815885, "PauseNs":[622226, 411576, 48793, 706098, 36027192], "NumGC":5} 2023-10-21T04:29:11.823+05:30 [Info] connected with 4 indexers 2023-10-21T04:29:11.823+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:29:11.865+05:30 [Info] num concurrent scans {0} 2023-10-21T04:29:11.866+05:30 [Info] average scan response {0 ms} 2023-10-21T04:29:11.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:11.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:11.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:11.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:11.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:11.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:11.878+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:29:11.902+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:29:11.961+05:30 [Info] Indexer::ReadMemstats Time Taken 420.329µs 2023-10-21T04:29:12.108+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 2023-10-21T04:29:12.108+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 2023-10-21T04:29:12.108+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 2023-10-21T04:29:12.108+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 2023-10-21T04:29:12.108+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 2023-10-21T04:29:12.108+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842641301316843, Old rollbacktime: 0 2023-10-21T04:29:12.405+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:29:12.430+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:29:12.765+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:29:12.772+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:12.772+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:12.775+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:29:13.746+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:29:13.746+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:29:13.746+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:29:13.747+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:29:13.747+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:29:13.749+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:29:13.749+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:29:23.327+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:29:23.666+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:29:23.765+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:23.765+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:23.779+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:29:23.921+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 172] updated cluster rebalancing status to true 2023-10-21T04:29:23.921+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 172] updating rebalancing status to true 2023-10-21T04:29:23.985+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:23.985+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:23.992+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:29:24.114+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:29:24.146+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:24.147+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:24.149+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:29:25.497+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {d2097fed4a508b720b6abe26f495f195 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:29:25.506+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.926µs 2023-10-21T04:29:25.506+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:29:25.506+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:29:25.506+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:29:25.509+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:29:25.509+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:29:25.509+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.995722ms 2023-10-21T04:29:25.509+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:29:25.509+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:29:25.509+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:29:25.509+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.97µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:25.509+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/d2097fed4a508b720b6abe26f495f195 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d2097fed4a508b720b6abe26f495f195]}]} 2023-10-21T04:29:25.516+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:29:25.518+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:29:26.574+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {8806f5706747af4e635b8da5fcd8aeee d2097fed4a508b720b6abe26f495f195 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:26.575+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {8806f5706747af4e635b8da5fcd8aeee d2097fed4a508b720b6abe26f495f195 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:26.576+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"d2097fed4a508b720b6abe26f495f195","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:29:26.578+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{8806f5706747af4e635b8da5fcd8aeee d2097fed4a508b720b6abe26f495f195 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:26.578+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{8806f5706747af4e635b8da5fcd8aeee d2097fed4a508b720b6abe26f495f195 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:29:26.578+05:30 [Info] ShardRebalancer::observeRebalance &{8806f5706747af4e635b8da5fcd8aeee d2097fed4a508b720b6abe26f495f195 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:29:26.582+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"d2097fed4a508b720b6abe26f495f195","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:29:26.582+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:29:26.720+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36384 2023-10-21T04:29:26.727+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36384 2023-10-21T04:29:26.755+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36384 2023-10-21T04:29:26.850+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:6c:2e:2a:3c:51:ea:e3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf2:2e:6e:90:74:5:92:2b Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1935423091803896170 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13109121521117022001 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:26.911+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf2:2e:6e:90:74:5:92:2b MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:6c:2e:2a:3c:51:ea:e3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14437379019640061228 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16718498499208256230 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:26.912+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenfc:42:88:1a:eb:60:57:7f MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:67:3b:2d:8f:23:c6:71 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10544176905436936582 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6192631895091929726 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:26.955+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf2:2e:6e:90:74:5:92:2b MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:6c:2e:2a:3c:51:ea:e3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14437379019640061228 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16718498499208256230 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.022+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:67:3b:2d:8f:23:c6:71 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenfc:42:88:1a:eb:60:57:7f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2993550332778537654 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 81841348502950914 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.025+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:6c:2e:2a:3c:51:ea:e3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf2:2e:6e:90:74:5:92:2b Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1935423091803896170 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13109121521117022001 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.025+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:29:27.025+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:29:27.025+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:29:27.025+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 39.151µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:27.045+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenfc:42:88:1a:eb:60:57:7f MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:67:3b:2d:8f:23:c6:71 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10544176905436936582 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6192631895091929726 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.045+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:6c:2e:2a:3c:51:ea:e3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf2:2e:6e:90:74:5:92:2b Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1935423091803896170 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13109121521117022001 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.089+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf2:2e:6e:90:74:5:92:2b MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:6c:2e:2a:3c:51:ea:e3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14437379019640061228 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16718498499208256230 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.116+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:67:3b:2d:8f:23:c6:71 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenfc:42:88:1a:eb:60:57:7f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2993550332778537654 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 81841348502950914 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.116+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:29:27.116+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:29:27.116+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:29:27.116+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.153µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:27.143+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenfc:42:88:1a:eb:60:57:7f MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:67:3b:2d:8f:23:c6:71 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10544176905436936582 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6192631895091929726 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.146+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:67:3b:2d:8f:23:c6:71 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenfc:42:88:1a:eb:60:57:7f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2993550332778537654 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 81841348502950914 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.236+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"d2097fed4a508b720b6abe26f495f195","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:29:27.236+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:29:27.236+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:29:27.236+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:29:27.236+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:29:27.236+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:29:27.236+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 19.23µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:27.251+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:6c:2e:2a:3c:51:ea:e3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf2:2e:6e:90:74:5:92:2b Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1935423091803896170 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13109121521117022001 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.298+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf2:2e:6e:90:74:5:92:2b MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:6c:2e:2a:3c:51:ea:e3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14437379019640061228 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16718498499208256230 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.298+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenfc:42:88:1a:eb:60:57:7f MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:67:3b:2d:8f:23:c6:71 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10544176905436936582 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6192631895091929726 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.299+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:67:3b:2d:8f:23:c6:71 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenfc:42:88:1a:eb:60:57:7f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2993550332778537654 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 81841348502950914 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.458+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenfc:42:88:1a:eb:60:57:7f MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:67:3b:2d:8f:23:c6:71 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10544176905436936582 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6192631895091929726 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.491+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf2:2e:6e:90:74:5:92:2b MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:6c:2e:2a:3c:51:ea:e3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14437379019640061228 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16718498499208256230 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.506+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:34488 closed 2023-10-21T04:29:27.507+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:51086. Error = read tcp4 127.0.0.1:9118->127.0.0.1:51086: read: connection reset by peer. Kill Pipe. 2023-10-21T04:29:27.507+05:30 [Info] messageListener.start(): message channel closed. Remove peer 36:c5:1b:f3:e3:68:28:d4 as follower. 2023-10-21T04:29:27.511+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{8806f5706747af4e635b8da5fcd8aeee d2097fed4a508b720b6abe26f495f195 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:29:27.514+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = EOF. Cleaning Pipe. 2023-10-21T04:29:27.514+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly. 2023-10-21T04:29:27.595+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:29:27.595+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/d2097fed4a508b720b6abe26f495f195 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d2097fed4a508b720b6abe26f495f195]}]} 2023-10-21T04:29:27.598+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:29:27.598+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:29:27.600+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/d2097fed4a508b720b6abe26f495f195, rev: [] 2023-10-21T04:29:27.601+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/d2097fed4a508b720b6abe26f495f195, rev: [] 2023-10-21T04:29:27.601+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:29:27.601+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:29:27.601+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:29:27.601+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:29:27.601+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:29:27.683+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:29:27.683+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken3a:67:3b:2d:8f:23:c6:71 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenfc:42:88:1a:eb:60:57:7f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2993550332778537654 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 81841348502950914 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.683+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokena9:6c:2e:2a:3c:51:ea:e3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf2:2e:6e:90:74:5:92:2b Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1935423091803896170 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13109121521117022001 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.684+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenf2:2e:6e:90:74:5:92:2b MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:6c:2e:2a:3c:51:ea:e3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14437379019640061228 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16718498499208256230 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.684+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenfc:42:88:1a:eb:60:57:7f MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: d2097fed4a508b720b6abe26f495f195 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:67:3b:2d:8f:23:c6:71 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10544176905436936582 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 6192631895091929726 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:27.727+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokenf2:2e:6e:90:74:5:92:2b 2023-10-21T04:29:27.727+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: d2097fed4a508b720b6abe26f495f195 TransferTokenId: ShardTokenf2:2e:6e:90:74:5:92:2b Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:29:27.727+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000116426 2023-10-21T04:29:27.727+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokenfc:42:88:1a:eb:60:57:7f 2023-10-21T04:29:27.727+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardTokenf2_2e_6e_90_74_5_92_2b, shardId :0 2023-10-21T04:29:27.728+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: d2097fed4a508b720b6abe26f495f195 TransferTokenId: ShardTokenfc:42:88:1a:eb:60:57:7f Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:29:27.728+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 8.456e-05 2023-10-21T04:29:27.729+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardToken3a:67:3b:2d:8f:23:c6:71 2023-10-21T04:29:27.729+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardTokenfc_42_88_1a_eb_60_57_7f, shardId :0 2023-10-21T04:29:27.730+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: d2097fed4a508b720b6abe26f495f195 TransferTokenId: ShardToken3a:67:3b:2d:8f:23:c6:71 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:29:27.731+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000156956 2023-10-21T04:29:27.731+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokena9:6c:2e:2a:3c:51:ea:e3 2023-10-21T04:29:27.731+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardToken3a_67_3b_2d_8f_23_c6_71, shardId :0 2023-10-21T04:29:27.731+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: d2097fed4a508b720b6abe26f495f195 TransferTokenId: ShardTokena9:6c:2e:2a:3c:51:ea:e3 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:29:27.731+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 5.7247e-05 2023-10-21T04:29:27.731+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:29:27.731+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:29:27.731+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.76µs 2023-10-21T04:29:27.731+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardTokena9_6c_2e_2a_3c_51_ea_e3, shardId :0 2023-10-21T04:29:27.733+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardToken3a_67_3b_2d_8f_23_c6_71, shardId :0 2023-10-21T04:29:27.733+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardTokenfc_42_88_1a_eb_60_57_7f, shardId :0 2023-10-21T04:29:27.733+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:29:27.733+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:29:27.733+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:29:27.733+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardTokena9_6c_2e_2a_3c_51_ea_e3, shardId :0 2023-10-21T04:29:27.741+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:29:27.741+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:29:27.741+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:29:27.738+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d2097fed4a508b720b6abe26f495f195_ShardTokenf2_2e_6e_90_74_5_92_2b, shardId :0 2023-10-21T04:29:27.742+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:29:27.751+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 8.713777ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:27.745+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:29:27.751+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:29:27.751+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:29:27.751+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:29:27.751+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:29:27.751+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:29:27.753+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:29:27.753+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 65.088µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:27.757+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:29:27.757+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:29:27.757+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/d2097fed4a508b720b6abe26f495f195, rev: [] call. err: 2023-10-21T04:29:27.757+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:29:27.757+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:29:27.766+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:29:27.766+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:29:27.783+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:29:27.783+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:29:27.784+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:29:27.795+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:29:27.835+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:29:28.112+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:28.112+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:28.124+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:29:28.205+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:29:28.227+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:28.227+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:28.238+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:29:28.317+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 172] updated cluster rebalancing status to false 2023-10-21T04:29:28.898+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:29:28.983+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:29:29.259+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9626971146420308, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:29:29.991+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:29:30.258+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.84575495764054, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:29:30.520+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:29:30.996+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:29:31.700+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53170 2023-10-21T04:29:31.721+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 17.744279ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:31.723+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.405µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:31.764+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:36524" doAuth() ... 2023-10-21T04:29:31.766+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:36524" doReceive() ... 2023-10-21T04:29:32.002+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36128 2023-10-21T04:29:32.222+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:32.919+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:29:32.924+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:32.924+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:32.926+05:30 [Info] switched currmeta from 54 -> 58 force true 2023-10-21T04:29:35.639+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 1697842641301316843 2023-10-21T04:29:35.639+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 1697842641301316843 2023-10-21T04:29:35.639+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 1697842641301316843 2023-10-21T04:29:35.639+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 1697842641301316843 {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 1697842615494622315 2023-10-21T04:29:35.639+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 1697842641301316843 {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 1697842615494622315 2023-10-21T04:29:35.639+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 1697842641301316843 {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 1697842615494622315 2023-10-21T04:29:41.310+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:29:45.789+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:29:45.858+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:45.858+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:45.871+05:30 [Info] switched currmeta from 58 -> 58 force true 2023-10-21T04:29:45.906+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:29:45.940+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:45.940+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:45.944+05:30 [Info] switched currmeta from 58 -> 58 force true 2023-10-21T04:29:45.956+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 172] updated cluster rebalancing status to true 2023-10-21T04:29:46.034+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:29:46.113+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:46.113+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:46.115+05:30 [Info] switched currmeta from 58 -> 58 force true 2023-10-21T04:29:47.403+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {b07e15c0226c8d6f8b5a5c433f28c2b8 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:29:47.405+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.841µs 2023-10-21T04:29:47.405+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:29:47.405+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:29:47.405+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:29:47.410+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:29:47.410+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:29:47.410+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 5.129485ms 2023-10-21T04:29:47.410+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:29:47.410+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:29:47.410+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:29:47.410+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 5.951µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:47.410+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/b07e15c0226c8d6f8b5a5c433f28c2b8 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:b07e15c0226c8d6f8b5a5c433f28c2b8]}]} 2023-10-21T04:29:47.415+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 6] 2023-10-21T04:29:47.415+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 6] 2023-10-21T04:29:48.503+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:48.504+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:48.504+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"b07e15c0226c8d6f8b5a5c433f28c2b8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:29:48.506+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:48.506+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:29:48.506+05:30 [Info] ShardRebalancer::observeRebalance &{8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:29:48.509+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"b07e15c0226c8d6f8b5a5c433f28c2b8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:29:48.509+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:29:48.612+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36384 2023-10-21T04:29:48.621+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36384 2023-10-21T04:29:48.647+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:36384 2023-10-21T04:29:48.730+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken99:9:6f:7f:a9:41:54:e7 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene:c0:34:bb:7f:e7:f1:19 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17202021211694468116 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4632491965115421117 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.758+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken72:2e:47:3f:e7:f8:64:25 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena0:32:ed:c2:60:29:55:b2 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8540516091499309889 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16513374393624409199 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.762+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena0:32:ed:c2:60:29:55:b2 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken72:2e:47:3f:e7:f8:64:25 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16160071316590384036 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15314206422277636043 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.844+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene:c0:34:bb:7f:e7:f1:19 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken99:9:6f:7f:a9:41:54:e7 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16218533996744021123 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3412896864974143866 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.859+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken72:2e:47:3f:e7:f8:64:25 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena0:32:ed:c2:60:29:55:b2 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8540516091499309889 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16513374393624409199 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.860+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:29:48.860+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:29:48.860+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:29:48.860+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.628µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:48.882+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken99:9:6f:7f:a9:41:54:e7 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene:c0:34:bb:7f:e7:f1:19 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17202021211694468116 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4632491965115421117 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.883+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:29:48.883+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:29:48.883+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:29:48.884+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.627µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:48.908+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken72:2e:47:3f:e7:f8:64:25 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena0:32:ed:c2:60:29:55:b2 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8540516091499309889 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16513374393624409199 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.909+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken99:9:6f:7f:a9:41:54:e7 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene:c0:34:bb:7f:e7:f1:19 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17202021211694468116 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4632491965115421117 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:48.954+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena0:32:ed:c2:60:29:55:b2 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken72:2e:47:3f:e7:f8:64:25 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16160071316590384036 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15314206422277636043 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.028+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene:c0:34:bb:7f:e7:f1:19 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken99:9:6f:7f:a9:41:54:e7 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16218533996744021123 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3412896864974143866 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.054+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena0:32:ed:c2:60:29:55:b2 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken72:2e:47:3f:e7:f8:64:25 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16160071316590384036 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15314206422277636043 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.115+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene:c0:34:bb:7f:e7:f1:19 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken99:9:6f:7f:a9:41:54:e7 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16218533996744021123 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3412896864974143866 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.152+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"b07e15c0226c8d6f8b5a5c433f28c2b8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:29:49.152+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:29:49.152+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:29:49.152+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:29:49.152+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:29:49.152+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:29:49.152+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 18.12µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:49.207+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken99:9:6f:7f:a9:41:54:e7 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene:c0:34:bb:7f:e7:f1:19 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17202021211694468116 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4632491965115421117 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.225+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken72:2e:47:3f:e7:f8:64:25 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena0:32:ed:c2:60:29:55:b2 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8540516091499309889 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16513374393624409199 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.229+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena0:32:ed:c2:60:29:55:b2 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken72:2e:47:3f:e7:f8:64:25 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16160071316590384036 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15314206422277636043 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.231+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene:c0:34:bb:7f:e7:f1:19 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken99:9:6f:7f:a9:41:54:e7 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16218533996744021123 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3412896864974143866 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.412+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:29:49.433+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene:c0:34:bb:7f:e7:f1:19 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken99:9:6f:7f:a9:41:54:e7 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16218533996744021123 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3412896864974143866 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.568+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken99:9:6f:7f:a9:41:54:e7 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene:c0:34:bb:7f:e7:f1:19 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17202021211694468116 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4632491965115421117 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.569+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:29:49.569+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:29:49.569+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:29:49.569+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.874µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:49.569+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[10170029854712240877:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/10170029854712240877/shards/shard10170029854712240877 17406544375239214575:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/17406544375239214575/shards/shard17406544375239214575] b07e15c0226c8d6f8b5a5c433f28c2b8 ShardToken99:9:6f:7f:a9:41:54:e7 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc004fa3380 0xc004fa33e0 map[] 0xc00789df80 0xc00777c7e0 false} 2023-10-21T04:29:49.569+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:29:49.569+05:30 [Info] Directory /opt/build/ns_server/data/n_3/data/@2i/shards does not exist. Skip shard recovery. 2023-10-21T04:29:49.569+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/17406544375239214575/shards/shard17406544375239214575 2023-10-21T04:29:49.571+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/10170029854712240877/shards/shard10170029854712240877 2023-10-21T04:29:49.571+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene:c0:34:bb:7f:e7:f1:19 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken99:9:6f:7f:a9:41:54:e7 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16218533996744021123 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3412896864974143866 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:49.573+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/10170029854712240877/shards/shard10170029854712240877/data: super block snapshot head offset :0 tail offset:651264 sbgen :126 2023-10-21T04:29:49.573+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/10170029854712240877/shards/shard10170029854712240877/data/recovery: super block snapshot head offset :0 tail offset:593920 sbgen :126 2023-10-21T04:29:49.580+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/17406544375239214575/shards/shard17406544375239214575/data: super block snapshot head offset :0 tail offset:757760 sbgen :136 2023-10-21T04:29:49.581+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/17406544375239214575/shards/shard17406544375239214575/data/recovery: super block snapshot head offset :0 tail offset:708608 sbgen :136 2023-10-21T04:29:49.582+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/10170029854712240877/shards/shard10170029854712240877: metadata saved successfully 2023-10-21T04:29:49.583+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:29:49.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:29:49.584+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 90306429 totalBytes: 1264290 numShards: 2 duration: 14.516597ms 2023-10-21T04:29:49.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/10170029854712240877/shards/shard10170029854712240877, newshardId 10170029854712240877, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, shardsize 1264290, duration 13.189118ms 2023-10-21T04:29:49.584+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10170029854712240877, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, err: , elapsed(sec): 0.014867423 2023-10-21T04:29:49.584+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10170029854712240877 bytesWritten: 1264290, totalBytes: 1264290, transferRate: 9.725307692307693e+07 2023-10-21T04:29:49.589+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/17406544375239214575/shards/shard17406544375239214575: metadata saved successfully 2023-10-21T04:29:49.590+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:29:49.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:29:49.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7/17406544375239214575/shards/shard17406544375239214575, newshardId 17406544375239214575, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, shardsize 1485636, duration 21.374746ms 2023-10-21T04:29:49.591+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17406544375239214575, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, err: , elapsed(sec): 0.021757961 2023-10-21T04:29:49.591+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17406544375239214575 bytesWritten: 1485636, totalBytes: 1485636, transferRate: 7.074457142857143e+07 2023-10-21T04:29:49.591+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: b07e15c0226c8d6f8b5a5c433f28c2b8, transferId: ShardToken99:9:6f:7f:a9:41:54:e7 2023-10-21T04:29:49.591+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: b07e15c0226c8d6f8b5a5c433f28c2b8, transferId: ShardToken99:9:6f:7f:a9:41:54:e7, elapsed(sec): 0.000209335 2023-10-21T04:29:49.591+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[10170029854712240877: 17406544375239214575:], shardPaths: map[10170029854712240877:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 17406544375239214575:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.022033535 2023-10-21T04:29:49.591+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17406544375239214575 10170029854712240877], ttid: ShardToken99:9:6f:7f:a9:41:54:e7, elapsed(sec): 0.022518202 2023-10-21T04:29:49.616+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 3, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 panic: TestCode::TestActionAtTag - Inducing artificial panic at tag: 3 as wished goroutine 2621 [running]: github.com/couchbase/indexing/secondary/testcode.TestActionAtTag(0xc006296f40?, 0x3) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/testcode/test_action_ci.go:45 +0x5e5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).startShardRestore(0xc00790c000, {0xc00776cc54, 0x20}, 0xc0078de400) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1606 +0x9d5 created by github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferTokenAsDest /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1421 +0x39b 2023-10-21T04:29:52.845+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=18194302e3f10df59bd076e1c4142d0b -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:29:52.845+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:29:52.847+05:30 [Info] Setting ipv6=false 2023-10-21T04:29:52.909+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:29:52.951+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:29:52.957+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:29:52.957+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:29:52.957+05:30 [Info] Setting log level to Info 2023-10-21T04:29:52.957+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:29:52.957+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:29:52.958+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:29:52.958+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:29:53.008+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:29:53 audit: created new audit service 2023-10-21T04:29:53.215+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:29:53.216+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:29:53.216+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:29:53.217+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:29:53.217+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:29:53.217+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:29:53.247+05:30 [Info] security context initialized 2023-10-21T04:29:53.248+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:29:53.249+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:29:53.249+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:29:53.249+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:29:53.249+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:29:53.258+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:29:53.258+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:29:53.258+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:29:53.258+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:29:53.265+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:29:53.271+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:29:53.273+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:29:53.274+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:29:53.299+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:29:53.300+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:29:53.302+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:29:53.302+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:29:53.302+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:29:53.336+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:29:53.336+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:29:53.338+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:29:53.346+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:29:53.346+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:29:53.349+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:29:53.349+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:29:53.349+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:29:53.350+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:29:53.350+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:29:53.352+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 232 us 2023-10-21T04:29:53.355+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:29:53.375+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:29:53.376+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:29:53.389+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:29:53.389+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:29:53.389+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:29:53.389+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:29:53.389+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:29:53.389+05:30 [Info] Indexer Id 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:29:53.389+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:29:53.390+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:29:53.393+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:29:53.394+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:29:53.398+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 110] updating cluster rev epoch 0 -> 1 2023-10-21T04:29:53.398+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 110] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:29:53.409+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 213] bucketNames has been updated (was map[], now map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43]) 2023-10-21T04:29:53.409+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 214] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:29:53.410+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 215] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:29:53.415+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 213] updated cluster rebalancing status to false 2023-10-21T04:29:53.415+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:29:53.415+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:29:53.415+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:29:53.441+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:29:53.441+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:29:53.441+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:29:53.442+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:29:53.442+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning true RebalanceToken &{8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:53.442+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:29:53.443+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:29:53.443+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:29:53.443+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:29:53.458+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:29:53.458+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:29:53.460+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:29:53.463+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:29:53.464+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:29:53.476+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap , elapsed: 32.867457ms 2023-10-21T04:29:53.476+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:29:53.476+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:29:53.476+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:29:53.476+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager true &{8806f5706747af4e635b8da5fcd8aeee b07e15c0226c8d6f8b5a5c433f28c2b8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:29:53.477+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:29:53.477+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:29:53.477+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:29:53.477+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:29:53.479+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:29:53.479+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:29:53.480+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:29:53.480+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:29:53.483+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:29:53.483+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:29:53.483+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:29:53.483+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:29:53.484+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:29:53.486+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:29:53.486+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:29:53.488+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:29:53.488+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:false Messages:[]} 2023-10-21T04:29:53.489+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:29:53.489+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:29:53.495+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:29:53.495+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:29:53.499+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:29:53.500+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:29:53.500+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:29:53.500+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:29:53.501+05:30 [Info] RebalanceServiceManager::recoverRebalance: Init Pending Cleanup 2023-10-21T04:29:53.501+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:29:53.501+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:29:53.504+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:29:53.504+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:29:53.504+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:29:53.504+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:29:53.504+05:30 [Info] progress stats collection done. 2023-10-21T04:29:53.504+05:30 [Info] send progress stats to clients 2023-10-21T04:29:53.506+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:29:53.508+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:29:53.509+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:29:53.513+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:29:53.514+05:30 [Info] Setting log level to Info 2023-10-21T04:29:53.514+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:29:53.516+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:29:53.522+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:29:53.522+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:29:53.522+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:29:53.523+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:29:53.542+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:29:53.543+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:29:53.560+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:29:53.585+05:30 [Info] security context initialized 2023-10-21T04:29:53.585+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:29:53.589+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:29:53.610+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:29:53.610+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:29:53.610+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:29:53.610+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:29:53.622+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:29:53.642+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:29:53.642+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:29:53.644+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:29:53.647+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:29:53.647+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:29:53.673+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:29:53.673+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:29:53.679+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:29:53.708+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:29:53.708+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:29:53.712+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:29:53.715+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53950 2023-10-21T04:29:53.716+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:29:53.729+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 11.702898ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:53.729+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:29:53.729+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:29:53.730+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 22.821µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:53.745+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:29:53.773+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:29:53.776+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:29:53.776+05:30 [Info] initialized currmeta 22 force true 2023-10-21T04:29:53.781+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:29:53.781+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:29:53.781+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:29:53.783+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:29:53.790+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:29:53.790+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:29:53.790+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:29:53.791+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:29:53.799+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:29:53.799+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:29:53.799+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:29:53.800+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:29:53.801+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:37306" doAuth() ... 2023-10-21T04:29:53.802+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:37306" doReceive() ... 2023-10-21T04:29:53.807+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:29:53.807+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:29:53.807+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:29:53.808+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:29:53.810+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:29:53.811+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:29:53.811+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:29:53.811+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:29:53.811+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:29:53.811+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:29:53.811+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:29:53.811+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:29:53.812+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken72:2e:47:3f:e7:f8:64:25 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena0:32:ed:c2:60:29:55:b2 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8540516091499309889 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16513374393624409199 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:53.812+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:29:53.812+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: b07e15c0226c8d6f8b5a5c433f28c2b8 TransferTokenId: ShardToken72:2e:47:3f:e7:f8:64:25 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:29:53.812+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000346472 2023-10-21T04:29:53.813+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken72:2e:47:3f:e7:f8:64:25, dropIndexes: false 2023-10-21T04:29:53.813+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken72_2e_47_3f_e7_f8_64_25, shardId :0 2023-10-21T04:29:53.813+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken72_2e_47_3f_e7_f8_64_25, shardId :0 2023-10-21T04:29:53.813+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:29:53.818+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 5.317431ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:53.840+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 57] will report aggregate recorder stats every 5m0s 2023-10-21T04:29:53.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:29:53.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:53.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:53.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:29:53.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:29:53.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[6 7 8 1 2 3 4 5] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:29:53.879+05:30 [Info] Indexer::ReadMemstats Time Taken 549.303µs 2023-10-21T04:29:54.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:29:54.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:29:54.143+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[3 4 5 6 7 8 1 2] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:29:54.143+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[3 4 5 6 7 8 1 2] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:29:54.143+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[3 4 5 6 7 8 1 2] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:29:54.144+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[3 4 5 6 7 8 1 2] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:29:54.230+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:29:54.436+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 249] aggregate storage info: map[bucket_%2:map[kv:2279616 index:1413380] bucket_1:map[kv:2345152 index:1384628]] 2023-10-21T04:29:54.497+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 16787445029868804812 2023-10-21T04:29:54.497+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2925063509317026816 2023-10-21T04:29:54.497+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 16787445029868804812 2023-10-21T04:29:54.497+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2925063509317026816 2023-10-21T04:29:54.497+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [16787445029868804812 2925063509317026816], elapsed: 684.460422ms 2023-10-21T04:29:54.497+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:29:54.498+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:29:54.498+05:30 [Info] : Recover shard shard10170029854712240877 from metadata completed. 2023-10-21T04:29:54.498+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:29:54.498+05:30 [Info] : Recover shard shard17406544375239214575 from metadata completed. 2023-10-21T04:29:54.498+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 16787445029868804812 from local file system, err: error destroy shard, shardId 16787445029868804812 not found 2023-10-21T04:29:54.498+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 2925063509317026816 from local file system, err: error destroy shard, shardId 2925063509317026816 not found 2023-10-21T04:29:54.498+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [16787445029868804812 2925063509317026816], elapsed(sec): 0.685896635 2023-10-21T04:29:54.501+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken99:9:6f:7f:a9:41:54:e7 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b07e15c0226c8d6f8b5a5c433f28c2b8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene:c0:34:bb:7f:e7:f1:19 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17202021211694468116 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4632491965115421117 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:29:54.501+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: b07e15c0226c8d6f8b5a5c433f28c2b8 TransferTokenId: ShardToken99:9:6f:7f:a9:41:54:e7 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:29:54.501+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000111248 2023-10-21T04:29:54.501+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken99:9:6f:7f:a9:41:54:e7, dropIndexes: false 2023-10-21T04:29:54.501+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7, shardId :0 2023-10-21T04:29:54.501+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b07e15c0226c8d6f8b5a5c433f28c2b8_ShardToken99_9_6f_7f_a9_41_54_e7, shardId :0 2023-10-21T04:29:54.501+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17406544375239214575 10170029854712240877] 2023-10-21T04:29:55.496+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17406544375239214575 2023-10-21T04:29:55.497+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 10170029854712240877 2023-10-21T04:29:55.497+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 10170029854712240877 2023-10-21T04:29:55.497+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17406544375239214575 2023-10-21T04:29:55.497+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17406544375239214575 10170029854712240877], elapsed: 995.1814ms 2023-10-21T04:29:55.497+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:29:55.497+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) removed 2023-10-21T04:29:55.497+05:30 [Info] : All instance closed 2023-10-21T04:29:55.497+05:30 [Info] : Shutdown completed 2023-10-21T04:29:55.497+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:29:55.497+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) removed 2023-10-21T04:29:55.497+05:30 [Info] : All instance closed 2023-10-21T04:29:55.497+05:30 [Info] : All instance destroyed 2023-10-21T04:29:55.513+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:29:55.513+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:29:55.513+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) removed 2023-10-21T04:29:55.513+05:30 [Info] : All instance closed 2023-10-21T04:29:55.513+05:30 [Info] : Shutdown completed 2023-10-21T04:29:55.513+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:29:55.513+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) removed 2023-10-21T04:29:55.513+05:30 [Info] : All instance closed 2023-10-21T04:29:55.513+05:30 [Info] : All instance destroyed 2023-10-21T04:29:55.514+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:29:55.514+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17406544375239214575 10170029854712240877], elapsed(sec): 1.012985366 2023-10-21T04:29:55.518+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:29:55.518+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:29:55.518+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.694µs 2023-10-21T04:29:55.518+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:29:55.518+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:29:55.518+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:29:55.525+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:29:55.525+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:29:55.525+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:29:55.526+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:29:55.526+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 46.557µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:29:55.531+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:29:55.531+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:29:55.531+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 2.030376576s in RebalanceServiceManager::recoverRebalance: 2023-10-21T04:29:55.531+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{18194302e3f10df59bd076e1c4142d0b 7060000 } 2023-10-21T04:29:55.850+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53996 2023-10-21T04:29:55.855+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53998 2023-10-21T04:29:55.859+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54000 2023-10-21T04:29:55.860+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54002 2023-10-21T04:29:55.861+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54004 2023-10-21T04:29:55.879+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54006 2023-10-21T04:29:58.477+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] 2023-10-21T04:29:58.478+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] 2023-10-21T04:29:58.478+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 4 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] 2023-10-21T04:29:58.478+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:29:58.478+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] IsBalanced:false Messages:[]} 2023-10-21T04:29:58.480+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:29:58.480+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:29:59.375+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:38838 2023-10-21T04:29:59.409+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:38838 2023-10-21T04:30:03.088+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 249] aggregate storage info: map[bucket_%2:map[kv:2304192 index:4350220] bucket_1:map[kv:2369728 index:4358328]] 2023-10-21T04:30:09.914+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:10.038+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:10.038+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:10.056+05:30 [Info] switched currmeta from 22 -> 22 force true 2023-10-21T04:30:10.076+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 213] updated cluster rebalancing status to true 2023-10-21T04:30:10.076+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 213] updating rebalancing status to true 2023-10-21T04:30:10.150+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:30:10.408+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:10.408+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:10.410+05:30 [Info] switched currmeta from 22 -> 22 force true 2023-10-21T04:30:10.410+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:10.425+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:10.425+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:10.438+05:30 [Info] switched currmeta from 22 -> 22 force true 2023-10-21T04:30:11.671+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {0cf7c80ee554dff9d153c0588632edfe [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:30:11.674+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 6.06µs 2023-10-21T04:30:11.674+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:30:11.674+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:30:11.674+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:30:11.676+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:30:11.676+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:11.677+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.569938ms 2023-10-21T04:30:11.677+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 7.432µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:11.677+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:30:11.677+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:30:11.677+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:11.677+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/0cf7c80ee554dff9d153c0588632edfe Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0cf7c80ee554dff9d153c0588632edfe]}]} 2023-10-21T04:30:11.679+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:30:11.679+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:30:11.708+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {0cf7c80ee554dff9d153c0588632edfe [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:30:11.710+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:30:11.740+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"0cf7c80ee554dff9d153c0588632edfe","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:30:11.743+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:11.745+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:12.759+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:30:12.773+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:30:12.773+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:30:12.779+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:30:12.779+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:30:12.779+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:30:12.779+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.071098555s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:30:12.779+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:30:12.780+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/0cf7c80ee554dff9d153c0588632edfe Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0cf7c80ee554dff9d153c0588632edfe]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/0cf7c80ee554dff9d153c0588632edfe Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0cf7c80ee554dff9d153c0588632edfe]}]} 2023-10-21T04:30:12.780+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:12.780+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:30:12.784+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:30:12.784+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:12.784+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:30:12.784+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/0cf7c80ee554dff9d153c0588632edfe Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0cf7c80ee554dff9d153c0588632edfe]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/0cf7c80ee554dff9d153c0588632edfe Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:0cf7c80ee554dff9d153c0588632edfe]}]} 2023-10-21T04:30:12.787+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:30:12.788+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:30:12.877+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{7f731cd927c72e28ff24e69497b93571 7f731cd927c72e28ff24e69497b93571 plasma map[] [{21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{6043349427976659583 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{6225851069678382014 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12824340686309573428 [{14217467447424830582 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{2535881715450691825 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593] []}] 1 0 0 0 false plasma 0}]}]} {21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{3345094951175719474 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{6377153279641237139 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 17797267771846799851 [{17205882817792019450 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{10601152754683910907 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {964f96068adef144590fc7ab85b2311c 964f96068adef144590fc7ab85b2311c plasma map[] [{21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{8965923595543532942 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{8732144615635911668 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12824340686309573428 [{12266944006962282785 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{17434304260397056806 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575] []}] 1 0 0 1 false plasma 0}]}]} {21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{15864358084358521096 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{10032435514095102923 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 17797267771846799851 [{3023333489681759512 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{3464125101225762400 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {18194302e3f10df59bd076e1c4142d0b 18194302e3f10df59bd076e1c4142d0b plasma map[] [] [] 0 0 0 false} {8806f5706747af4e635b8da5fcd8aeee 8806f5706747af4e635b8da5fcd8aeee plasma map[] [{4 bucket_%2 _default c1 []} {4 bucket_1 _default c1 []}] [] 0 0 0 false}] map[]} 2023-10-21T04:30:13.008+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:38806 2023-10-21T04:30:13.016+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:30:13.016+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:30:13.016+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:30:13.016+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:30:13.017+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:38806 2023-10-21T04:30:13.054+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9004 2023-10-21T04:30:13.054+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9001 2023-10-21T04:30:13.054+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9002 2023-10-21T04:30:13.054+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9003 2023-10-21T04:30:13.060+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:38806 2023-10-21T04:30:13.067+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:30:13.067+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:30:13.068+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:30:13.068+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:30:13.123+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:30:13.123+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:30:13.123+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:30:13.123+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:30:13.123+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 17797267771846799851:17205882817792019450:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 11276338709617092948:10601152754683910907:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 12824340686309573428:14217467447424830582:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 3381524789655566900:2535881715450691825:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.123+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 17797267771846799851:3023333489681759512:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 3381524789655566900:17434304260397056806:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 11276338709617092948:3464125101225762400:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Moving index 12824340686309573428:12266944006962282785:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:13.124+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17581676 Units 0 2023-10-21T04:30:13.124+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 18185215 Units 0 2023-10-21T04:30:13.124+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 20643 Units 0 Tenants 0 2023-10-21T04:30:13.124+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 624214 Units 0 Tenants 0 2023-10-21T04:30:13.124+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9003 127.0.0.1:9004]] 2023-10-21T04:30:13.124+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:30:13.124+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:30:13.124+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:30:13.124+05:30 [Info] ---------------------------------------- 2023-10-21T04:30:13.124+05:30 [Info] 2023-10-21T04:30:13.124+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:18194302e3f10df59bd076e1c4142d0b, useLiveData:false 2023-10-21T04:30:13.124+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:30:13.124+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:30:13.124+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.124+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:157327869 (150.04M), mem:35904 (35.0625K), overhead:157291965 (150.005M), min:157327869 (150.04M), data:35904 (35.0625K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:157326969 (150.039M), mem:35125 (34.3018K), overhead:157291844 (150.005M), min:157326969 (150.039M), data:35125 (34.3018K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:157327402 (150.039M), mem:35500 (34.668K), overhead:157291902 (150.005M), min:157327402 (150.039M), data:35500 (34.668K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:157327740 (150.039M), mem:35793 (34.9541K), overhead:157291947 (150.005M), min:157327740 (150.039M), data:35793 (34.9541K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.125+05:30 [Info] Index total memory:157327968 (150.04M), mem:35990 (35.1465K), overhead:157291978 (150.005M), min:157327968 (150.04M), data:35990 (35.1465K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.125+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.125+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.125+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:17797267771846799851, instId:3023333489681759512, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:17434304260397056806, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157293232 (150.007M), mem:5916 (5.77734K), overhead:157287316 (150.001M), min:157293232 (150.007M), data:5916 (5.77734K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157327056 (150.039M), mem:35200 (34.375K), overhead:157291856 (150.005M), min:157327056 (150.039M), data:35200 (34.375K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157327287 (150.039M), mem:35400 (34.5703K), overhead:157291887 (150.005M), min:157327287 (150.039M), data:35400 (34.5703K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157328319 (150.04M), mem:36294 (35.4434K), overhead:157292025 (150.005M), min:157328319 (150.04M), data:36294 (35.4434K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157327252 (150.039M), mem:35370 (34.541K), overhead:157291882 (150.005M), min:157327252 (150.039M), data:35370 (34.541K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:157326760 (150.038M), mem:34944 (34.125K), overhead:157291816 (150.005M), min:157326760 (150.038M), data:34944 (34.125K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.126+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.126+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.126+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.126+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:3464125101225762400, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:157293004 (150.006M), mem:5718 (5.58398K), overhead:157287286 (150.001M), min:157293004 (150.006M), data:5718 (5.58398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:12824340686309573428, instId:12266944006962282785, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] 2023-10-21T04:30:13.127+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:8806f5706747af4e635b8da5fcd8aeee, useLiveData:false 2023-10-21T04:30:13.127+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:30:13.127+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.127+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.127+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.127+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.127+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:17797267771846799851, instId:17205882817792019450, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0813 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:10601152754683910907, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:157293697 (150.007M), mem:6318 (6.16992K), overhead:157287379 (150.001M), min:157293697 (150.007M), data:6318 (6.16992K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0080 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.128+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.128+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.128+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.128+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:157327258 (150.039M), mem:35375 (34.5459K), overhead:157291883 (150.005M), min:157327258 (150.039M), data:35375 (34.5459K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:157327980 (150.04M), mem:36000 (35.1562K), overhead:157291980 (150.005M), min:157327980 (150.04M), data:36000 (35.1562K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:157327827 (150.04M), mem:35868 (35.0273K), overhead:157291959 (150.005M), min:157327827 (150.04M), data:35868 (35.0273K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:157328340 (150.04M), mem:36312 (35.4609K), overhead:157292028 (150.005M), min:157328340 (150.04M), data:36312 (35.4609K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0066 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.129+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.129+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.129+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.129+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:12824340686309573428, instId:14217467447424830582, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0789 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:2535881715450691825, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:157293697 (150.007M), mem:6318 (6.16992K), overhead:157287379 (150.001M), min:157293697 (150.007M), data:6318 (6.16992K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:157328090 (150.04M), mem:36096 (35.25K), overhead:157291994 (150.005M), min:157328090 (150.04M), data:36096 (35.25K) cpu:0.0142 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0136 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:157328309 (150.04M), mem:36285 (35.4346K), overhead:157292024 (150.005M), min:157328309 (150.04M), data:36285 (35.4346K) cpu:0.0136 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.130+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.130+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.130+05:30 [Info] Index total memory:157327695 (150.039M), mem:35754 (34.916K), overhead:157291941 (150.005M), min:157327695 (150.039M), data:35754 (34.916K) cpu:0.0130 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.130+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.131+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.131+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.131+05:30 [Info] Index total memory:157328797 (150.04M), mem:36708 (35.8477K), overhead:157292089 (150.005M), min:157328797 (150.04M), data:36708 (35.8477K) cpu:0.0154 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.131+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.131+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.131+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.131+05:30 [Info] Index total memory:157327911 (150.04M), mem:35941 (35.0986K), overhead:157291970 (150.005M), min:157327911 (150.04M), data:35941 (35.0986K) cpu:0.0142 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.131+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.131+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.131+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.131+05:30 [Info] Index total memory:157328160 (150.04M), mem:36156 (35.3086K), overhead:157292004 (150.005M), min:157328160 (150.04M), data:36156 (35.3086K) cpu:0.0146 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.131+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.131+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:13.131+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:13.131+05:30 [Info] Index total memory:157327148 (150.039M), mem:35280 (34.4531K), overhead:157291868 (150.005M), min:157327148 (150.039M), data:35280 (34.4531K) cpu:0.0124 io:0 (0) scan:0 drain:0 2023-10-21T04:30:13.131+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:13.131+05:30 [Info] **************************************** 2023-10-21T04:30:13.132+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 20643 Units 0 Tenants 0 2023-10-21T04:30:13.132+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 624214 Units 0 Tenants 0 2023-10-21T04:30:13.132+05:30 [Info] Generating Shard Transfer Token (ShardToken4d:21:b:32:a5:fb:1e:4a) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4a:d0:fd:77:44:72:a5:8a Shards: [16787445029868804812 2925063509317026816] Destination: Region: InstId: 9169561265200546106 RealInstId: 10032435514095102923 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 15298811879390062834 RealInstId: 15864358084358521096 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3023333489681759512 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3464125101225762400 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:13.133+05:30 [Info] Generating Shard Transfer Token (ShardTokenab:c1:ca:ac:c7:98:c3:16) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken5:f4:88:1:79:ef:31:3f Shards: [17406544375239214575 10170029854712240877] Destination: Region: InstId: 12997526226294542081 RealInstId: 8965923595543532942 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 14537831791783005540 RealInstId: 8732144615635911668 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12266944006962282785 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17434304260397056806 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:13.134+05:30 [Info] Generating Shard Transfer Token (ShardToken5:f4:88:1:79:ef:31:3f) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenab:c1:ca:ac:c7:98:c3:16 Shards: [7434639758576776593 1249143295319901191] Destination: Region: InstId: 11147967906305107998 RealInstId: 6225851069678382014 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6225851069678382014 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 14217467447424830582 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14217467447424830582 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 13601468463969824886 RealInstId: 6043349427976659583 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6043349427976659583 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2535881715450691825 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2535881715450691825 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:13.136+05:30 [Info] Generating Shard Transfer Token (ShardToken4a:d0:fd:77:44:72:a5:8a) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Shards: [4718895791748289908 15531034489713578274] Destination: Region: InstId: 17205882817792019450 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17205882817792019450 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11598697160549978176 RealInstId: 3345094951175719474 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3345094951175719474 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11110094464605480874 RealInstId: 6377153279641237139 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6377153279641237139 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10601152754683910907 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10601152754683910907 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:13.137+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:30:13.137+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:30:13.141+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken4d:21:b:32:a5:fb:1e:4a 2023-10-21T04:30:13.193+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenab:c1:ca:ac:c7:98:c3:16 2023-10-21T04:30:13.238+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken5:f4:88:1:79:ef:31:3f 2023-10-21T04:30:13.251+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken4a:d0:fd:77:44:72:a5:8a 2023-10-21T04:30:13.251+05:30 [Info] ShardRebalancer::observeRebalance &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:30:13.301+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4a:d0:fd:77:44:72:a5:8a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11598697160549978176 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11110094464605480874 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.308+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken5:f4:88:1:79:ef:31:3f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenab:c1:ca:ac:c7:98:c3:16 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11147967906305107998 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13601468463969824886 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.309+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"0cf7c80ee554dff9d153c0588632edfe","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:30:13.309+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:30:13.311+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4d:21:b:32:a5:fb:1e:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4a:d0:fd:77:44:72:a5:8a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9169561265200546106 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15298811879390062834 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.311+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:30:13.311+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:30:13.311+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:30:13.311+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 136.704µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:13.407+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenab:c1:ca:ac:c7:98:c3:16 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken5:f4:88:1:79:ef:31:3f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12997526226294542081 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14537831791783005540 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.408+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenab:c1:ca:ac:c7:98:c3:16 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken5:f4:88:1:79:ef:31:3f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12997526226294542081 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14537831791783005540 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.409+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:30:13.409+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:30:13.409+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:30:13.409+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.401µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:13.446+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4a:d0:fd:77:44:72:a5:8a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11598697160549978176 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11110094464605480874 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.447+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken5:f4:88:1:79:ef:31:3f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenab:c1:ca:ac:c7:98:c3:16 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11147967906305107998 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13601468463969824886 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.447+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4d:21:b:32:a5:fb:1e:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4a:d0:fd:77:44:72:a5:8a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9169561265200546106 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15298811879390062834 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.465+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:13.466+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenab:c1:ca:ac:c7:98:c3:16 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken5:f4:88:1:79:ef:31:3f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12997526226294542081 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14537831791783005540 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.471+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:13.500+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4a:d0:fd:77:44:72:a5:8a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11598697160549978176 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11110094464605480874 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.507+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:13.526+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken5:f4:88:1:79:ef:31:3f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenab:c1:ca:ac:c7:98:c3:16 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11147967906305107998 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13601468463969824886 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.542+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:13.542+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:30:13.557+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:30:13.557+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 2 2023-10-21T04:30:13.638+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[7f731cd927c72e28ff24e69497b93571:ShardToken5:f4:88:1:79:ef:31:3f 964f96068adef144590fc7ab85b2311c:ShardTokenab:c1:ca:ac:c7:98:c3:16], currActiveTransfers: map[7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:2] 2023-10-21T04:30:13.638+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"0cf7c80ee554dff9d153c0588632edfe","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:30:13.638+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:30:13.638+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:30:13.638+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:30:13.638+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:30:13.638+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:30:13.639+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4d:21:b:32:a5:fb:1e:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4a:d0:fd:77:44:72:a5:8a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9169561265200546106 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15298811879390062834 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.640+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4a:d0:fd:77:44:72:a5:8a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11598697160549978176 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11110094464605480874 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.642+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 17.65µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:13.642+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenab:c1:ca:ac:c7:98:c3:16 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken5:f4:88:1:79:ef:31:3f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12997526226294542081 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14537831791783005540 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.643+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken5:f4:88:1:79:ef:31:3f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenab:c1:ca:ac:c7:98:c3:16 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11147967906305107998 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13601468463969824886 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.754+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4d:21:b:32:a5:fb:1e:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4a:d0:fd:77:44:72:a5:8a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9169561265200546106 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15298811879390062834 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.754+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:30:13.754+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:13.754+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:13.754+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:30:13.754+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.058µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:13.755+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2925063509317026816:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/2925063509317026816/shards/shard2925063509317026816 16787445029868804812:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/16787445029868804812/shards/shard16787445029868804812] 0cf7c80ee554dff9d153c0588632edfe ShardToken4d:21:b:32:a5:fb:1e:4a /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc006a1ac60 0xc006a1acc0 map[] 0xc0071255c0 0xc00714fc20 false} 2023-10-21T04:30:13.755+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/2925063509317026816/shards/shard2925063509317026816 2023-10-21T04:30:13.756+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/16787445029868804812/shards/shard16787445029868804812 2023-10-21T04:30:13.762+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/16787445029868804812/shards/shard16787445029868804812/data: super block snapshot head offset :0 tail offset:761856 sbgen :136 2023-10-21T04:30:13.763+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/16787445029868804812/shards/shard16787445029868804812/data/recovery: super block snapshot head offset :0 tail offset:716800 sbgen :136 2023-10-21T04:30:13.766+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/2925063509317026816/shards/shard2925063509317026816/data: super block snapshot head offset :0 tail offset:651264 sbgen :126 2023-10-21T04:30:13.766+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/2925063509317026816/shards/shard2925063509317026816/data/recovery: super block snapshot head offset :0 tail offset:593920 sbgen :126 2023-10-21T04:30:13.771+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/16787445029868804812/shards/shard16787445029868804812: metadata saved successfully 2023-10-21T04:30:13.772+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:30:13.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:30:13.772+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 93619688 totalBytes: 1497915 numShards: 2 duration: 16.776655ms 2023-10-21T04:30:13.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/16787445029868804812/shards/shard16787445029868804812, newshardId 16787445029868804812, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, shardsize 1497915, duration 15.602626ms 2023-10-21T04:30:13.772+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 16787445029868804812, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, err: , elapsed(sec): 0.017119044 2023-10-21T04:30:13.772+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 16787445029868804812 bytesWritten: 1497915, totalBytes: 1497915, transferRate: 9.9861e+07 2023-10-21T04:30:13.773+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/2925063509317026816/shards/shard2925063509317026816: metadata saved successfully 2023-10-21T04:30:13.774+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:30:13.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:30:13.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a/2925063509317026816/shards/shard2925063509317026816, newshardId 2925063509317026816, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, shardsize 1264284, duration 18.935617ms 2023-10-21T04:30:13.774+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2925063509317026816, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, err: , elapsed(sec): 0.019364345 2023-10-21T04:30:13.774+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2925063509317026816 bytesWritten: 1264284, totalBytes: 1264284, transferRate: 7.0238e+07 2023-10-21T04:30:13.774+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 0cf7c80ee554dff9d153c0588632edfe, transferId: ShardToken4d:21:b:32:a5:fb:1e:4a 2023-10-21T04:30:13.774+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 0cf7c80ee554dff9d153c0588632edfe, transferId: ShardToken4d:21:b:32:a5:fb:1e:4a, elapsed(sec): 0.000200918 2023-10-21T04:30:13.774+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2925063509317026816: 16787445029868804812:], shardPaths: map[2925063509317026816:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 16787445029868804812:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.019624934 2023-10-21T04:30:13.774+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [16787445029868804812 2925063509317026816], ttid: ShardToken4d:21:b:32:a5:fb:1e:4a, elapsed(sec): 0.020166099 2023-10-21T04:30:13.778+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:13.825+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4d:21:b:32:a5:fb:1e:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4a:d0:fd:77:44:72:a5:8a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9169561265200546106 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15298811879390062834 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:13.825+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:30:13.825+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken4d:21:b:32:a5:fb:1e:4a 2023-10-21T04:30:13.825+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2925063509317026816:true 16787445029868804812:true] 2023-10-21T04:30:13.825+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.571µs 2023-10-21T04:30:13.825+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [16787445029868804812 2925063509317026816], isLockedForRecovery: true 2023-10-21T04:30:13.825+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [16787445029868804812 2925063509317026816], errMap: map[2925063509317026816: 16787445029868804812:], elapsed: 8.633µs 2023-10-21T04:30:13.834+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:30:13.835+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:30:13.947+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10032435514095102923,realInstId: 0, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:13.947+05:30 [Info] Indexer::handleRecoverIndex InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:30:13.948+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 532.019µs 2023-10-21T04:30:13.948+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 576.576µs 2023-10-21T04:30:13.948+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:30:13.948+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 3 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:30:13.949+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:30:13.948+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10032435514095102923, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:30:13.949+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 114.406776ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:13.954+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:30:13.954+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner initialized 2023-10-21T04:30:13.954+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:13.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:13.955+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCleaner initialized for recovery 2023-10-21T04:30:13.955+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:13.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery 2023-10-21T04:30:13.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:13.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true] 2023-10-21T04:30:13.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[593920] Data log Head [0] Tail [651264] 2023-10-21T04:30:13.955+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [593920] 2023-10-21T04:30:13.956+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.957+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:30:13.957+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:30:13.957+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.957+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:30:13.957+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.958+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:30:13.958+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.959+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:30:13.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.959+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:30:13.959+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.959+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:30:13.960+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:30:13.960+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:30:13.963+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.963+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:30:13.965+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner initialized 2023-10-21T04:30:13.965+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:13.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:13.975+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCleaner initialized for recovery 2023-10-21T04:30:13.975+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:13.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery 2023-10-21T04:30:13.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:13.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true] 2023-10-21T04:30:13.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[716800] Data log Head [0] Tail [761856] 2023-10-21T04:30:13.975+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [716800] 2023-10-21T04:30:13.979+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.979+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:13.979+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.979+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:13.979+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.979+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:13.979+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.979+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:13.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.993+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:30:13.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:30:13.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:30:13.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:30:13.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:30:13.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:30:13.995+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.995+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:30:13.995+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.995+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:30:13.995+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.995+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:30:13.996+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:13.996+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:30:14.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.001+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:14.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.001+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:14.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.002+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:14.002+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.002+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:14.004+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : replayOffset: 643072 (RP:643072) (CP:360448) 2023-10-21T04:30:14.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [643072] took [48.722952ms] 2023-10-21T04:30:14.004+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:30:14.061+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.062+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053288, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:30:14.062+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.062+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1056202, resident 1, memSz 66746, memSzIndex 1058, numRecords 1000 2023-10-21T04:30:14.062+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.063+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.063+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053288, resident 1, memSz 9501, memSzIndex 76, numRecords 127 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053288, resident 1, memSz 10500, memSzIndex 77, numRecords 136 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1053364, resident 1, memSz 9269, memSzIndex 92, numRecords 125 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.064+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 37414193 controller mem 5346 2023-10-21T04:30:14.064+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [651264] replayOffset [643072] 2023-10-21T04:30:14.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [110.012815ms] 2023-10-21T04:30:14.065+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.066+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.066+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.066+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.066+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.067+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.067+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.067+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.068+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.068+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.068+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.068+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.068+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.021+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken4a:d0:fd:77:44:72:a5:8a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11598697160549978176 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11110094464605480874 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:14.033+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:14.083+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.099+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:14.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true]. Built [17] plasmas, took [144.814317ms] 2023-10-21T04:30:14.103+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:14.104+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 24094 recovery UsedSpace 8236 2023-10-21T04:30:14.103+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:1 8806f5706747af4e635b8da5fcd8aeee:1 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:30:14.103+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken4d:21:b:32:a5:fb:1e:4a 2023-10-21T04:30:14.103+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.106+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:14.106+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.106+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:14.106+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.107+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:14.107+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.107+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:14.107+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.107+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:14.107+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.107+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:14.107+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:14.107+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:14.108+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : replayOffset: 753664 (RP:753664) (CP:450560) 2023-10-21T04:30:14.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [753664] took [133.563612ms] 2023-10-21T04:30:14.108+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057426, resident 1, memSz 12491, memSzIndex 0, numRecords 125 2023-10-21T04:30:14.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057426, resident 1, memSz 10600, memSzIndex 0, numRecords 106 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057426, resident 1, memSz 12489, memSzIndex 0, numRecords 125 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057426, resident 1, memSz 12281, memSzIndex 0, numRecords 123 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057276, resident 1, memSz 13565, memSzIndex 0, numRecords 136 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057276, resident 1, memSz 12657, memSzIndex 0, numRecords 127 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1055176, resident 1, memSz 820, memSzIndex 0, numRecords 7 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057276, resident 1, memSz 13546, memSzIndex 0, numRecords 136 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1057276, resident 1, memSz 12169, memSzIndex 0, numRecords 122 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1055988, resident 1, memSz 70520, memSzIndex 401, numRecords 1000 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:14.112+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5378 2023-10-21T04:30:14.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [761856] replayOffset [753664] 2023-10-21T04:30:14.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [138.123552ms] 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.113+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:14.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true]. Built [18] plasmas, took [139.039886ms] 2023-10-21T04:30:14.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:14.114+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 27306 recovery UsedSpace 8236 panic: TestCode::TestActionAtTag - Inducing artificial panic at tag: 4 as wished goroutine 1033 [running]: github.com/couchbase/indexing/secondary/testcode.TestActionAtTag(0x180aba0?, 0x4) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/testcode/test_action_ci.go:45 +0x5e5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).createDeferredIndex(0xc00683f680, 0xc0073a5da0, {0xc0073d1014, 0x20}, 0xc0073a1000, 0x1, 0x0) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1699 +0x445 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).startShardRecovery(0xc00683f680, {0xc0073d1014, 0x20}, 0xc0073a1000) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1853 +0x1015 created by github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferTokenAsDest /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1431 +0x28d 2023-10-21T04:30:15.758+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=18194302e3f10df59bd076e1c4142d0b -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:30:15.759+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:30:15.759+05:30 [Info] Setting ipv6=false 2023-10-21T04:30:15.773+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:30:15.799+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:30:15.823+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:30:15.823+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:30:15.823+05:30 [Info] Setting log level to Info 2023-10-21T04:30:15.823+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:30:15.823+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:30:15.824+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:30:15.824+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:30:15.842+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:30:16 audit: created new audit service 2023-10-21T04:30:16.020+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:30:16.020+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:30:16.020+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:30:16.021+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:30:16.021+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:30:16.021+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:30:16.049+05:30 [Info] security context initialized 2023-10-21T04:30:16.049+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:30:16.049+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:30:16.049+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:30:16.050+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:30:16.050+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:30:16.055+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:30:16.056+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:30:16.056+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:30:16.056+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:30:16.060+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:30:16.065+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:30:16.065+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:30:16.066+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:16.068+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:30:16.076+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:30:16.076+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:30:16.076+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:30:16.077+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:30:16.102+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:30:16.102+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:30:16.103+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:30:16.106+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:30:16.106+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:30:16.108+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:30:16.108+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:30:16.108+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:30:16.109+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:30:16.110+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:30:16.111+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 288 us 2023-10-21T04:30:16.112+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:30:16.117+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:30:16.118+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:30:16.131+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:30:16.131+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:30:16.131+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:30:16.132+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:30:16.132+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:30:16.132+05:30 [Info] Indexer Id 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:16.133+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:30:16.133+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:30:16.136+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:30:16.138+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:30:16.140+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 207] updating cluster rev epoch 0 -> 1 2023-10-21T04:30:16.141+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 207] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:30:16.147+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 148] bucketNames has been updated (was map[], now map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43]) 2023-10-21T04:30:16.147+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 149] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:30:16.148+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 150] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:30:16.152+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 148] updated cluster rebalancing status to false 2023-10-21T04:30:16.152+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:30:16.152+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:30:16.153+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:30:16.158+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:30:16.158+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:30:16.158+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:30:16.158+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:30:16.158+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning true RebalanceToken &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:16.158+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:30:16.159+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:30:16.159+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:30:16.160+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:30:16.174+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:30:16.174+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:30:16.176+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:30:16.178+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:30:16.178+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:30:16.181+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 , elapsed: 21.153643ms 2023-10-21T04:30:16.181+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:16.182+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:30:16.182+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:30:16.182+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:30:16.182+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager true &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:16.182+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:30:16.182+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:30:16.182+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:16.185+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:30:16.185+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:30:16.185+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:30:16.185+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:16.186+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:30:16.186+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:30:16.186+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:30:16.186+05:30 [Warn] Indexer::validateIndexInstMap State INDEX_STATE_CREATED Stream NIL_STREAM Deferred true Found. Updating State to Ready InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:30:16.187+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:16.187+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:16.194+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:30:16.195+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:30:16.196+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.197+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:30:16.199+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:30:16.199+05:30 [Info] : Recover shard shard16787445029868804812 from metadata completed. 2023-10-21T04:30:16.199+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:30:16.199+05:30 [Info] : Recover shard shard2925063509317026816 from metadata completed. 2023-10-21T04:30:16.199+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:30:16.200+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.431442ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:16.201+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:30:16.201+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:30:16.201+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:30:16.202+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner initialized 2023-10-21T04:30:16.202+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:16.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.203+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCleaner initialized for recovery 2023-10-21T04:30:16.203+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:16.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery 2023-10-21T04:30:16.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true] 2023-10-21T04:30:16.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[593920] Data log Head [0] Tail [651264] 2023-10-21T04:30:16.204+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [593920] 2023-10-21T04:30:16.206+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.206+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:30:16.206+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:30:16.206+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.207+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:30:16.207+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.207+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:30:16.207+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.207+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:30:16.207+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.208+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:30:16.208+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.209+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:30:16.209+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.209+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:30:16.209+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.210+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:30:16.211+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:30:16.211+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:false Messages:[]} 2023-10-21T04:30:16.214+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner initialized 2023-10-21T04:30:16.216+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:16.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.214+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:30:16.214+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:30:16.217+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.217+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:30:16.218+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.219+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.219+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCleaner initialized for recovery 2023-10-21T04:30:16.219+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:16.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery 2023-10-21T04:30:16.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true] 2023-10-21T04:30:16.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[716800] Data log Head [0] Tail [761856] 2023-10-21T04:30:16.219+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [716800] 2023-10-21T04:30:16.219+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.220+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.220+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.221+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.221+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:30:16.221+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.221+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:30:16.221+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.221+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.221+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.222+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.222+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.222+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.222+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.222+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.222+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.223+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.224+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : replayOffset: 643072 (RP:643072) (CP:360448) 2023-10-21T04:30:16.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [643072] took [19.602875ms] 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9269, memSzIndex 92, numRecords 125 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 10500, memSzIndex 77, numRecords 136 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9501, memSzIndex 76, numRecords 127 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1056202, resident 1, memSz 66746, memSzIndex 1058, numRecords 1000 2023-10-21T04:30:16.224+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 28945092 controller mem 3920 2023-10-21T04:30:16.225+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [651264] replayOffset [643072] 2023-10-21T04:30:16.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [20.757877ms] 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.225+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.226+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:30:16.226+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.226+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:30:16.226+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.226+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:30:16.226+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.226+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:30:16.226+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.226+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:30:16.226+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.226+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:30:16.227+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.227+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:30:16.228+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.228+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:30:16.228+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.229+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.229+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.230+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.230+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.230+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.230+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true]. Built [17] plasmas, took [25.937877ms] 2023-10-21T04:30:16.230+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.230+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 24094 recovery UsedSpace 8236 2023-10-21T04:30:16.231+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.231+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.231+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.231+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.232+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.232+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.232+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.232+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.232+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.233+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.233+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.233+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.233+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.234+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : replayOffset: 753664 (RP:753664) (CP:450560) 2023-10-21T04:30:16.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [753664] took [15.203985ms] 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12489, memSzIndex 0, numRecords 125 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13565, memSzIndex 0, numRecords 136 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12491, memSzIndex 0, numRecords 125 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12281, memSzIndex 0, numRecords 123 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 10600, memSzIndex 0, numRecords 106 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1055988, resident 1, memSz 70520, memSzIndex 401, numRecords 1000 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12169, memSzIndex 0, numRecords 122 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12657, memSzIndex 0, numRecords 127 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:16.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.236+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1055176, resident 1, memSz 820, memSzIndex 0, numRecords 7 2023-10-21T04:30:16.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.236+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.236+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13546, memSzIndex 0, numRecords 136 2023-10-21T04:30:16.236+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:30:16.236+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754261 controller mem 5330 2023-10-21T04:30:16.236+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [761856] replayOffset [753664] 2023-10-21T04:30:16.236+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [16.755843ms] 2023-10-21T04:30:16.236+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.236+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.236+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:30:16.237+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.237+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.237+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.237+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:16.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true]. Built [18] plasmas, took [19.296372ms] 2023-10-21T04:30:16.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.239+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 27306 recovery UsedSpace 8236 2023-10-21T04:30:16.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.244+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:30:16.244+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:30:16.244+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:16.244+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.244+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:16.244+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:16.245+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:30:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.251+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:16.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:30:16.251+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.251+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.251+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.251+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:16.251+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.251+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:16.337+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:16.337+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:30:16.337+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.337+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.337+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.337+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.342+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:16.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:30:16.342+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:30:16.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:30:16.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:30:16.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:30:16.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:30:16.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.353+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.353+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.357+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.357+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.361+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.361+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:16.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:30:16.361+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.361+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.374+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.374+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:16.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:30:16.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.374+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Warmup took 177.013037ms 2023-10-21T04:30:16.374+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 6 WriterThreads 1 2023-10-21T04:30:16.374+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index Index: 10032435514095102923 Partition: 6 2023-10-21T04:30:16.374+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.375+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.375+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 28248 recovery UsedSpace 8236 2023-10-21T04:30:16.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.375+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.375+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 24824 recovery UsedSpace 8236 2023-10-21T04:30:16.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.385+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:30:16.385+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.385+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.386+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:30:16.386+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:16.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.386+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:16.386+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:16.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.394+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.394+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:16.394+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:30:16.394+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.394+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.394+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:16.395+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.395+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:16.395+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:16.395+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:30:16.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:16.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:30:16.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:30:16.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:30:16.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:30:16.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:30:16.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:30:16.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.424+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.425+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.428+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.428+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.433+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.433+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:16.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:30:16.433+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.433+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.466+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.466+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:16.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:30:16.466+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Warmup took 91.631533ms 2023-10-21T04:30:16.467+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 7 WriterThreads 1 2023-10-21T04:30:16.467+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index Index: 10032435514095102923 Partition: 7 2023-10-21T04:30:16.467+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 25600 recovery UsedSpace 8236 2023-10-21T04:30:16.468+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.468+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.468+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.468+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 28248 recovery UsedSpace 8236 2023-10-21T04:30:16.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.488+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:30:16.488+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.489+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.491+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:30:16.491+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:30:16.491+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:16.491+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.491+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:16.491+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:16.491+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.491+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.497+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:16.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:30:16.497+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.497+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:16.498+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.498+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:16.498+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:16.498+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:30:16.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.506+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:16.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:30:16.507+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:30:16.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:30:16.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:30:16.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:30:16.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.520+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:30:16.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.525+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.525+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.530+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.530+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.539+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.540+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.540+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:16.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:30:16.540+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.540+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.541+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.541+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:16.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:30:16.541+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Warmup took 73.833939ms 2023-10-21T04:30:16.541+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 8 WriterThreads 1 2023-10-21T04:30:16.541+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index Index: 10032435514095102923 Partition: 8 2023-10-21T04:30:16.541+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.542+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.542+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 25600 recovery UsedSpace 8236 2023-10-21T04:30:16.542+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.542+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.542+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.542+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 29257 recovery UsedSpace 8236 2023-10-21T04:30:16.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.551+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:30:16.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.552+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:30:16.552+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:30:16.552+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:16.552+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.552+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:16.552+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:16.552+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.552+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.558+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:16.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:16.559+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.559+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:16.559+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:16.559+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.559+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.564+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:16.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:30:16.564+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:30:16.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:30:16.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:30:16.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:30:16.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:30:16.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.573+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.573+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.580+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.580+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.581+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.581+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:16.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:30:16.581+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.581+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.584+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.585+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:16.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:30:16.585+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.585+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Warmup took 43.200258ms 2023-10-21T04:30:16.585+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.585+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 1 WriterThreads 1 2023-10-21T04:30:16.585+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index Index: 10032435514095102923 Partition: 1 2023-10-21T04:30:16.585+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.585+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.585+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.586+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.586+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.586+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 26425 recovery UsedSpace 8236 2023-10-21T04:30:16.586+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 29257 recovery UsedSpace 8236 2023-10-21T04:30:16.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.593+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:30:16.593+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.593+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.594+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:30:16.594+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:30:16.595+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:16.595+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.595+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:16.595+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:16.595+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:16.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:16.605+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.605+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:16.605+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:16.605+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.605+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.618+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:30:16.618+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:30:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:30:16.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:30:16.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:30:16.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:30:16.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.637+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.637+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.644+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.644+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.651+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.651+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:16.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:30:16.651+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.651+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.657+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.657+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:16.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:30:16.657+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.657+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Warmup took 72.311376ms 2023-10-21T04:30:16.658+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 2 WriterThreads 1 2023-10-21T04:30:16.658+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index Index: 10032435514095102923 Partition: 2 2023-10-21T04:30:16.658+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.659+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 26425 recovery UsedSpace 8236 2023-10-21T04:30:16.658+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.659+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.659+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 29257 recovery UsedSpace 8236 2023-10-21T04:30:16.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.666+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:30:16.666+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.667+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:30:16.667+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:30:16.667+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:16.667+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.667+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:16.667+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:16.667+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.667+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.675+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:16.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:30:16.675+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:16.676+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.676+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:16.676+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:16.676+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:30:16.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.682+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:16.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:30:16.682+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:30:16.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:30:16.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:30:16.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:30:16.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:30:16.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.691+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.691+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.695+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.695+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.700+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.700+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:16.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:30:16.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.701+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.701+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:16.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:30:16.701+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.701+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.701+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.701+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Warmup took 43.048868ms 2023-10-21T04:30:16.701+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 3 WriterThreads 1 2023-10-21T04:30:16.701+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index Index: 10032435514095102923 Partition: 3 2023-10-21T04:30:16.701+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.702+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.702+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 27306 recovery UsedSpace 8236 2023-10-21T04:30:16.702+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.702+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.702+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 30340 recovery UsedSpace 8236 2023-10-21T04:30:16.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.707+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:30:16.707+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.708+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:30:16.708+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:30:16.708+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.708+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.708+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:16.708+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:16.708+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.708+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.712+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:16.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:30:16.713+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.713+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.713+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.713+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:16.713+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.713+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:16.713+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:16.713+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:30:16.714+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.714+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.714+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.714+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.718+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:16.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:30:16.718+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:30:16.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:30:16.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:30:16.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:30:16.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:30:16.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.726+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.727+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.730+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.730+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.736+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.736+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:16.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:30:16.736+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.737+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.743+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.743+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:16.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:30:16.743+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Warmup took 41.075955ms 2023-10-21T04:30:16.743+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.743+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.743+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 4 WriterThreads 1 2023-10-21T04:30:16.743+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index Index: 10032435514095102923 Partition: 4 2023-10-21T04:30:16.743+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:30:16.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.744+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.744+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 27306 recovery UsedSpace 8236 2023-10-21T04:30:16.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.744+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:30:16.744+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 30340 recovery UsedSpace 8236 2023-10-21T04:30:16.753+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:30:16.757+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 55] will report aggregate recorder stats every 5m0s 2023-10-21T04:30:16.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.765+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:30:16.765+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.765+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.766+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:30:16.766+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:30:16.766+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:16.766+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.766+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:16.766+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:16.766+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.773+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:16.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:30:16.773+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:16.773+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:16.773+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:16.773+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:16.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:16.774+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.774+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:16.774+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:16.774+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:30:16.774+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:16.774+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:16.774+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:16.774+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:16.783+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:16.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:16.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:30:16.783+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:16.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:30:16.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:30:16.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:30:16.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:30:16.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:30:16.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:30:16.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:30:16.796+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.796+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:16.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:16.801+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:16.801+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:30:16.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:16.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:30:16.803+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.803+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:16.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:30:16.803+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.804+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:16.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:16.807+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.807+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:16.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:30:16.807+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Warmup took 63.694398ms 2023-10-21T04:30:16.807+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 5 WriterThreads 1 2023-10-21T04:30:16.807+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index Index: 10032435514095102923 Partition: 5 2023-10-21T04:30:16.807+05:30 [Warn] Indexer::updateTopologyOnShardIdChange. Persisted shardIds: map[1:[] 2:[] 3:[] 4:[] 5:[] 6:[] 7:[] 8:[]] are different from currShardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] for index inst: 10032435514095102923. Updating topology with new shardIds 2023-10-21T04:30:16.807+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005b89920 2023-10-21T04:30:16.808+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10032435514095102923 2023-10-21T04:30:16.808+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:30:16.808+05:30 [Info] StorageMgr::openSnapshot IndexInst:10032435514095102923 Partition:4 No Snapshot Found. 2023-10-21T04:30:16.808+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Adding Nil Snapshot. 2023-10-21T04:30:16.808+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00010e8a0 ScanCoordinator {10032435514095102923 {16337157693634799600 idx_partitioned_defer plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 false [`balance`] N1QL KEY [false] true true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 0 1 0 0xc005b66690 [] 1 0 false plasma 0} map[1:{{1 1 [:9123] []} 0xc005a1c210} 2:{{2 1 [:9123] []} 0xc005a1d7f0} 3:{{3 1 [:9123] []} 0xc005a1dbe0} 4:{{4 1 [:9123] []} 0xc0058e1990} 5:{{5 1 [:9123] []} 0xc005b091e0} 6:{{6 1 [:9123] []} 0xc004dba970} 7:{{7 1 [:9123] []} 0xc0058e0c50} 8:{{8 1 [:9123] []} 0xc004f4f7a0}] 0xc004d37900 0xc005b89aa0} 2023-10-21T04:30:16.808+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:16.808+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:16.810+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:30:16.810+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:16.810+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:30:16.812+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:30:16.814+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:30:16.814+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:30:16.816+05:30 [Info] RebalanceServiceManager::recoverRebalance: Init Pending Cleanup 2023-10-21T04:30:16.816+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:30:16.816+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:30:16.817+05:30 [Info] Setting log level to Info 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:30:16.817+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : all deamons started 2023-10-21T04:30:16.817+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : all deamons started 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:30:16.817+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:30:16.818+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:30:16.818+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:30:16.818+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:30:16.818+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:30:16.818+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:30:16.818+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : all deamons started 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : all deamons started 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:30:16.818+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:30:16.818+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:30:16.824+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.014019ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:16.825+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:30:16.825+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:30:16.825+05:30 [Info] RebalanceServiceManager::doRecoverRebalance Found Global Rebalance Token &{18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:30:16.825+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:30:16.826+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:30:16.826+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:30:16.826+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:30:16.826+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:30:16.826+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:30:16.833+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:30:16.834+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:30:16.834+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:30:16.839+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {18194302e3f10df59bd076e1c4142d0b 0cf7c80ee554dff9d153c0588632edfe Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:30:16.904+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:30:16.923+05:30 [Info] security context initialized 2023-10-21T04:30:16.923+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:30:16.940+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:30:16.976+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:30:16.977+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:16.978+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:16.981+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:30:16.995+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:30:16.996+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:30:16.996+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:30:16.997+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:30:16.999+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:30:16.999+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:30:17.020+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:30:17.021+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:30:17.027+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:30:17.049+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:30:17.049+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:30:17.055+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54548 2023-10-21T04:30:17.056+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:30:17.068+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 9.890444ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:17.068+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:30:17.068+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:30:17.069+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 33.453µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:17.074+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:30:17.093+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:30:17.095+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:30:17.095+05:30 [Info] initialized currmeta 24 force true 2023-10-21T04:30:17.100+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:17.100+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:30:17.100+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:30:17.101+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:30:17.102+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:37900" doAuth() ... 2023-10-21T04:30:17.103+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:37900" doReceive() ... 2023-10-21T04:30:17.108+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:17.108+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:30:17.108+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:30:17.109+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:30:17.142+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:30:17.142+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:17.142+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0] 15864358084358521096:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:17.142+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:17.142+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:17.142+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[7 8 1 2 3 4 5 6] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:17.143+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:17.143+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:30:17.143+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:30:17.145+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:30:17.151+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:17.152+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:30:17.152+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:30:17.153+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:30:17.155+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:30:17.156+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:30:17.156+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:30:17.156+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:30:17.156+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:30:17.156+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:30:17.156+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:30:17.157+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:30:17.157+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:30:17.158+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken4a:d0:fd:77:44:72:a5:8a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 11598697160549978176 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11110094464605480874 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:17.158+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken4d:21:b:32:a5:fb:1e:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken4a:d0:fd:77:44:72:a5:8a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9169561265200546106 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15298811879390062834 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:17.158+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 0cf7c80ee554dff9d153c0588632edfe TransferTokenId: ShardToken4d:21:b:32:a5:fb:1e:4a Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:30:17.159+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.00074072 2023-10-21T04:30:17.160+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken4d:21:b:32:a5:fb:1e:4a, dropIndexes: true 2023-10-21T04:30:17.161+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a, shardId :0 2023-10-21T04:30:17.161+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardToken4d_21_b_32_a5_fb_1e_4a, shardId :0 2023-10-21T04:30:17.163+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16337157693634799600 instance id 9169561265200546106 real instance id 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:17.163+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16337157693634799600 instance 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:17.163+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16337157693634799600 instance id 10032435514095102923 2023-10-21T04:30:17.163+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16337157693634799600 2023-10-21T04:30:17.167+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 8.59228ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:17.167+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 222] aggregate storage info: map[bucket_%2:map[kv:2304192 index:4350220] bucket_1:map[kv:2369728 index:4358328]] 2023-10-21T04:30:17.179+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10032435514095102923 &{1} 2023-10-21T04:30:17.180+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10032435514095102923 2023-10-21T04:30:17.180+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16337157693634799600, instId: 10032435514095102923 2023-10-21T04:30:17.180+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:30:17.180+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:30:17.180+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10032435514095102923] 2023-10-21T04:30:17.180+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.180+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.180+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.180+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:17.180+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:17.181+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:17.181+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:17.181+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:17.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.181+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:17.181+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:17.181+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:17.181+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:17.181+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:17.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:30:17.181+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:17.182+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:17.181+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.182+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.182+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.182+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:17.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.182+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:17.182+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:17.182+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:17.182+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:17.182+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:17.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.183+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:30:17.183+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.565604ms 2023-10-21T04:30:17.183+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 3.652229ms 2023-10-21T04:30:17.183+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.183+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.183+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.183+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:17.184+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:17.184+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:17.184+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:17.184+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:17.184+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:30:17.184+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.184+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.184+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.184+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.184+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:17.185+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:17.185+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:17.185+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:17.185+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:17.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:30:17.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.180+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:17.186+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10032435514095102923 2023-10-21T04:30:17.186+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:17.186+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:17.186+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:17.186+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:17.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:30:17.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.187+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.190+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.190+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:17.190+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:17.190+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:17.190+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:17.190+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:17.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:30:17.191+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.191+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.191+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.191+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.224+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:17.226+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:17.234+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 71.014697ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:17.234+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:30:17.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:30:17.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.235+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.235+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.235+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:17.236+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:17.236+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:17.236+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:17.236+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:30:17.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:30:17.236+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.236+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.236+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.236+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.236+05:30 [Info] switched currmeta from 24 -> 25 force true 2023-10-21T04:30:17.235+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:44020:172.31.5.112:12000] 2023-10-21T04:30:17.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:30:17.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:30:17.238+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.238+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.238+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.238+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:17.239+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:17.239+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:17.239+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:17.239+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:30:17.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:30:17.239+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.239+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.239+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.240+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.240+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:30:17.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:17.241+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:17.241+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:17.241+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:17.241+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:30:17.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.241+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.246+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16961575949607441659 instance id 15298811879390062834 real instance id 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:17.246+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 137.975µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:17.255+05:30 [Info] LifecycleMgr.DeleteIndex() : index 17797267771846799851 does not exist. 2023-10-21T04:30:17.255+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 66.782µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:17.266+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.266+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:30:17.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:30:17.266+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.266+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:30:17.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:17.274+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:17.274+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:17.274+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:17.274+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:30:17.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.274+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:30:17.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:30:17.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:30:17.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:30:17.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:30:17.286+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.286+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:30:17.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:30:17.286+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Destroy Done 2023-10-21T04:30:17.286+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.286+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.286+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.286+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:17.286+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:17.286+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:17.286+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:17.287+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:30:17.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:30:17.287+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.287+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.287+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.287+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.294+05:30 [Info] LifecycleMgr.DeleteIndex() : index 11276338709617092948 does not exist. 2023-10-21T04:30:17.294+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 129.199µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:17.294+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:30:17.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:17.295+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:17.295+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:17.295+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:17.295+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:30:17.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.295+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.297+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:17.298+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:17.298+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:30:17.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:30:17.300+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.300+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:30:17.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:30:17.300+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.300+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.300+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.300+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:17.300+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:17.300+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:17.300+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:17.300+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:30:17.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:30:17.300+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.301+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.301+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.301+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.301+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:30:17.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:30:17.301+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.301+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:30:17.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:30:17.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:30:17.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:17.302+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:17.302+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:17.304+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.304+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:30:17.304+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:30:17.304+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.298+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.299+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.311+05:30 [Info] switched currmeta from 25 -> 26 force true 2023-10-21T04:30:17.313+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.315+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.325+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:17.325+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:17.328+05:30 [Info] switched currmeta from 26 -> 26 force true 2023-10-21T04:30:17.360+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44064:172.31.5.112:12000] 2023-10-21T04:30:17.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:30:17.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:30:17.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:30:17.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:30:17.372+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:17.372+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:30:17.372+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[2535881715450691825:[0] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:30:17.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:30:17.372+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:30:17.372+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[6 7 8 1 2 3 4 5] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:30:17.372+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:30:17.372+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:30:17.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:30:17.372+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:30:17.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:30:17.372+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.372+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:30:17.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:30:17.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:30:17.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:30:17.376+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:30:17.376+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Destroy Done 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:30:17.376+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:30:17.376+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Close Done 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:30:17.376+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.376+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:30:17.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.377+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.377+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.377+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.377+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.379+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:30:17.380+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.380+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Destroy Done 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:30:17.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:30:17.381+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:30:17.381+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:17.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:30:17.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:30:17.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:30:17.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:30:17.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:30:17.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:30:17.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:30:17.388+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.388+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Destroy Done 2023-10-21T04:30:17.396+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44072:172.31.5.112:12000] 2023-10-21T04:30:17.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:30:17.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:30:17.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:30:17.400+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.400+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Destroy Done 2023-10-21T04:30:17.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:30:17.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:30:17.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:30:17.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:30:17.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:30:17.410+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.410+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Destroy Done 2023-10-21T04:30:17.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:30:17.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:30:17.416+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44070:172.31.5.112:12000] 2023-10-21T04:30:17.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:30:17.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:30:17.418+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.418+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Destroy Done 2023-10-21T04:30:17.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:30:17.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:30:17.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:30:17.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:30:17.425+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:30:17.425+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:30:17.425+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Destroy Done 2023-10-21T04:30:17.433+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44074:172.31.5.112:12000] 2023-10-21T04:30:17.482+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44078:172.31.5.112:12000] 2023-10-21T04:30:17.486+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44076:172.31.5.112:12000] 2023-10-21T04:30:17.543+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44080:172.31.5.112:12000] 2023-10-21T04:30:17.560+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44082:172.31.5.112:12000] 2023-10-21T04:30:17.593+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44084:172.31.5.112:12000] 2023-10-21T04:30:17.606+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:44086:172.31.5.112:12000] 2023-10-21T04:30:17.607+05:30 [Info] progress stats collection done. 2023-10-21T04:30:17.607+05:30 [Info] send progress stats to clients 2023-10-21T04:30:17.611+05:30 [Info] Indexer::ReadMemstats Time Taken 1.418664ms 2023-10-21T04:30:18.068+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:30:18.196+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 16787445029868804812 2023-10-21T04:30:18.196+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2925063509317026816 2023-10-21T04:30:18.196+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2925063509317026816 2023-10-21T04:30:18.196+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 16787445029868804812 2023-10-21T04:30:18.196+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [16787445029868804812 2925063509317026816], elapsed: 770.96137ms 2023-10-21T04:30:18.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:30:18.196+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:30:18.196+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper stopped 2023-10-21T04:30:18.196+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': stopped 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:30:18.196+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:30:18.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:30:18.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:30:18.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:30:18.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:30:18.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:30:18.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:30:18.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:30:18.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:30:18.199+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:30:18.199+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:30:18.199+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:30:18.199+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:30:18.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:30:18.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shutdown completed 2023-10-21T04:30:18.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:30:18.199+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:30:18.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:30:18.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:30:18.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance destroyed 2023-10-21T04:30:18.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:30:18.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:30:18.201+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:30:18.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:30:18.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:30:18.204+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:30:18.204+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:30:18.204+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:30:18.204+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:30:18.204+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:30:18.204+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:30:18.206+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:30:18.206+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:30:18.206+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:30:18.206+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:30:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:30:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shutdown completed 2023-10-21T04:30:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:30:18.206+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:30:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:30:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:30:18.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance destroyed 2023-10-21T04:30:18.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:30:18.207+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Cleaned up Tenant with name:bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:30:18.207+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [16787445029868804812 2925063509317026816], elapsed(sec): 0.782146515 2023-10-21T04:30:18.210+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenab:c1:ca:ac:c7:98:c3:16 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 0cf7c80ee554dff9d153c0588632edfe ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken5:f4:88:1:79:ef:31:3f Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12997526226294542081 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14537831791783005540 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:18.210+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 0cf7c80ee554dff9d153c0588632edfe TransferTokenId: ShardTokenab:c1:ca:ac:c7:98:c3:16 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:30:18.211+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000201414 2023-10-21T04:30:18.211+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenab:c1:ca:ac:c7:98:c3:16, dropIndexes: false 2023-10-21T04:30:18.211+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardTokenab_c1_ca_ac_c7_98_c3_16, shardId :0 2023-10-21T04:30:18.211+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/0cf7c80ee554dff9d153c0588632edfe_ShardTokenab_c1_ca_ac_c7_98_c3_16, shardId :0 2023-10-21T04:30:18.211+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17406544375239214575 10170029854712240877] 2023-10-21T04:30:18.375+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54668 2023-10-21T04:30:18.376+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54670 2023-10-21T04:30:18.376+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54674 2023-10-21T04:30:18.376+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54672 2023-10-21T04:30:18.397+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54678 2023-10-21T04:30:18.397+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54676 2023-10-21T04:30:19.196+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17406544375239214575 2023-10-21T04:30:19.196+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17406544375239214575 2023-10-21T04:30:19.196+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 10170029854712240877 2023-10-21T04:30:19.196+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 10170029854712240877 2023-10-21T04:30:19.196+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17406544375239214575 10170029854712240877], elapsed: 984.522775ms 2023-10-21T04:30:19.196+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 17406544375239214575 from local file system, err: error destroy shard, shardId 17406544375239214575 not found 2023-10-21T04:30:19.196+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 10170029854712240877 from local file system, err: error destroy shard, shardId 10170029854712240877 not found 2023-10-21T04:30:19.196+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17406544375239214575 10170029854712240877], elapsed(sec): 0.984612228 2023-10-21T04:30:19.201+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:30:19.201+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:30:19.201+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.557µs 2023-10-21T04:30:19.201+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:30:19.201+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:30:19.201+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:30:19.212+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:30:19.212+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:30:19.212+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:30:19.213+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:30:19.213+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 30.936µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:19.215+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:30:19.215+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:30:19.215+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 2.398351963s in RebalanceServiceManager::recoverRebalance: 2023-10-21T04:30:19.215+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{18194302e3f10df59bd076e1c4142d0b 7060000 } 2023-10-21T04:30:21.184+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:30:21.184+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:30:21.184+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 4 NodeUUIDs: [7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:30:21.184+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:30:21.184+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:30:21.189+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:30:21.190+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:30:21.596+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39368 2023-10-21T04:30:21.646+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39368 2023-10-21T04:30:33.104+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:33.292+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:33.292+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:33.302+05:30 [Info] switched currmeta from 26 -> 26 force true 2023-10-21T04:30:33.350+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:30:33.360+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 148] updated cluster rebalancing status to true 2023-10-21T04:30:33.360+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 148] updating rebalancing status to true 2023-10-21T04:30:33.373+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:33.373+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:33.376+05:30 [Info] switched currmeta from 26 -> 26 force true 2023-10-21T04:30:33.671+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:33.774+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:33.774+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:33.785+05:30 [Info] switched currmeta from 26 -> 26 force true 2023-10-21T04:30:35.011+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {b27aa80dc2f9db32777f0cff5dbc5a10 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:30:35.015+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.894µs 2023-10-21T04:30:35.015+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:30:35.015+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:30:35.015+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:30:35.018+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:30:35.018+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:35.018+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.928959ms 2023-10-21T04:30:35.018+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:30:35.018+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:30:35.018+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:35.018+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 33.975µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:35.018+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/b27aa80dc2f9db32777f0cff5dbc5a10 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:b27aa80dc2f9db32777f0cff5dbc5a10]}]} 2023-10-21T04:30:35.025+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:30:35.029+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:30:35.032+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {b27aa80dc2f9db32777f0cff5dbc5a10 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:30:35.034+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:30:35.072+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"b27aa80dc2f9db32777f0cff5dbc5a10","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:30:35.075+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:35.077+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:36.090+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:30:36.097+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:30:36.097+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:30:36.106+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:30:36.106+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:30:36.107+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:30:36.107+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.074934796s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:30:36.107+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:30:36.107+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/b27aa80dc2f9db32777f0cff5dbc5a10 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:b27aa80dc2f9db32777f0cff5dbc5a10]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/b27aa80dc2f9db32777f0cff5dbc5a10 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:b27aa80dc2f9db32777f0cff5dbc5a10]}]} 2023-10-21T04:30:36.107+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:36.107+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:30:36.109+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:30:36.109+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:30:36.109+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:30:36.109+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/b27aa80dc2f9db32777f0cff5dbc5a10 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:b27aa80dc2f9db32777f0cff5dbc5a10]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/b27aa80dc2f9db32777f0cff5dbc5a10 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:b27aa80dc2f9db32777f0cff5dbc5a10]}]} 2023-10-21T04:30:36.109+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:30:36.109+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:36.113+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:30:36.113+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:30:36.177+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{7f731cd927c72e28ff24e69497b93571 7f731cd927c72e28ff24e69497b93571 plasma map[] [{21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{3345094951175719474 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{6377153279641237139 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 17797267771846799851 [{17205882817792019450 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{10601152754683910907 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908] []}] 1 0 0 1 false plasma 0}]}]} {21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{6043349427976659583 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{6225851069678382014 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12824340686309573428 [{14217467447424830582 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{2535881715450691825 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {964f96068adef144590fc7ab85b2311c 964f96068adef144590fc7ab85b2311c plasma map[] [{21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{8965923595543532942 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{8732144615635911668 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12824340686309573428 [{12266944006962282785 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{17434304260397056806 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575] []}] 1 0 0 1 false plasma 0}]}]} {21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{15864358084358521096 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{10032435514095102923 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 17797267771846799851 [{3023333489681759512 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{3464125101225762400 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {18194302e3f10df59bd076e1c4142d0b 18194302e3f10df59bd076e1c4142d0b plasma map[] [{5 bucket_1 _default c1 []}] [] 0 0 0 false} {8806f5706747af4e635b8da5fcd8aeee 8806f5706747af4e635b8da5fcd8aeee plasma map[] [{4 bucket_%2 _default c1 []} {8 bucket_1 _default c1 []} {4 bucket_%2 _system _query []}] [] 0 0 0 false}] map[]} 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.202+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:30:36.284+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39400 2023-10-21T04:30:36.292+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:30:36.292+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:30:36.293+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:30:36.293+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:30:36.303+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39400 2023-10-21T04:30:36.321+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9003 2023-10-21T04:30:36.321+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9004 2023-10-21T04:30:36.321+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9002 2023-10-21T04:30:36.321+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9001 2023-10-21T04:30:36.326+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39400 2023-10-21T04:30:36.329+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:30:36.329+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:30:36.329+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:30:36.329+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:30:36.378+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:30:36.378+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:30:36.378+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:30:36.378+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.378+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 17797267771846799851:3023333489681759512:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 3381524789655566900:17434304260397056806:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 11276338709617092948:3464125101225762400:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 12824340686309573428:12266944006962282785:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.379+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:36.380+05:30 [Info] Moving index 11276338709617092948:10601152754683910907:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.380+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 12824340686309573428:14217467447424830582:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 3381524789655566900:2535881715450691825:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.381+05:30 [Info] Moving index 17797267771846799851:17205882817792019450:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:36.382+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 18185143 Units 0 2023-10-21T04:30:36.382+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17581676 Units 0 2023-10-21T04:30:36.382+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 20715 Units 0 Tenants 0 2023-10-21T04:30:36.382+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 624214 Units 0 Tenants 0 2023-10-21T04:30:36.382+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9003 127.0.0.1:9004]] 2023-10-21T04:30:36.382+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:30:36.382+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:30:36.382+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:30:36.382+05:30 [Info] ---------------------------------------- 2023-10-21T04:30:36.382+05:30 [Info] 2023-10-21T04:30:36.383+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:18194302e3f10df59bd076e1c4142d0b, useLiveData:false 2023-10-21T04:30:36.383+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:30:36.384+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:30:36.384+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.384+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.384+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.384+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.384+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.384+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.384+05:30 [Info] Index total memory:157327869 (150.04M), mem:35904 (35.0625K), overhead:157291965 (150.005M), min:157327869 (150.04M), data:35904 (35.0625K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.384+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.384+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.384+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.384+05:30 [Info] Index total memory:157326969 (150.039M), mem:35125 (34.3018K), overhead:157291844 (150.005M), min:157326969 (150.039M), data:35125 (34.3018K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.384+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.384+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.384+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.384+05:30 [Info] Index total memory:157327402 (150.039M), mem:35500 (34.668K), overhead:157291902 (150.005M), min:157327402 (150.039M), data:35500 (34.668K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.384+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.384+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.384+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.384+05:30 [Info] Index total memory:157327740 (150.039M), mem:35793 (34.9541K), overhead:157291947 (150.005M), min:157327740 (150.039M), data:35793 (34.9541K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.384+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.384+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.384+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.384+05:30 [Info] Index total memory:157327968 (150.04M), mem:35990 (35.1465K), overhead:157291978 (150.005M), min:157327968 (150.04M), data:35990 (35.1465K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.384+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.384+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.384+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.384+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:17797267771846799851, instId:3023333489681759512, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:17434304260397056806, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157293232 (150.007M), mem:5916 (5.77734K), overhead:157287316 (150.001M), min:157293232 (150.007M), data:5916 (5.77734K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157327056 (150.039M), mem:35200 (34.375K), overhead:157291856 (150.005M), min:157327056 (150.039M), data:35200 (34.375K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157327287 (150.039M), mem:35400 (34.5703K), overhead:157291887 (150.005M), min:157327287 (150.039M), data:35400 (34.5703K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157328319 (150.04M), mem:36294 (35.4434K), overhead:157292025 (150.005M), min:157328319 (150.04M), data:36294 (35.4434K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157327252 (150.039M), mem:35370 (34.541K), overhead:157291882 (150.005M), min:157327252 (150.039M), data:35370 (34.541K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:157326760 (150.038M), mem:34944 (34.125K), overhead:157291816 (150.005M), min:157326760 (150.038M), data:34944 (34.125K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.385+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.385+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.385+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.385+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.386+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.386+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.386+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.386+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.386+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.386+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.386+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.386+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.386+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.386+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.386+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.386+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.386+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.386+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.386+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.386+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.386+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.386+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.386+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.386+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.386+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.386+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.386+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.386+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.386+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:3464125101225762400, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:157292546 (150.006M), mem:5322 (5.19727K), overhead:157287224 (150.001M), min:157292546 (150.006M), data:5322 (5.19727K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:12824340686309573428, instId:12266944006962282785, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.387+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.387+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.387+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.387+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.388+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.388+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.388+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:36.388+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.388+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.388+05:30 [Info] 2023-10-21T04:30:36.388+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:8806f5706747af4e635b8da5fcd8aeee, useLiveData:false 2023-10-21T04:30:36.388+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:30:36.388+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:30:36.388+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.388+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:10601152754683910907, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908], shardIdsInDefn: map[] 2023-10-21T04:30:36.388+05:30 [Info] Index total memory:157293697 (150.007M), mem:6318 (6.16992K), overhead:157287379 (150.001M), min:157293697 (150.007M), data:6318 (6.16992K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.388+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.388+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.388+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0080 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157327258 (150.039M), mem:35375 (34.5459K), overhead:157291883 (150.005M), min:157327258 (150.039M), data:35375 (34.5459K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157327980 (150.04M), mem:36000 (35.1562K), overhead:157291980 (150.005M), min:157327980 (150.04M), data:36000 (35.1562K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157327827 (150.04M), mem:35868 (35.0273K), overhead:157291959 (150.005M), min:157327827 (150.04M), data:35868 (35.0273K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157328340 (150.04M), mem:36312 (35.4609K), overhead:157292028 (150.005M), min:157328340 (150.04M), data:36312 (35.4609K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0066 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.389+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.389+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.389+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.389+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.390+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.390+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.390+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.390+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:12824340686309573428, instId:14217467447424830582, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.390+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0789 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:2535881715450691825, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593], shardIdsInDefn: map[] 2023-10-21T04:30:36.390+05:30 [Info] Index total memory:157293697 (150.007M), mem:6318 (6.16992K), overhead:157287379 (150.001M), min:157293697 (150.007M), data:6318 (6.16992K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.390+05:30 [Info] Index total memory:157328090 (150.04M), mem:36096 (35.25K), overhead:157291994 (150.005M), min:157328090 (150.04M), data:36096 (35.25K) cpu:0.0070 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.390+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.390+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.390+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.391+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0068 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.391+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.391+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.391+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.391+05:30 [Info] Index total memory:157328309 (150.04M), mem:36285 (35.4346K), overhead:157292024 (150.005M), min:157328309 (150.04M), data:36285 (35.4346K) cpu:0.0068 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.391+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.391+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.391+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.391+05:30 [Info] Index total memory:157327695 (150.039M), mem:35754 (34.916K), overhead:157291941 (150.005M), min:157327695 (150.039M), data:35754 (34.916K) cpu:0.0064 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.391+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.391+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.391+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.392+05:30 [Info] Index total memory:157328797 (150.04M), mem:36708 (35.8477K), overhead:157292089 (150.005M), min:157328797 (150.04M), data:36708 (35.8477K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.392+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.392+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.392+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.392+05:30 [Info] Index total memory:157327911 (150.04M), mem:35941 (35.0986K), overhead:157291970 (150.005M), min:157327911 (150.04M), data:35941 (35.0986K) cpu:0.0070 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.392+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.392+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.392+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.392+05:30 [Info] Index total memory:157328160 (150.04M), mem:36156 (35.3086K), overhead:157292004 (150.005M), min:157328160 (150.04M), data:36156 (35.3086K) cpu:0.0072 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.392+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.392+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.392+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.393+05:30 [Info] Index total memory:157327148 (150.039M), mem:35280 (34.4531K), overhead:157291868 (150.005M), min:157327148 (150.039M), data:35280 (34.4531K) cpu:0.0062 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.393+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.393+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.393+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.393+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.393+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.393+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.393+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.393+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.393+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.393+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.393+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.393+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.393+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.393+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.393+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.393+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.393+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.393+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.393+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.393+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.393+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.394+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.394+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.394+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.394+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.394+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.394+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.394+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.394+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.394+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.394+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:36.394+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.394+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.394+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:36.394+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:17797267771846799851, instId:17205882817792019450, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:36.394+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0813 io:0 (0) scan:0 drain:0 2023-10-21T04:30:36.394+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:36.394+05:30 [Info] **************************************** 2023-10-21T04:30:36.394+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 20715 Units 0 Tenants 0 2023-10-21T04:30:36.394+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 624214 Units 0 Tenants 0 2023-10-21T04:30:36.395+05:30 [Info] Generating Shard Transfer Token (ShardTokene3:eb:62:ad:c7:57:e9:4f) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Shards: [4718895791748289908 15531034489713578274] Destination: Region: InstId: 17995937906335902569 RealInstId: 3345094951175719474 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3345094951175719474 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10056669601346277951 RealInstId: 6377153279641237139 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6377153279641237139 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17205882817792019450 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17205882817792019450 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10601152754683910907 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10601152754683910907 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:36.396+05:30 [Info] Generating Shard Transfer Token (ShardTokenff:94:b2:b5:a5:cb:6c:40) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1c:d3:d3:9c:3a:b1:37:7f Shards: [7434639758576776593 1249143295319901191] Destination: Region: InstId: 14217467447424830582 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14217467447424830582 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 12362723730601043795 RealInstId: 6043349427976659583 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6043349427976659583 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 16206786221010696630 RealInstId: 6225851069678382014 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6225851069678382014 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 2535881715450691825 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2535881715450691825 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:36.397+05:30 [Info] Generating Shard Transfer Token (ShardToken42:27:2e:e5:3f:d:74:c0) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:eb:62:ad:c7:57:e9:4f Shards: [16787445029868804812 2925063509317026816] Destination: Region: InstId: 3284861513092822309 RealInstId: 15864358084358521096 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3023333489681759512 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 14016162161437609020 RealInstId: 10032435514095102923 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3464125101225762400 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:36.398+05:30 [Info] Generating Shard Transfer Token (ShardToken1c:d3:d3:9c:3a:b1:37:7f) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: Region: InstId: 9080388638359195747 RealInstId: 8965923595543532942 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11904179533192894084 RealInstId: 8732144615635911668 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12266944006962282785 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17434304260397056806 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:36.398+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:30:36.398+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:30:36.401+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenff:94:b2:b5:a5:cb:6c:40 2023-10-21T04:30:36.406+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken42:27:2e:e5:3f:d:74:c0 2023-10-21T04:30:36.460+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken1c:d3:d3:9c:3a:b1:37:7f 2023-10-21T04:30:36.465+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokene3:eb:62:ad:c7:57:e9:4f 2023-10-21T04:30:36.465+05:30 [Info] ShardRebalancer::observeRebalance &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:30:36.509+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:eb:62:ad:c7:57:e9:4f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17995937906335902569 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10056669601346277951 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.509+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1c:d3:d3:9c:3a:b1:37:7f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9080388638359195747 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11904179533192894084 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.510+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"b27aa80dc2f9db32777f0cff5dbc5a10","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:30:36.510+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:30:36.510+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken42:27:2e:e5:3f:d:74:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:eb:62:ad:c7:57:e9:4f Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3284861513092822309 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14016162161437609020 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.511+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:94:b2:b5:a5:cb:6c:40 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1c:d3:d3:9c:3a:b1:37:7f Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12362723730601043795 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16206786221010696630 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.511+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken42:27:2e:e5:3f:d:74:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:eb:62:ad:c7:57:e9:4f Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3284861513092822309 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14016162161437609020 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.512+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:30:36.512+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:30:36.512+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:30:36.512+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 21.593µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:36.518+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken42:27:2e:e5:3f:d:74:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:eb:62:ad:c7:57:e9:4f Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3284861513092822309 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14016162161437609020 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.520+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:36.726+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1c:d3:d3:9c:3a:b1:37:7f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9080388638359195747 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11904179533192894084 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.727+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:30:36.727+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:30:36.727+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:30:36.727+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 33.964µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:36.761+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:eb:62:ad:c7:57:e9:4f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17995937906335902569 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10056669601346277951 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.762+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:94:b2:b5:a5:cb:6c:40 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1c:d3:d3:9c:3a:b1:37:7f Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12362723730601043795 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16206786221010696630 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.764+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:36.764+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1c:d3:d3:9c:3a:b1:37:7f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9080388638359195747 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11904179533192894084 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.766+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:36.813+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:eb:62:ad:c7:57:e9:4f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17995937906335902569 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10056669601346277951 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.817+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:36.817+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:30:36.825+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:30:36.825+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 2 2023-10-21T04:30:36.895+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[7f731cd927c72e28ff24e69497b93571:ShardTokenff:94:b2:b5:a5:cb:6c:40 964f96068adef144590fc7ab85b2311c:ShardToken1c:d3:d3:9c:3a:b1:37:7f], currActiveTransfers: map[7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:2] 2023-10-21T04:30:36.895+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"b27aa80dc2f9db32777f0cff5dbc5a10","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:30:36.895+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:30:36.895+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:30:36.895+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:30:36.895+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:30:36.895+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:30:36.896+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:eb:62:ad:c7:57:e9:4f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17995937906335902569 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10056669601346277951 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.897+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken42:27:2e:e5:3f:d:74:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:eb:62:ad:c7:57:e9:4f Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3284861513092822309 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14016162161437609020 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.897+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:94:b2:b5:a5:cb:6c:40 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1c:d3:d3:9c:3a:b1:37:7f Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12362723730601043795 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16206786221010696630 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:36.898+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 21.494µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:36.899+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1c:d3:d3:9c:3a:b1:37:7f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9080388638359195747 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11904179533192894084 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:37.025+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1c:d3:d3:9c:3a:b1:37:7f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9080388638359195747 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11904179533192894084 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:37.025+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:30:37.025+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:37.025+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:37.025+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:30:37.025+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.136µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:37.025+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[10170029854712240877:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/10170029854712240877/shards/shard10170029854712240877 17406544375239214575:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/17406544375239214575/shards/shard17406544375239214575] b27aa80dc2f9db32777f0cff5dbc5a10 ShardToken1c:d3:d3:9c:3a:b1:37:7f /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0080ad920 0xc0080ad980 map[] 0xc004f42000 0xc004e8c060 false} 2023-10-21T04:30:37.026+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/17406544375239214575/shards/shard17406544375239214575 2023-10-21T04:30:37.026+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/10170029854712240877/shards/shard10170029854712240877 2023-10-21T04:30:37.028+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/17406544375239214575/shards/shard17406544375239214575/data: super block snapshot head offset :0 tail offset:757760 sbgen :136 2023-10-21T04:30:37.028+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/17406544375239214575/shards/shard17406544375239214575/data/recovery: super block snapshot head offset :0 tail offset:708608 sbgen :136 2023-10-21T04:30:37.029+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/10170029854712240877/shards/shard10170029854712240877/data: super block snapshot head offset :0 tail offset:651264 sbgen :126 2023-10-21T04:30:37.030+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/10170029854712240877/shards/shard10170029854712240877/data/recovery: super block snapshot head offset :0 tail offset:593920 sbgen :126 2023-10-21T04:30:37.037+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/17406544375239214575/shards/shard17406544375239214575: metadata saved successfully 2023-10-21T04:30:37.038+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:30:37.038+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:30:37.038+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 123803000 totalBytes: 1485636 numShards: 2 duration: 12.753568ms 2023-10-21T04:30:37.038+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17406544375239214575 bytesWritten: 1485636, totalBytes: 1485636, transferRate: 1.23803e+08 2023-10-21T04:30:37.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/17406544375239214575/shards/shard17406544375239214575, newshardId 17406544375239214575, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, shardsize 1485636, duration 12.783699ms 2023-10-21T04:30:37.039+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17406544375239214575, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, err: , elapsed(sec): 0.013069742 2023-10-21T04:30:37.039+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/10170029854712240877/shards/shard10170029854712240877: metadata saved successfully 2023-10-21T04:30:37.040+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:30:37.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:30:37.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f/10170029854712240877/shards/shard10170029854712240877, newshardId 10170029854712240877, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, shardsize 1264290, duration 13.432283ms 2023-10-21T04:30:37.040+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10170029854712240877, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, err: , elapsed(sec): 0.014376641 2023-10-21T04:30:37.040+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: b27aa80dc2f9db32777f0cff5dbc5a10, transferId: ShardToken1c:d3:d3:9c:3a:b1:37:7f 2023-10-21T04:30:37.040+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: b27aa80dc2f9db32777f0cff5dbc5a10, transferId: ShardToken1c:d3:d3:9c:3a:b1:37:7f, elapsed(sec): 0.000168173 2023-10-21T04:30:37.040+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[10170029854712240877: 17406544375239214575:], shardPaths: map[10170029854712240877:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 17406544375239214575:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.014610088 2023-10-21T04:30:37.040+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10170029854712240877 bytesWritten: 1264290, totalBytes: 1264290, transferRate: 9.725307692307693e+07 2023-10-21T04:30:37.040+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17406544375239214575 10170029854712240877], ttid: ShardToken1c:d3:d3:9c:3a:b1:37:7f, elapsed(sec): 0.015108666 2023-10-21T04:30:37.042+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:37.049+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1c:d3:d3:9c:3a:b1:37:7f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9080388638359195747 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11904179533192894084 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:37.049+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:30:37.049+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken1c:d3:d3:9c:3a:b1:37:7f 2023-10-21T04:30:37.049+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[10170029854712240877:true 17406544375239214575:true] 2023-10-21T04:30:37.049+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.494µs 2023-10-21T04:30:37.049+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [17406544375239214575 10170029854712240877], isLockedForRecovery: true 2023-10-21T04:30:37.049+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [17406544375239214575 10170029854712240877], errMap: map[10170029854712240877: 17406544375239214575:], elapsed: 8.633µs 2023-10-21T04:30:37.072+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:30:37.072+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:30:37.118+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8965923595543532942,realInstId: 0, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:37.118+05:30 [Info] Indexer::handleRecoverIndex InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:30:37.119+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 456.605µs 2023-10-21T04:30:37.119+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 536.143µs 2023-10-21T04:30:37.120+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:30:37.120+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 7 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:30:37.120+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken42:27:2e:e5:3f:d:74:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:eb:62:ad:c7:57:e9:4f Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3284861513092822309 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14016162161437609020 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:37.120+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:37.121+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:37.121+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:37.121+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': added 2023-10-21T04:30:37.121+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:30:37.121+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2925063509317026816:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/2925063509317026816/shards/shard2925063509317026816 16787445029868804812:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/16787445029868804812/shards/shard16787445029868804812] b27aa80dc2f9db32777f0cff5dbc5a10 ShardToken42:27:2e:e5:3f:d:74:c0 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0080ad920 0xc0080ad980 map[] 0xc0053cc960 0xc0002194a0 false} 2023-10-21T04:30:37.121+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/2925063509317026816/shards/shard2925063509317026816 2023-10-21T04:30:37.122+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8965923595543532942, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:30:37.122+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 49.826803ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:37.122+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 30.33µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:30:37.122+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/16787445029868804812/shards/shard16787445029868804812 2023-10-21T04:30:37.124+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner initialized 2023-10-21T04:30:37.124+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:37.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:30:37.125+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/2925063509317026816/shards/shard2925063509317026816/data: super block snapshot head offset :0 tail offset:794624 sbgen :144 2023-10-21T04:30:37.129+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/2925063509317026816/shards/shard2925063509317026816/data/recovery: super block snapshot head offset :0 tail offset:729088 sbgen :144 2023-10-21T04:30:37.130+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/16787445029868804812/shards/shard16787445029868804812/data: super block snapshot head offset :0 tail offset:921600 sbgen :156 2023-10-21T04:30:37.130+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner initialized 2023-10-21T04:30:37.130+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:37.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:30:37.130+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/16787445029868804812/shards/shard16787445029868804812/data/recovery: super block snapshot head offset :0 tail offset:876544 sbgen :156 2023-10-21T04:30:37.134+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCleaner initialized for recovery 2023-10-21T04:30:37.134+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:37.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery 2023-10-21T04:30:37.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:30:37.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true] 2023-10-21T04:30:37.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[708608] Data log Head [0] Tail [757760] 2023-10-21T04:30:37.134+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [708608] 2023-10-21T04:30:37.136+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:eb:62:ad:c7:57:e9:4f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17995937906335902569 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10056669601346277951 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:37.136+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:1 8806f5706747af4e635b8da5fcd8aeee:1 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:37.136+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.136+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:30:37.136+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:30:37.137+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.137+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:30:37.137+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.137+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:30:37.138+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.138+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:30:37.138+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.138+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:30:37.138+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.138+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:30:37.138+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.138+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:30:37.138+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/16787445029868804812/shards/shard16787445029868804812: metadata saved successfully 2023-10-21T04:30:37.138+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.139+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:30:37.139+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.139+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:30:37.139+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.140+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:30:37.140+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCleaner initialized for recovery 2023-10-21T04:30:37.140+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:37.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery 2023-10-21T04:30:37.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:30:37.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true] 2023-10-21T04:30:37.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[593920] Data log Head [0] Tail [651264] 2023-10-21T04:30:37.140+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [593920] 2023-10-21T04:30:37.142+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:30:37.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:30:37.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/16787445029868804812/shards/shard16787445029868804812, newshardId 16787445029868804812, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, shardsize 1817403, duration 19.367738ms 2023-10-21T04:30:37.142+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 16787445029868804812, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, err: , elapsed(sec): 0.020924631 2023-10-21T04:30:37.142+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 16787445029868804812 bytesWritten: 1817403, totalBytes: 1817403, transferRate: 9.56527894736842e+07 2023-10-21T04:30:37.143+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:30:37.143+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:30:37.143+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:30:37.143+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:30:37.143+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:30:37.145+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/2925063509317026816/shards/shard2925063509317026816: metadata saved successfully 2023-10-21T04:30:37.146+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:30:37.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:30:37.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0/2925063509317026816/shards/shard2925063509317026816, newshardId 2925063509317026816, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, shardsize 1542812, duration 24.94557ms 2023-10-21T04:30:37.146+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2925063509317026816, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, err: , elapsed(sec): 0.025236753 2023-10-21T04:30:37.146+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: b27aa80dc2f9db32777f0cff5dbc5a10, transferId: ShardToken42:27:2e:e5:3f:d:74:c0 2023-10-21T04:30:37.146+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: b27aa80dc2f9db32777f0cff5dbc5a10, transferId: ShardToken42:27:2e:e5:3f:d:74:c0, elapsed(sec): 0.000161324 2023-10-21T04:30:37.146+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2925063509317026816: 16787445029868804812:], shardPaths: map[2925063509317026816:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 16787445029868804812:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.02546801 2023-10-21T04:30:37.146+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2925063509317026816 bytesWritten: 1542812, totalBytes: 1542812, transferRate: 6.4283833333333336e+07 2023-10-21T04:30:37.146+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [16787445029868804812 2925063509317026816], ttid: ShardToken42:27:2e:e5:3f:d:74:c0, elapsed(sec): 0.025917439 2023-10-21T04:30:37.147+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.147+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:30:37.147+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.147+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:30:37.147+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.147+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:30:37.148+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.148+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:30:37.149+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.149+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:30:37.149+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.149+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:30:37.149+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.149+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:30:37.150+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.150+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:30:37.150+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.150+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:30:37.150+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.150+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:30:37.150+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.150+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:30:37.150+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.150+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:30:37.150+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : replayOffset: 638976 (RP:638976) (CP:360448) 2023-10-21T04:30:37.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [638976] took [9.922113ms] 2023-10-21T04:30:37.150+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053364, resident 1, memSz 9853, memSzIndex 0, numRecords 123 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053288, resident 1, memSz 9287, memSzIndex 0, numRecords 118 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1056128, resident 1, memSz 67959, memSzIndex 841, numRecords 1000 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053288, resident 1, memSz 9511, memSzIndex 76, numRecords 127 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053288, resident 1, memSz 9997, memSzIndex 77, numRecords 131 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053288, resident 1, memSz 9669, memSzIndex 76, numRecords 128 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053288, resident 1, memSz 8667, memSzIndex 0, numRecords 112 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053288, resident 1, memSz 10687, memSzIndex 75, numRecords 138 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1053364, resident 1, memSz 9834, memSzIndex 0, numRecords 123 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.151+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37418553 controller mem 3898 2023-10-21T04:30:37.151+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [651264] replayOffset [638976] 2023-10-21T04:30:37.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [11.579415ms] 2023-10-21T04:30:37.152+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.153+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.153+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.153+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.153+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.153+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.154+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.154+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.154+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.154+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.154+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.154+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:37.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true]. Built [17] plasmas, took [14.540714ms] 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 131 metaSz: 2514 rp [1] sn: 15 count: 131 metaSz: 2518 rp [2] sn: 16 count: 131 metaSz: 2520 2023-10-21T04:30:37.155+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: doInit: data UsedSpace 46503 recovery UsedSpace 50324 2023-10-21T04:30:37.157+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:30:37.157+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:30:37.157+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.158+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:30:37.158+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.158+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:30:37.158+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.158+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:30:37.158+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.158+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:30:37.158+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.159+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:30:37.159+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:37.159+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:30:37.159+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : replayOffset: 745472 (RP:745472) (CP:446464) 2023-10-21T04:30:37.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [745472] took [25.273314ms] 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057276, resident 1, memSz 11767, memSzIndex 0, numRecords 118 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057276, resident 1, memSz 12749, memSzIndex 0, numRecords 128 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1055942, resident 1, memSz 70473, memSzIndex 354, numRecords 1000 2023-10-21T04:30:37.159+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1055176, resident 1, memSz 938, memSzIndex 0, numRecords 8 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057276, resident 1, memSz 12667, memSzIndex 0, numRecords 127 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057426, resident 1, memSz 12294, memSzIndex 0, numRecords 123 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057426, resident 1, memSz 12313, memSzIndex 0, numRecords 123 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057276, resident 1, memSz 13729, memSzIndex 0, numRecords 138 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057276, resident 1, memSz 13063, memSzIndex 0, numRecords 131 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: request quota after header replay 1057276, resident 1, memSz 11171, memSzIndex 0, numRecords 112 2023-10-21T04:30:37.160+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:7 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4782 2023-10-21T04:30:37.160+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [757760] replayOffset [745472] 2023-10-21T04:30:37.171+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:37.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:30:37.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:30:37.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:30:37.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper started 2023-10-21T04:30:37.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:30:37.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex started 2023-10-21T04:30:37.181+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:37.181+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:37.182+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:37.182+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken1c:d3:d3:9c:3a:b1:37:7f panic: TestCode::TestActionAtTag - Inducing artificial panic at tag: 5 as wished goroutine 1458 [running]: github.com/couchbase/indexing/secondary/testcode.TestActionAtTag(0x7ba84f40974eb827?, 0x5) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/testcode/test_action_ci.go:45 +0x5e5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).startShardRecovery(0xc0081a0900, {0xc0076f8f14, 0x21}, 0xc006176400) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1870 +0x118c created by github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferTokenAsDest /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1431 +0x28d 2023-10-21T04:30:39.396+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=18194302e3f10df59bd076e1c4142d0b -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:30:39.396+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:30:39.396+05:30 [Info] Setting ipv6=false 2023-10-21T04:30:39.414+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:30:39.442+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:30:39.532+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:30:39.532+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:30:39.533+05:30 [Info] Setting log level to Info 2023-10-21T04:30:39.533+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:30:39.533+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:30:39.534+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:30:39.534+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:30:39.608+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:30:39 audit: created new audit service 2023-10-21T04:30:39.809+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:30:39.809+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:30:39.809+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:30:39.810+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:30:39.810+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:30:39.810+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:30:39.833+05:30 [Info] security context initialized 2023-10-21T04:30:39.833+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:30:39.833+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:30:39.833+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:30:39.833+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:30:39.833+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:30:39.837+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:30:39.838+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:30:39.838+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:30:39.839+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:30:39.846+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:30:39.846+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:30:39.848+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:30:39.848+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:30:39.852+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:39.858+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:30:39.859+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:30:39.860+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:30:39.860+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:30:39.899+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:30:39.900+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:30:39.900+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:30:39.905+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:30:39.905+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:30:39.910+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 207 us 2023-10-21T04:30:39.911+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:30:39.908+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:30:39.908+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:30:39.908+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:30:39.909+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:30:39.909+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:30:39.923+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:30:39.926+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:30:39.944+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:30:39.944+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:30:39.945+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:30:39.945+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:30:39.945+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:30:39.945+05:30 [Info] Indexer Id 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:39.945+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:30:39.945+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:30:39.949+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:30:39.951+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:30:39.955+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 202] updating cluster rev epoch 0 -> 1 2023-10-21T04:30:39.955+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 202] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:30:39.959+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 207] bucketNames has been updated (was map[], now map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43]) 2023-10-21T04:30:39.959+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 208] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:30:39.960+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 209] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:30:39.965+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 207] updated cluster rebalancing status to false 2023-10-21T04:30:39.965+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:30:39.965+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:30:39.965+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:30:39.968+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:30:39.968+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:30:39.968+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:30:39.968+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:30:39.969+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:30:39.969+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning true RebalanceToken &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:39.969+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:30:39.970+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:30:39.970+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:30:39.990+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:30:39.990+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:30:39.993+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:30:39.995+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:30:39.995+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:30:39.996+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 , elapsed: 26.416094ms 2023-10-21T04:30:39.997+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:39.997+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:30:39.997+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:30:39.997+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:30:39.997+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager true &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:39.997+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:30:39.997+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:39.997+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:30:39.999+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:30:39.999+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:30:39.999+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:30:39.999+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:40.000+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:30:40.000+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:30:40.001+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:30:40.001+05:30 [Warn] Indexer::validateIndexInstMap State INDEX_STATE_CREATED Stream NIL_STREAM Deferred false Not Valid For Recovery. Cleanup Index InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:30:40.001+05:30 [Info] ClustMgr:handleCleanupIndex mType: CLUST_MGR_CLEANUP_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:40.002+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:30:40.002+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:30:40.002+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:40.002+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:30:40.002+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 8910459001587546151 instance id 8965923595543532942 2023-10-21T04:30:40.003+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 8910459001587546151 2023-10-21T04:30:40.005+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index. Scheduling it for cleanup. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:30:40.006+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:30:40.006+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:30:40.006+05:30 [Info] : Recover shard shard10170029854712240877 from metadata completed. 2023-10-21T04:30:40.007+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:30:40.007+05:30 [Info] : Recover shard shard16787445029868804812 from metadata completed. 2023-10-21T04:30:40.007+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:30:40.007+05:30 [Info] : Recover shard shard17406544375239214575 from metadata completed. 2023-10-21T04:30:40.007+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:30:40.007+05:30 [Info] : Recover shard shard2925063509317026816 from metadata completed. 2023-10-21T04:30:40.007+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:30:40.008+05:30 [Info] : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:30:40.011+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:30:40.011+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:false Messages:[]} 2023-10-21T04:30:40.013+05:30 [Info] : metadata saved successfully 2023-10-21T04:30:40.013+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex sucessfully destroyed 2023-10-21T04:30:40.013+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:30:40.013+05:30 [Info] : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:30:40.016+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:30:40.017+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:30:40.018+05:30 [Info] : metadata saved successfully 2023-10-21T04:30:40.018+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index destroyed 2023-10-21T04:30:40.018+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index. 2023-10-21T04:30:40.022+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:30:40.022+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:30:40.026+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:30:40.027+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:30:40.027+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:40.027+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:30:40.027+05:30 [Info] RebalanceServiceManager::recoverRebalance: Init Pending Cleanup 2023-10-21T04:30:40.028+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:30:40.028+05:30 [Info] No instances in shard being recovered. Destroy shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:30:40.028+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:30:40.028+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) removed 2023-10-21T04:30:40.028+05:30 [Info] : All instance closed 2023-10-21T04:30:40.028+05:30 [Info] : All instance destroyed 2023-10-21T04:30:40.029+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:30:40.029+05:30 [Info] No instances in shard being recovered. Destroy shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:40.029+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:30:40.029+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:30:40.029+05:30 [Info] : All instance closed 2023-10-21T04:30:40.029+05:30 [Info] : All instance destroyed 2023-10-21T04:30:40.032+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:30:40.033+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:30:40.033+05:30 [Info] No instances in shard being recovered. Destroy shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:30:40.034+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:30:40.034+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) removed 2023-10-21T04:30:40.034+05:30 [Info] : All instance closed 2023-10-21T04:30:40.034+05:30 [Info] : All instance destroyed 2023-10-21T04:30:40.036+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:30:40.039+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:30:40.039+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:30:40.045+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:30:40.046+05:30 [Info] Setting log level to Info 2023-10-21T04:30:40.046+05:30 [Info] RebalanceServiceManager::doRecoverRebalance Found Global Rebalance Token &{18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:30:40.047+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:30:40.046+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:30:40.050+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:30:40.052+05:30 [Info] No instances in shard being recovered. Destroy shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:30:40.052+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:30:40.055+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:30:40.056+05:30 [Info] : All instance closed 2023-10-21T04:30:40.054+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:30:40.054+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {18194302e3f10df59bd076e1c4142d0b b27aa80dc2f9db32777f0cff5dbc5a10 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:30:40.057+05:30 [Info] : All instance destroyed 2023-10-21T04:30:40.058+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:30:40.058+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:30:40.058+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:30:40.060+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:30:40.066+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLEANUP_INDEX elapsed 63.809266ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:40.068+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:30:40.068+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:30:40.078+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:30:40.079+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:30:40.079+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:30:40.105+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:30:40.105+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:30:40.190+05:30 [Info] security context initialized 2023-10-21T04:30:40.190+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:30:40.194+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:30:40.217+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:30:40.217+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:40.217+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:40.217+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:30:40.231+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:30:40.252+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:30:40.253+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:30:40.254+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:30:40.256+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:30:40.256+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:30:40.258+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:30:40.258+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:30:40.263+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:30:40.284+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:30:40.284+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:30:40.290+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55286 2023-10-21T04:30:40.290+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:30:40.303+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 10.109829ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:40.303+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:30:40.304+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:30:40.304+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 96.847µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:40.310+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:30:40.329+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:30:40.332+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:30:40.333+05:30 [Info] initialized currmeta 25 force true 2023-10-21T04:30:40.339+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:40.339+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:30:40.339+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:30:40.341+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:30:40.352+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:40.353+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:30:40.353+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:30:40.355+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:30:40.356+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:38640" doAuth() ... 2023-10-21T04:30:40.356+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:38640" doReceive() ... 2023-10-21T04:30:40.363+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:40.363+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:30:40.363+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:30:40.365+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:30:40.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:30:40.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:30:40.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:30:40.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[2 3 4 5 6 7 8 1] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:30:40.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[2 3 4 5 6 7 8 1] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[7 8 1 2 3 4 5 6] 10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:30:40.375+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[7 8 1 2 3 4 5 6] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[2 3 4 5 6 7 8 1] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:30:40.376+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:30:40.376+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:30:40.376+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:30:40.378+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:30:40.380+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:30:40.380+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:30:40.380+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:30:40.380+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:30:40.380+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:30:40.380+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:30:40.380+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:30:40.381+05:30 [Info] progress stats collection done. 2023-10-21T04:30:40.381+05:30 [Info] send progress stats to clients 2023-10-21T04:30:40.381+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:30:40.381+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:30:40.381+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken1c:d3:d3:9c:3a:b1:37:7f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:94:b2:b5:a5:cb:6c:40 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9080388638359195747 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11904179533192894084 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:40.383+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: b27aa80dc2f9db32777f0cff5dbc5a10 TransferTokenId: ShardToken1c:d3:d3:9c:3a:b1:37:7f Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:30:40.383+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000189245 2023-10-21T04:30:40.383+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken1c:d3:d3:9c:3a:b1:37:7f, dropIndexes: true 2023-10-21T04:30:40.383+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f, shardId :0 2023-10-21T04:30:40.384+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken1c_d3_d3_9c_3a_b1_37_7f, shardId :0 2023-10-21T04:30:40.386+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 3.641336ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 1 error 2023-10-21T04:30:40.388+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 8910459001587546151 instance id 9080388638359195747 real instance id 8965923595543532942 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:40.388+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 203.374µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:40.389+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 10606939961365971246 instance id 11904179533192894084 real instance id 8732144615635911668 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:40.389+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 158.529µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:40.390+05:30 [Info] LifecycleMgr.DeleteIndex() : index 12824340686309573428 does not exist. 2023-10-21T04:30:40.390+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 84.708µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:40.392+05:30 [Info] LifecycleMgr.DeleteIndex() : index 3381524789655566900 does not exist. 2023-10-21T04:30:40.392+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 67.076µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:40.392+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17406544375239214575 10170029854712240877] 2023-10-21T04:30:40.395+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 27] will report aggregate recorder stats every 5m0s 2023-10-21T04:30:40.453+05:30 [Info] Indexer::ReadMemstats Time Taken 5.795099ms 2023-10-21T04:30:40.640+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:30:40.640+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:40.640+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[8 1 2 3 4 5 6 7] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:40.640+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[8 1 2 3 4 5 6 7] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:40.640+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[8 1 2 3 4 5 6 7] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:40.640+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[8 1 2 3 4 5 6 7] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:30:40.804+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:30:40.977+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 266] aggregate storage info: map[bucket_%2:map[kv:2304192 index:4350220] bucket_1:map[kv:2369728 index:4358328]] 2023-10-21T04:30:40.996+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:30:41.023+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17406544375239214575 2023-10-21T04:30:41.023+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 10170029854712240877 2023-10-21T04:30:41.023+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 10170029854712240877 2023-10-21T04:30:41.023+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17406544375239214575 2023-10-21T04:30:41.023+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17406544375239214575 10170029854712240877], elapsed: 630.880384ms 2023-10-21T04:30:41.023+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 17406544375239214575 from local file system, err: error destroy shard, shardId 17406544375239214575 not found 2023-10-21T04:30:41.023+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 10170029854712240877 from local file system, err: error destroy shard, shardId 10170029854712240877 not found 2023-10-21T04:30:41.023+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17406544375239214575 10170029854712240877], elapsed(sec): 0.630963791 2023-10-21T04:30:41.025+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken42:27:2e:e5:3f:d:74:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:eb:62:ad:c7:57:e9:4f Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3284861513092822309 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14016162161437609020 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:41.025+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: b27aa80dc2f9db32777f0cff5dbc5a10 TransferTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:30:41.026+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000138292 2023-10-21T04:30:41.026+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken42:27:2e:e5:3f:d:74:c0, dropIndexes: true 2023-10-21T04:30:41.026+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0, shardId :0 2023-10-21T04:30:41.026+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/b27aa80dc2f9db32777f0cff5dbc5a10_ShardToken42_27_2e_e5_3f_d_74_c0, shardId :0 2023-10-21T04:30:41.032+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16961575949607441659 instance id 3284861513092822309 real instance id 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:41.032+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 138.498µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:41.033+05:30 [Info] LifecycleMgr.DeleteIndex() : index 17797267771846799851 does not exist. 2023-10-21T04:30:41.033+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 61.876µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:41.033+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16337157693634799600 instance id 14016162161437609020 real instance id 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:41.033+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 116.944µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:41.044+05:30 [Info] LifecycleMgr.DeleteIndex() : index 11276338709617092948 does not exist. 2023-10-21T04:30:41.044+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 92.124µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:41.044+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:30:42.023+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 16787445029868804812 2023-10-21T04:30:42.023+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2925063509317026816 2023-10-21T04:30:42.023+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2925063509317026816 2023-10-21T04:30:42.023+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 16787445029868804812 2023-10-21T04:30:42.023+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [16787445029868804812 2925063509317026816], elapsed: 979.281243ms 2023-10-21T04:30:42.023+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 16787445029868804812 from local file system, err: error destroy shard, shardId 16787445029868804812 not found 2023-10-21T04:30:42.023+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 2925063509317026816 from local file system, err: error destroy shard, shardId 2925063509317026816 not found 2023-10-21T04:30:42.023+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [16787445029868804812 2925063509317026816], elapsed(sec): 0.97934902 2023-10-21T04:30:42.026+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokene3:eb:62:ad:c7:57:e9:4f MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: b27aa80dc2f9db32777f0cff5dbc5a10 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardToken42:27:2e:e5:3f:d:74:c0 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17995937906335902569 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10056669601346277951 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:42.028+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:30:42.028+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:30:42.028+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.622µs 2023-10-21T04:30:42.028+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:30:42.028+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:30:42.028+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:30:42.032+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:30:42.032+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:30:42.032+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:30:42.033+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:30:42.033+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 28.868µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:42.046+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:30:42.046+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:30:42.046+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 2.018740895s in RebalanceServiceManager::recoverRebalance: 2023-10-21T04:30:42.046+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{18194302e3f10df59bd076e1c4142d0b 7060000 } 2023-10-21T04:30:42.496+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55326 2023-10-21T04:30:42.496+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55324 2023-10-21T04:30:42.497+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55330 2023-10-21T04:30:42.499+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55332 2023-10-21T04:30:42.501+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55334 2023-10-21T04:30:42.521+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55336 2023-10-21T04:30:43.237+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:30:43.243+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:43.244+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:43.246+05:30 [Info] switched currmeta from 25 -> 25 force true 2023-10-21T04:30:44.998+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] 2023-10-21T04:30:44.999+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] 2023-10-21T04:30:44.999+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 4 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] 2023-10-21T04:30:44.999+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c] IsBalanced:false Messages:[]} 2023-10-21T04:30:44.999+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:30:45.001+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:30:45.001+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:30:45.034+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40084 2023-10-21T04:30:45.066+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40084 2023-10-21T04:30:52.140+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:52.164+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:52.165+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:52.167+05:30 [Info] switched currmeta from 25 -> 25 force true 2023-10-21T04:30:55.665+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:55.736+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:55.736+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:55.758+05:30 [Info] switched currmeta from 25 -> 25 force true 2023-10-21T04:30:55.887+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:30:55.887+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:30:55.926+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:55.926+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:55.939+05:30 [Info] switched currmeta from 25 -> 25 force true 2023-10-21T04:30:55.951+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 207] updated cluster rebalancing status to true 2023-10-21T04:30:55.951+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 207] updating rebalancing status to true 2023-10-21T04:30:56.193+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:30:56.194+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:56.194+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:56.200+05:30 [Info] switched currmeta from 25 -> 25 force true 2023-10-21T04:30:56.236+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:56.236+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:56.242+05:30 [Info] switched currmeta from 25 -> 25 force true 2023-10-21T04:30:57.337+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {098bb6b302f203e2ac2093f52370af18 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:30:57.341+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 6.02µs 2023-10-21T04:30:57.341+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:30:57.341+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:30:57.341+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:30:57.346+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:30:57.346+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:30:57.347+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 5.468664ms 2023-10-21T04:30:57.347+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:30:57.347+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:30:57.347+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:57.347+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.159µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:57.347+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/098bb6b302f203e2ac2093f52370af18 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:098bb6b302f203e2ac2093f52370af18]}]} 2023-10-21T04:30:57.353+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:30:57.353+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:30:57.374+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {098bb6b302f203e2ac2093f52370af18 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:30:57.376+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:30:57.412+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"098bb6b302f203e2ac2093f52370af18","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:30:57.415+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:57.418+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:30:58.432+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:30:58.438+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:30:58.438+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:30:58.445+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:30:58.445+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:30:58.445+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:30:58.445+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.07120706s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:30:58.445+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/098bb6b302f203e2ac2093f52370af18 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:098bb6b302f203e2ac2093f52370af18]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/098bb6b302f203e2ac2093f52370af18 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:098bb6b302f203e2ac2093f52370af18]}]} 2023-10-21T04:30:58.445+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:30:58.445+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:58.445+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:30:58.448+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:30:58.448+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:30:58.448+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:30:58.448+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/098bb6b302f203e2ac2093f52370af18 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:098bb6b302f203e2ac2093f52370af18]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/098bb6b302f203e2ac2093f52370af18 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:098bb6b302f203e2ac2093f52370af18]}]} 2023-10-21T04:30:58.450+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:30:58.450+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:30:58.499+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{7f731cd927c72e28ff24e69497b93571 7f731cd927c72e28ff24e69497b93571 plasma map[] [{21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{3345094951175719474 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{6377153279641237139 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 17797267771846799851 [{17205882817792019450 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{10601152754683910907 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908] []}] 1 0 0 1 false plasma 0}]}]} {21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{6043349427976659583 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{6225851069678382014 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12824340686309573428 [{14217467447424830582 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{2535881715450691825 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {964f96068adef144590fc7ab85b2311c 964f96068adef144590fc7ab85b2311c plasma map[] [{21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{15864358084358521096 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{10032435514095102923 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 17797267771846799851 [{3023333489681759512 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{3464125101225762400 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812] []}] 1 0 0 0 false plasma 0}]}]} {21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{8965923595543532942 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{8732144615635911668 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 12824340686309573428 [{12266944006962282785 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{17434304260397056806 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {18194302e3f10df59bd076e1c4142d0b 18194302e3f10df59bd076e1c4142d0b plasma map[] [{5 bucket_1 _default c1 []} {3 bucket_%2 _default c1 []}] [] 0 0 0 false} {8806f5706747af4e635b8da5fcd8aeee 8806f5706747af4e635b8da5fcd8aeee plasma map[] [{8 bucket_%2 _default c1 []} {12 bucket_1 _default c1 []} {4 bucket_%2 _system _query []}] [] 0 0 0 false}] map[]} 2023-10-21T04:30:58.589+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40144 2023-10-21T04:30:58.597+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:30:58.598+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:30:58.598+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:30:58.598+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:30:58.598+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40144 2023-10-21T04:30:58.626+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9002 2023-10-21T04:30:58.627+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9001 2023-10-21T04:30:58.627+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9004 2023-10-21T04:30:58.627+05:30 [Info] Planner::processDeleteToken: processing 36 tokens for node 127.0.0.1:9003 2023-10-21T04:30:58.628+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40144 2023-10-21T04:30:58.634+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:30:58.634+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:30:58.634+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:30:58.634+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:30:58.681+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:30:58.681+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:30:58.681+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:30:58.681+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:30:58.681+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 17797267771846799851:17205882817792019450:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 11276338709617092948:10601152754683910907:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 12824340686309573428:14217467447424830582:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.681+05:30 [Info] Moving index 3381524789655566900:2535881715450691825:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 11276338709617092948:3464125101225762400:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 12824340686309573428:12266944006962282785:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.682+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.683+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.683+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.683+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.683+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.683+05:30 [Info] Moving index 17797267771846799851:3023333489681759512:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.683+05:30 [Info] Moving index 3381524789655566900:17434304260397056806:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:30:58.683+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17581676 Units 0 2023-10-21T04:30:58.683+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 18185143 Units 0 2023-10-21T04:30:58.683+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 20715 Units 0 Tenants 0 2023-10-21T04:30:58.683+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 624214 Units 0 Tenants 0 2023-10-21T04:30:58.683+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9003 127.0.0.1:9004]] 2023-10-21T04:30:58.683+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:30:58.683+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:30:58.683+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:30:58.683+05:30 [Info] ---------------------------------------- 2023-10-21T04:30:58.683+05:30 [Info] 2023-10-21T04:30:58.683+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:18194302e3f10df59bd076e1c4142d0b, useLiveData:false 2023-10-21T04:30:58.683+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:30:58.683+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:30:58.683+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.683+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.683+05:30 [Info] Index total memory:157327056 (150.039M), mem:35200 (34.375K), overhead:157291856 (150.005M), min:157327056 (150.039M), data:35200 (34.375K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.683+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.683+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.683+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.683+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.683+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.683+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.683+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.683+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.683+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.683+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.683+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.683+05:30 [Info] Index total memory:157327287 (150.039M), mem:35400 (34.5703K), overhead:157291887 (150.005M), min:157327287 (150.039M), data:35400 (34.5703K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.683+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.683+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.683+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.683+05:30 [Info] Index total memory:157328319 (150.04M), mem:36294 (35.4434K), overhead:157292025 (150.005M), min:157328319 (150.04M), data:36294 (35.4434K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:157327252 (150.039M), mem:35370 (34.541K), overhead:157291882 (150.005M), min:157327252 (150.039M), data:35370 (34.541K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:157326760 (150.038M), mem:34944 (34.125K), overhead:157291816 (150.005M), min:157326760 (150.038M), data:34944 (34.125K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.684+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.684+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.684+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.684+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:3464125101225762400, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:157292546 (150.006M), mem:5322 (5.19727K), overhead:157287224 (150.001M), min:157292546 (150.006M), data:5322 (5.19727K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:12824340686309573428, instId:12266944006962282785, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.685+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.685+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.685+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.685+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157327869 (150.04M), mem:35904 (35.0625K), overhead:157291965 (150.005M), min:157327869 (150.04M), data:35904 (35.0625K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157326969 (150.039M), mem:35125 (34.3018K), overhead:157291844 (150.005M), min:157326969 (150.039M), data:35125 (34.3018K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157327402 (150.039M), mem:35500 (34.668K), overhead:157291902 (150.005M), min:157327402 (150.039M), data:35500 (34.668K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157327740 (150.039M), mem:35793 (34.9541K), overhead:157291947 (150.005M), min:157327740 (150.039M), data:35793 (34.9541K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157327968 (150.04M), mem:35990 (35.1465K), overhead:157291978 (150.005M), min:157327968 (150.04M), data:35990 (35.1465K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:17797267771846799851, instId:3023333489681759512, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:17434304260397056806, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:157293232 (150.007M), mem:5916 (5.77734K), overhead:157287316 (150.001M), min:157293232 (150.007M), data:5916 (5.77734K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] 2023-10-21T04:30:58.686+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:8806f5706747af4e635b8da5fcd8aeee, useLiveData:false 2023-10-21T04:30:58.686+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:30:58.686+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.686+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.686+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.686+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.686+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:17797267771846799851, instId:17205882817792019450, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0813 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:10601152754683910907, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:157293697 (150.007M), mem:6318 (6.16992K), overhead:157287379 (150.001M), min:157293697 (150.007M), data:6318 (6.16992K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0080 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.687+05:30 [Info] Index total memory:157327258 (150.039M), mem:35375 (34.5459K), overhead:157291883 (150.005M), min:157327258 (150.039M), data:35375 (34.5459K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.687+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.687+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.687+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:157327980 (150.04M), mem:36000 (35.1562K), overhead:157291980 (150.005M), min:157327980 (150.04M), data:36000 (35.1562K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.688+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.688+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.688+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.688+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.688+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.688+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:157327827 (150.04M), mem:35868 (35.0273K), overhead:157291959 (150.005M), min:157327827 (150.04M), data:35868 (35.0273K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.688+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.688+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.688+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:157328340 (150.04M), mem:36312 (35.4609K), overhead:157292028 (150.005M), min:157328340 (150.04M), data:36312 (35.4609K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.688+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.688+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.688+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0066 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.688+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.688+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.688+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.688+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.688+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.688+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.688+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.688+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.688+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.688+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:12824340686309573428, instId:14217467447424830582, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0789 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:2535881715450691825, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:157293697 (150.007M), mem:6318 (6.16992K), overhead:157287379 (150.001M), min:157293697 (150.007M), data:6318 (6.16992K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:157328090 (150.04M), mem:36096 (35.25K), overhead:157291994 (150.005M), min:157328090 (150.04M), data:36096 (35.25K) cpu:0.0070 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.689+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.689+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0068 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.689+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.689+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.690+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.690+05:30 [Info] Index total memory:157328309 (150.04M), mem:36285 (35.4346K), overhead:157292024 (150.005M), min:157328309 (150.04M), data:36285 (35.4346K) cpu:0.0068 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.690+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.690+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.690+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.690+05:30 [Info] Index total memory:157327695 (150.039M), mem:35754 (34.916K), overhead:157291941 (150.005M), min:157327695 (150.039M), data:35754 (34.916K) cpu:0.0064 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.690+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.690+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.690+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.690+05:30 [Info] Index total memory:157328797 (150.04M), mem:36708 (35.8477K), overhead:157292089 (150.005M), min:157328797 (150.04M), data:36708 (35.8477K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.690+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.690+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.690+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.690+05:30 [Info] Index total memory:157327911 (150.04M), mem:35941 (35.0986K), overhead:157291970 (150.005M), min:157327911 (150.04M), data:35941 (35.0986K) cpu:0.0070 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.690+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.690+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.690+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.690+05:30 [Info] Index total memory:157328160 (150.04M), mem:36156 (35.3086K), overhead:157292004 (150.005M), min:157328160 (150.04M), data:36156 (35.3086K) cpu:0.0072 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.690+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.690+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:30:58.690+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:30:58.690+05:30 [Info] Index total memory:157327148 (150.039M), mem:35280 (34.4531K), overhead:157291868 (150.005M), min:157327148 (150.039M), data:35280 (34.4531K) cpu:0.0062 io:0 (0) scan:0 drain:0 2023-10-21T04:30:58.690+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:30:58.690+05:30 [Info] **************************************** 2023-10-21T04:30:58.691+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 20715 Units 0 Tenants 0 2023-10-21T04:30:58.691+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 624214 Units 0 Tenants 0 2023-10-21T04:30:58.691+05:30 [Info] Generating Shard Transfer Token (ShardToken77:a6:d4:cd:41:6c:45:bd) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2f:a7:bd:b5:1a:16:61:f6 Shards: [17406544375239214575 10170029854712240877] Destination: Region: InstId: 7832199063701440128 RealInstId: 8965923595543532942 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 16895258949616040095 RealInstId: 8732144615635911668 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12266944006962282785 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17434304260397056806 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:58.692+05:30 [Info] Generating Shard Transfer Token (ShardToken17:ba:f0:81:68:83:f1:eb) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: Region: InstId: 15448912456688985852 RealInstId: 10032435514095102923 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 5684298908800426981 RealInstId: 15864358084358521096 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3023333489681759512 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3464125101225762400 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:58.694+05:30 [Info] Generating Shard Transfer Token (ShardToken2f:a7:bd:b5:1a:16:61:f6) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Shards: [7434639758576776593 1249143295319901191] Destination: Region: InstId: 1684463754565405483 RealInstId: 6225851069678382014 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6225851069678382014 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 14217467447424830582 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14217467447424830582 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 5235575068708680783 RealInstId: 6043349427976659583 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6043349427976659583 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2535881715450691825 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2535881715450691825 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:58.694+05:30 [Info] Generating Shard Transfer Token (ShardTokenf9:5d:ac:9c:32:8:f6:4a) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Shards: [4718895791748289908 15531034489713578274] Destination: Region: InstId: 17205882817792019450 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17205882817792019450 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 13981755107353251708 RealInstId: 3345094951175719474 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3345094951175719474 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 7140146642632649428 RealInstId: 6377153279641237139 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6377153279641237139 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10601152754683910907 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10601152754683910907 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:30:58.695+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:30:58.695+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:30:58.714+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken77:a6:d4:cd:41:6c:45:bd 2023-10-21T04:30:58.732+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken17:ba:f0:81:68:83:f1:eb 2023-10-21T04:30:58.746+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken2f:a7:bd:b5:1a:16:61:f6 2023-10-21T04:30:58.778+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenf9:5d:ac:9c:32:8:f6:4a 2023-10-21T04:30:58.778+05:30 [Info] ShardRebalancer::observeRebalance &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:30:58.827+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf9:5d:ac:9c:32:8:f6:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13981755107353251708 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7140146642632649428 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:58.846+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"098bb6b302f203e2ac2093f52370af18","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:30:58.846+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:30:58.850+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken17:ba:f0:81:68:83:f1:eb MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15448912456688985852 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5684298908800426981 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:58.861+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2f:a7:bd:b5:1a:16:61:f6 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1684463754565405483 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 5235575068708680783 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:58.865+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken77:a6:d4:cd:41:6c:45:bd MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2f:a7:bd:b5:1a:16:61:f6 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7832199063701440128 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16895258949616040095 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:58.865+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:30:58.865+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:30:58.865+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:30:58.865+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.182µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:58.943+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken17:ba:f0:81:68:83:f1:eb MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15448912456688985852 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5684298908800426981 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:58.943+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:30:58.943+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:30:58.944+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:30:58.944+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 21.733µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:59.026+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken77:a6:d4:cd:41:6c:45:bd MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2f:a7:bd:b5:1a:16:61:f6 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7832199063701440128 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16895258949616040095 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.036+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.037+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken17:ba:f0:81:68:83:f1:eb MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15448912456688985852 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5684298908800426981 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.059+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.059+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2f:a7:bd:b5:1a:16:61:f6 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1684463754565405483 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 5235575068708680783 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.127+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2f:a7:bd:b5:1a:16:61:f6 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1684463754565405483 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 5235575068708680783 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.164+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.198+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf9:5d:ac:9c:32:8:f6:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13981755107353251708 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7140146642632649428 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.222+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf9:5d:ac:9c:32:8:f6:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13981755107353251708 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7140146642632649428 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.230+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.230+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:30:59.241+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:30:59.241+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 2 2023-10-21T04:30:59.299+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[7f731cd927c72e28ff24e69497b93571:ShardTokenf9:5d:ac:9c:32:8:f6:4a 964f96068adef144590fc7ab85b2311c:ShardToken17:ba:f0:81:68:83:f1:eb], currActiveTransfers: map[7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:2] 2023-10-21T04:30:59.299+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"098bb6b302f203e2ac2093f52370af18","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:30:59.299+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:30:59.299+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:30:59.299+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:30:59.299+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:30:59.299+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:30:59.300+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken77:a6:d4:cd:41:6c:45:bd MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2f:a7:bd:b5:1a:16:61:f6 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7832199063701440128 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16895258949616040095 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.301+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2f:a7:bd:b5:1a:16:61:f6 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1684463754565405483 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 5235575068708680783 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.301+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken17:ba:f0:81:68:83:f1:eb MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15448912456688985852 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5684298908800426981 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.301+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 17.369µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:59.310+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf9:5d:ac:9c:32:8:f6:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13981755107353251708 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7140146642632649428 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.502+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken77:a6:d4:cd:41:6c:45:bd MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2f:a7:bd:b5:1a:16:61:f6 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7832199063701440128 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16895258949616040095 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.503+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:30:59.503+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:59.503+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:59.503+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:30:59.503+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 23.249µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:59.503+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[10170029854712240877:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/10170029854712240877/shards/shard10170029854712240877 17406544375239214575:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/17406544375239214575/shards/shard17406544375239214575] 098bb6b302f203e2ac2093f52370af18 ShardToken77:a6:d4:cd:41:6c:45:bd /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc006ea4120 0xc006ea4180 map[] 0xc0071a2ea0 0xc0074c6d20 false} 2023-10-21T04:30:59.504+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/17406544375239214575/shards/shard17406544375239214575 2023-10-21T04:30:59.505+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/10170029854712240877/shards/shard10170029854712240877 2023-10-21T04:30:59.506+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/17406544375239214575/shards/shard17406544375239214575/data: super block snapshot head offset :0 tail offset:757760 sbgen :136 2023-10-21T04:30:59.507+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/17406544375239214575/shards/shard17406544375239214575/data/recovery: super block snapshot head offset :0 tail offset:708608 sbgen :136 2023-10-21T04:30:59.509+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/10170029854712240877/shards/shard10170029854712240877/data: super block snapshot head offset :0 tail offset:651264 sbgen :126 2023-10-21T04:30:59.509+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/10170029854712240877/shards/shard10170029854712240877/data/recovery: super block snapshot head offset :0 tail offset:593920 sbgen :126 2023-10-21T04:30:59.514+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/17406544375239214575/shards/shard17406544375239214575: metadata saved successfully 2023-10-21T04:30:59.515+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:30:59.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:30:59.516+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 123803000 totalBytes: 1485636 numShards: 2 duration: 12.193141ms 2023-10-21T04:30:59.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/17406544375239214575/shards/shard17406544375239214575, newshardId 17406544375239214575, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, shardsize 1485636, duration 12.124654ms 2023-10-21T04:30:59.516+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17406544375239214575, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, err: , elapsed(sec): 0.012758313 2023-10-21T04:30:59.516+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17406544375239214575 bytesWritten: 1485636, totalBytes: 1485636, transferRate: 1.23803e+08 2023-10-21T04:30:59.516+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/10170029854712240877/shards/shard10170029854712240877: metadata saved successfully 2023-10-21T04:30:59.517+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:30:59.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:30:59.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd/10170029854712240877/shards/shard10170029854712240877, newshardId 10170029854712240877, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, shardsize 1264290, duration 12.256922ms 2023-10-21T04:30:59.517+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10170029854712240877, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, err: , elapsed(sec): 0.013752035 2023-10-21T04:30:59.517+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10170029854712240877 bytesWritten: 1264290, totalBytes: 1264290, transferRate: 1.053575e+08 2023-10-21T04:30:59.517+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 098bb6b302f203e2ac2093f52370af18, transferId: ShardToken77:a6:d4:cd:41:6c:45:bd 2023-10-21T04:30:59.517+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 098bb6b302f203e2ac2093f52370af18, transferId: ShardToken77:a6:d4:cd:41:6c:45:bd, elapsed(sec): 0.000208715 2023-10-21T04:30:59.517+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[10170029854712240877: 17406544375239214575:], shardPaths: map[10170029854712240877:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 17406544375239214575:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.014041781 2023-10-21T04:30:59.517+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17406544375239214575 10170029854712240877], ttid: ShardToken77:a6:d4:cd:41:6c:45:bd, elapsed(sec): 0.014533414 2023-10-21T04:30:59.520+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.545+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken17:ba:f0:81:68:83:f1:eb MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15448912456688985852 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5684298908800426981 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.549+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:2 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:59.549+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:59.549+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:30:59.549+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:30:59.549+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.118µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:59.549+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2925063509317026816:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/2925063509317026816/shards/shard2925063509317026816 16787445029868804812:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/16787445029868804812/shards/shard16787445029868804812] 098bb6b302f203e2ac2093f52370af18 ShardToken17:ba:f0:81:68:83:f1:eb /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc006ea4120 0xc006ea4180 map[] 0xc0076f6600 0xc007728120 false} 2023-10-21T04:30:59.549+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/2925063509317026816/shards/shard2925063509317026816 2023-10-21T04:30:59.550+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/16787445029868804812/shards/shard16787445029868804812 2023-10-21T04:30:59.553+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken77:a6:d4:cd:41:6c:45:bd MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2f:a7:bd:b5:1a:16:61:f6 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7832199063701440128 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16895258949616040095 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.553+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:59.553+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken77:a6:d4:cd:41:6c:45:bd 2023-10-21T04:30:59.553+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[10170029854712240877:true 17406544375239214575:true] 2023-10-21T04:30:59.553+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.767µs 2023-10-21T04:30:59.553+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [17406544375239214575 10170029854712240877], isLockedForRecovery: true 2023-10-21T04:30:59.553+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [17406544375239214575 10170029854712240877], errMap: map[10170029854712240877: 17406544375239214575:], elapsed: 8.67µs 2023-10-21T04:30:59.554+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/16787445029868804812/shards/shard16787445029868804812/data: super block snapshot head offset :0 tail offset:921600 sbgen :156 2023-10-21T04:30:59.555+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/16787445029868804812/shards/shard16787445029868804812/data/recovery: super block snapshot head offset :0 tail offset:876544 sbgen :156 2023-10-21T04:30:59.556+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:30:59.556+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:30:59.558+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/2925063509317026816/shards/shard2925063509317026816/data: super block snapshot head offset :0 tail offset:794624 sbgen :144 2023-10-21T04:30:59.558+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/2925063509317026816/shards/shard2925063509317026816/data/recovery: super block snapshot head offset :0 tail offset:729088 sbgen :144 2023-10-21T04:30:59.559+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/16787445029868804812/shards/shard16787445029868804812: metadata saved successfully 2023-10-21T04:30:59.559+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:30:59.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:30:59.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/16787445029868804812/shards/shard16787445029868804812, newshardId 16787445029868804812, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, shardsize 1817403, duration 9.164666ms 2023-10-21T04:30:59.560+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 16787445029868804812, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, err: , elapsed(sec): 0.010622001 2023-10-21T04:30:59.560+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 16787445029868804812 bytesWritten: 1817403, totalBytes: 1817403, transferRate: 2.0193366666666666e+08 2023-10-21T04:30:59.574+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/2925063509317026816/shards/shard2925063509317026816: metadata saved successfully 2023-10-21T04:30:59.576+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:30:59.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:30:59.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb/2925063509317026816/shards/shard2925063509317026816, newshardId 2925063509317026816, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, shardsize 1542812, duration 27.167737ms 2023-10-21T04:30:59.577+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2925063509317026816, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, err: , elapsed(sec): 0.027788381 2023-10-21T04:30:59.577+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 098bb6b302f203e2ac2093f52370af18, transferId: ShardToken17:ba:f0:81:68:83:f1:eb 2023-10-21T04:30:59.577+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 098bb6b302f203e2ac2093f52370af18, transferId: ShardToken17:ba:f0:81:68:83:f1:eb, elapsed(sec): 0.00040852 2023-10-21T04:30:59.578+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2925063509317026816: 16787445029868804812:], shardPaths: map[2925063509317026816:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 16787445029868804812:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.02861154 2023-10-21T04:30:59.577+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2925063509317026816 bytesWritten: 1542812, totalBytes: 1542812, transferRate: 5.714118518518519e+07 2023-10-21T04:30:59.581+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [16787445029868804812 2925063509317026816], ttid: ShardToken17:ba:f0:81:68:83:f1:eb, elapsed(sec): 0.032176273 2023-10-21T04:30:59.605+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.619+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8965923595543532942,realInstId: 0, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:30:59.620+05:30 [Info] Indexer::handleRecoverIndex InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:30:59.621+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 776.369µs 2023-10-21T04:30:59.621+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 877.508µs 2023-10-21T04:30:59.621+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:30:59.621+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8965923595543532942, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:30:59.622+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 65.976737ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:59.621+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken17:ba:f0:81:68:83:f1:eb MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15448912456688985852 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5684298908800426981 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.622+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:59.622+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken17:ba:f0:81:68:83:f1:eb 2023-10-21T04:30:59.622+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2925063509317026816:true 16787445029868804812:true] 2023-10-21T04:30:59.622+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 6 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:30:59.622+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.657µs 2023-10-21T04:30:59.623+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [16787445029868804812 2925063509317026816], isLockedForRecovery: true 2023-10-21T04:30:59.623+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [16787445029868804812 2925063509317026816], errMap: map[2925063509317026816: 16787445029868804812:], elapsed: 10.87µs 2023-10-21T04:30:59.623+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': added 2023-10-21T04:30:59.625+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner initialized 2023-10-21T04:30:59.625+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:59.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:30:59.626+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:30:59.626+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner initialized 2023-10-21T04:30:59.626+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:59.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:30:59.627+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCleaner initialized for recovery 2023-10-21T04:30:59.627+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:59.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery 2023-10-21T04:30:59.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:30:59.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true] 2023-10-21T04:30:59.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[593920] Data log Head [0] Tail [651264] 2023-10-21T04:30:59.627+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [593920] 2023-10-21T04:30:59.628+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCleaner initialized for recovery 2023-10-21T04:30:59.628+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:59.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery 2023-10-21T04:30:59.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:30:59.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true] 2023-10-21T04:30:59.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[708608] Data log Head [0] Tail [757760] 2023-10-21T04:30:59.628+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [708608] 2023-10-21T04:30:59.631+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.631+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:30:59.631+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:30:59.631+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:30:59.631+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:30:59.631+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.632+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:30:59.632+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.632+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:30:59.636+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.637+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:30:59.637+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.637+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:30:59.637+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.637+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:30:59.636+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.638+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:30:59.639+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.639+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:30:59.639+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.639+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:30:59.640+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.638+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.640+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:30:59.648+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.653+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.653+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:30:59.654+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.654+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:30:59.654+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.657+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:30:59.659+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:30:59.662+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.662+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:30:59.662+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.663+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:30:59.663+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.663+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:30:59.666+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.666+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:30:59.667+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.667+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:30:59.667+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.667+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:30:59.667+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.667+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:30:59.667+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.668+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:30:59.668+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.668+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:30:59.669+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.669+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:30:59.669+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.670+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:30:59.670+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.670+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:30:59.670+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : replayOffset: 745472 (RP:745472) (CP:446464) 2023-10-21T04:30:59.670+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [745472] took [42.041908ms] 2023-10-21T04:30:59.671+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12313, memSzIndex 0, numRecords 123 2023-10-21T04:30:59.671+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.671+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1055942, resident 1, memSz 70473, memSzIndex 354, numRecords 1000 2023-10-21T04:30:59.671+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.671+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.671+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.672+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.672+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.672+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1055176, resident 1, memSz 938, memSzIndex 0, numRecords 8 2023-10-21T04:30:59.672+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.673+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12294, memSzIndex 0, numRecords 123 2023-10-21T04:30:59.673+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.673+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.674+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.674+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.675+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.675+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.676+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.676+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13063, memSzIndex 0, numRecords 131 2023-10-21T04:30:59.676+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.676+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12749, memSzIndex 0, numRecords 128 2023-10-21T04:30:59.676+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.677+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.677+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36362311 controller mem 2628 2023-10-21T04:30:59.677+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057276, resident 1, memSz 11171, memSzIndex 0, numRecords 112 2023-10-21T04:30:59.678+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36368027 controller mem 2692 2023-10-21T04:30:59.678+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057276, resident 1, memSz 11767, memSzIndex 0, numRecords 118 2023-10-21T04:30:59.678+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36368027 controller mem 2692 2023-10-21T04:30:59.678+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.679+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36366127 controller mem 2692 2023-10-21T04:30:59.680+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.680+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36366127 controller mem 2692 2023-10-21T04:30:59.681+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12667, memSzIndex 0, numRecords 127 2023-10-21T04:30:59.681+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36364987 controller mem 2692 2023-10-21T04:30:59.699+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13729, memSzIndex 0, numRecords 138 2023-10-21T04:30:59.683+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3464125101225762400,realInstId: 0, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:30:59.700+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 36364987 controller mem 2692 2023-10-21T04:30:59.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [757760] replayOffset [745472] 2023-10-21T04:30:59.700+05:30 [Info] Indexer::handleRecoverIndex InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:30:59.700+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 672.906µs 2023-10-21T04:30:59.701+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 887.895µs 2023-10-21T04:30:59.701+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3464125101225762400 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[16787445029868804812 2925063509317026816]] 2023-10-21T04:30:59.701+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3464125101225762400, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:30:59.709+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 77.870501ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:30:59.701+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3464125101225762400 PartitionId: 0 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:30:59.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [77.773363ms] 2023-10-21T04:30:59.708+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.712+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:30:59.713+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.713+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:30:59.719+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:30:59.719+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.726+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.730+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.730+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.730+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.730+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.731+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.731+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.731+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.732+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.732+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.732+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.732+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.732+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.752+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.751+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:59.751+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner initialized 2023-10-21T04:30:59.752+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:59.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:59.753+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.753+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true]. Built [18] plasmas, took [124.565746ms] 2023-10-21T04:30:59.753+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:30:59.753+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 127 metaSz: 2518 rp [1] sn: 16 count: 127 metaSz: 2520 2023-10-21T04:30:59.753+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 51127 recovery UsedSpace 58084 2023-10-21T04:30:59.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:59.759+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.760+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:30:59.760+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.760+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:30:59.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.761+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:30:59.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.761+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:30:59.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.761+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:30:59.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.762+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:30:59.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.762+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:30:59.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.763+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:30:59.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : replayOffset: 638976 (RP:638976) (CP:360448) 2023-10-21T04:30:59.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [638976] took [135.939153ms] 2023-10-21T04:30:59.796+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9997, memSzIndex 77, numRecords 131 2023-10-21T04:30:59.796+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.797+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053288, resident 1, memSz 8667, memSzIndex 0, numRecords 112 2023-10-21T04:30:59.797+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCleaner initialized for recovery 2023-10-21T04:30:59.797+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:30:59.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery 2023-10-21T04:30:59.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:30:59.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true] 2023-10-21T04:30:59.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[876544] Data log Head [0] Tail [921600] 2023-10-21T04:30:59.798+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [876544] 2023-10-21T04:30:59.799+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.799+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:30:59.800+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.800+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:30:59.800+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.800+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:30:59.801+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.801+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:30:59.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:30:59.767+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:30:59.770+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf9:5d:ac:9c:32:8:f6:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13981755107353251708 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7140146642632649428 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.777+05:30 [Info] switched currmeta from 25 -> 27 force true 2023-10-21T04:30:59.797+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9834, memSzIndex 0, numRecords 123 2023-10-21T04:30:59.802+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.802+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.802+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.802+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.802+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.802+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.803+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.803+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.803+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.803+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9853, memSzIndex 0, numRecords 123 2023-10-21T04:30:59.803+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.803+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053288, resident 1, memSz 10687, memSzIndex 75, numRecords 138 2023-10-21T04:30:59.803+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.804+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9511, memSzIndex 76, numRecords 127 2023-10-21T04:30:59.805+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:1 8806f5706747af4e635b8da5fcd8aeee:1 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:59.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:30:59.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:30:59.807+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper started 2023-10-21T04:30:59.807+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:30:59.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex started 2023-10-21T04:30:59.807+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:59.805+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.807+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:30:59.808+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.808+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:30:59.808+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.809+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:30:59.809+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.813+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:30:59.809+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:59.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.814+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.815+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:30:59.817+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.815+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9669, memSzIndex 76, numRecords 128 2023-10-21T04:30:59.844+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2f:a7:bd:b5:1a:16:61:f6 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1684463754565405483 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 5235575068708680783 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.852+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:0 8806f5706747af4e635b8da5fcd8aeee:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:59.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9287, memSzIndex 0, numRecords 118 2023-10-21T04:30:59.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: request quota after header replay 1056128, resident 1, memSz 67959, memSzIndex 841, numRecords 1000 2023-10-21T04:30:59.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 44753651 controller mem 4750 2023-10-21T04:30:59.852+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [651264] replayOffset [638976] 2023-10-21T04:30:59.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [225.759038ms] 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true]. Built [17] plasmas, took [226.673283ms] 2023-10-21T04:30:59.853+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:30:59.854+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 127 metaSz: 2514 rp [1] sn: 15 count: 127 metaSz: 2518 rp [2] sn: 16 count: 127 metaSz: 2520 2023-10-21T04:30:59.854+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 45996 recovery UsedSpace 50224 2023-10-21T04:30:59.855+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:30:59.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:30:59.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:30:59.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:30:59.858+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:30:59.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex started 2023-10-21T04:30:59.858+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:30:59.861+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:59.861+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:59.874+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.875+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:30:59.877+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.886+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:30:59.887+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.888+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:30:59.891+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.891+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:30:59.895+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.896+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:30:59.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.898+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:30:59.901+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.901+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:30:59.900+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Warmup took 277.208081ms 2023-10-21T04:30:59.902+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:30:59.907+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:30:59.902+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 6 WriterThreads 1 2023-10-21T04:30:59.907+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index Index: 8965923595543532942 Partition: 6 2023-10-21T04:30:59.907+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:30:59.908+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 7 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:30:59.908+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf9:5d:ac:9c:32:8:f6:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13981755107353251708 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7140146642632649428 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:30:59.908+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:0 8806f5706747af4e635b8da5fcd8aeee:1 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:30:59.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:30:59.908+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:30:59.908+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 131 metaSz: 2514 rp [1] sn: 15 count: 131 metaSz: 2518 rp [2] sn: 16 count: 131 metaSz: 2520 2023-10-21T04:30:59.908+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 47956 recovery UsedSpace 50324 2023-10-21T04:30:59.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:30:59.909+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:30:59.909+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 131 metaSz: 2518 rp [1] sn: 16 count: 131 metaSz: 2520 2023-10-21T04:30:59.909+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : replayOffset: 913408 (RP:905216) (CP:913408) 2023-10-21T04:30:59.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [913408] took [112.481111ms] 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12169, memSzIndex 0, numRecords 122 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13565, memSzIndex 0, numRecords 136 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057426, resident 1, memSz 10600, memSzIndex 0, numRecords 106 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13546, memSzIndex 0, numRecords 136 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12491, memSzIndex 0, numRecords 125 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1055988, resident 1, memSz 70520, memSzIndex 401, numRecords 1000 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:30:59.912+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1055176, resident 0.12, memSz 347, memSzIndex 0, numRecords 8 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12489, memSzIndex 0, numRecords 125 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12281, memSzIndex 0, numRecords 123 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12657, memSzIndex 0, numRecords 127 2023-10-21T04:30:59.913+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27934973 controller mem 1490 2023-10-21T04:30:59.913+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [921600] replayOffset [913408] 2023-10-21T04:30:59.911+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 51541 recovery UsedSpace 58084 2023-10-21T04:30:59.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [116.562415ms] 2023-10-21T04:30:59.914+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.915+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.915+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.916+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.916+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.916+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.916+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:30:59.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:30:59.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:30:59.918+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:30:59.918+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex started 2023-10-21T04:30:59.918+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.918+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:59.918+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:30:59.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true]. Built [18] plasmas, took [120.552847ms] 2023-10-21T04:30:59.918+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:30:59.918+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 8 count: 5 metaSz: 2481 rp [1] sn: 360006 count: 6 metaSz: 2481 2023-10-21T04:30:59.918+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: doInit: data UsedSpace 50688 recovery UsedSpace 74828 2023-10-21T04:30:59.918+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:59.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:30:59.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:30:59.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:30:59.935+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:30:59.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex started 2023-10-21T04:30:59.935+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:30:59.935+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:59.935+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:59.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:30:59.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:30:59.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:30:59.940+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:30:59.941+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:30:59.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex started 2023-10-21T04:30:59.942+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:30:59.944+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:30:59.944+05:30 [Info] bucket_1/#primary/Mainstore#3464125101225762400:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:30:59.992+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:30:59.992+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:30:59.994+05:30 [Info] switched currmeta from 27 -> 28 force true 2023-10-21T04:30:59.999+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Warmup took 91.60981ms 2023-10-21T04:31:00.001+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 7 WriterThreads 1 2023-10-21T04:31:00.001+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index Index: 8965923595543532942 Partition: 7 2023-10-21T04:31:00.002+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.004+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 8 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.006+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:00.007+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 112 metaSz: 2518 rp [1] sn: 16 count: 112 metaSz: 2520 2023-10-21T04:31:00.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.009+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:31:00.009+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 112 metaSz: 2514 rp [1] sn: 15 count: 112 metaSz: 2518 rp [2] sn: 16 count: 112 metaSz: 2520 2023-10-21T04:31:00.021+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Warmup took 309.304991ms 2023-10-21T04:31:00.021+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3464125101225762400 partitionId 0 WriterThreads 4 2023-10-21T04:31:00.021+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3464125101225762400 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index Index: 3464125101225762400 Partition: 0 2023-10-21T04:31:00.022+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3464125101225762400, shardIdMap: map[0:[16787445029868804812]] err: 2023-10-21T04:31:00.022+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3464125101225762400, partnShardIdMap: map[0:[16787445029868804812]], err: 2023-10-21T04:31:00.022+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:00.023+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3464125101225762400 2023-10-21T04:31:00.023+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:00.023+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:31:00.024+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_1:1697842860022393050] 2023-10-21T04:31:00.024+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:00.024+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:00.024+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3464125101225762400 Partitions [0] 2023-10-21T04:31:00.027+05:30 [Info] StorageMgr::openSnapshot IndexInst:3464125101225762400 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:31:00.027+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 52039 recovery UsedSpace 58084 2023-10-21T04:31:00.028+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 46951 recovery UsedSpace 49870 2023-10-21T04:31:00.043+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.427729ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:00.043+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 92.662µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:00.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:31:00.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.061+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:31:00.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex started 2023-10-21T04:31:00.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:31:00.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.073+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:31:00.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex started 2023-10-21T04:31:00.073+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:31:00.073+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.073+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.075+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:00.075+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:00.080+05:30 [Info] switched currmeta from 28 -> 29 force true 2023-10-21T04:31:00.091+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:00.092+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:00.105+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Warmup took 101.542794ms 2023-10-21T04:31:00.106+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 8 WriterThreads 1 2023-10-21T04:31:00.106+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index Index: 8965923595543532942 Partition: 8 2023-10-21T04:31:00.106+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.107+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 1 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.107+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:31:00.108+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 13 count: 128 metaSz: 2514 rp [1] sn: 14 count: 128 metaSz: 2518 rp [2] sn: 15 count: 128 metaSz: 2520 2023-10-21T04:31:00.108+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 50963 recovery UsedSpace 50318 2023-10-21T04:31:00.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.110+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:00.110+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 14 count: 128 metaSz: 2518 rp [1] sn: 15 count: 128 metaSz: 2520 2023-10-21T04:31:00.111+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 54992 recovery UsedSpace 58084 2023-10-21T04:31:00.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:31:00.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.116+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:31:00.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex started 2023-10-21T04:31:00.116+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.116+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.117+05:30 [Info] switched currmeta from 29 -> 29 force true 2023-10-21T04:31:00.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:31:00.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.152+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:31:00.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex started 2023-10-21T04:31:00.158+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.159+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.157+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:31:00.192+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:00.193+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:00.197+05:30 [Info] switched currmeta from 29 -> 30 force true 2023-10-21T04:31:00.217+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Warmup took 109.54854ms 2023-10-21T04:31:00.217+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 1 WriterThreads 1 2023-10-21T04:31:00.217+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index Index: 8965923595543532942 Partition: 1 2023-10-21T04:31:00.217+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.218+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 2 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.219+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:00.219+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 123 metaSz: 2517 rp [1] sn: 16 count: 123 metaSz: 2520 2023-10-21T04:31:00.219+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 54759 recovery UsedSpace 58084 2023-10-21T04:31:00.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.225+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:31:00.225+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 123 metaSz: 2514 rp [1] sn: 15 count: 123 metaSz: 2517 rp [2] sn: 16 count: 123 metaSz: 2520 2023-10-21T04:31:00.225+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 51093 recovery UsedSpace 49870 2023-10-21T04:31:00.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:31:00.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.226+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:31:00.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex started 2023-10-21T04:31:00.226+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.226+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:31:00.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.246+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:31:00.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex started 2023-10-21T04:31:00.247+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:31:00.247+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.247+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.252+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2f:a7:bd:b5:1a:16:61:f6 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1684463754565405483 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 5235575068708680783 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:00.253+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:0 8806f5706747af4e635b8da5fcd8aeee:0 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:31:00.278+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Warmup took 59.295302ms 2023-10-21T04:31:00.278+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 2 WriterThreads 1 2023-10-21T04:31:00.278+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index Index: 8965923595543532942 Partition: 2 2023-10-21T04:31:00.279+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.279+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 3 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.280+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:00.280+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 14 count: 123 metaSz: 2517 rp [1] sn: 15 count: 123 metaSz: 2520 2023-10-21T04:31:00.281+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 56888 recovery UsedSpace 58084 2023-10-21T04:31:00.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.282+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:31:00.282+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 13 count: 123 metaSz: 2514 rp [1] sn: 14 count: 123 metaSz: 2517 rp [2] sn: 15 count: 123 metaSz: 2520 2023-10-21T04:31:00.283+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 52785 recovery UsedSpace 49870 2023-10-21T04:31:00.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:31:00.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.301+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:31:00.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex started 2023-10-21T04:31:00.302+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.302+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:31:00.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.307+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:31:00.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex started 2023-10-21T04:31:00.308+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:31:00.308+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.308+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.332+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:00.332+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:00.337+05:30 [Info] switched currmeta from 30 -> 31 force true 2023-10-21T04:31:00.361+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Warmup took 81.260672ms 2023-10-21T04:31:00.361+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 3 WriterThreads 1 2023-10-21T04:31:00.361+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index Index: 8965923595543532942 Partition: 3 2023-10-21T04:31:00.362+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.363+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 4 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:00.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 118 metaSz: 2517 rp [1] sn: 16 count: 118 metaSz: 2520 2023-10-21T04:31:00.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 58779 recovery UsedSpace 58084 2023-10-21T04:31:00.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.365+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:31:00.365+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 118 metaSz: 2514 rp [1] sn: 15 count: 118 metaSz: 2517 rp [2] sn: 16 count: 118 metaSz: 2520 2023-10-21T04:31:00.365+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 52414 recovery UsedSpace 49870 2023-10-21T04:31:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:31:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.371+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:31:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex started 2023-10-21T04:31:00.371+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.371+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:31:00.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.372+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:31:00.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex started 2023-10-21T04:31:00.372+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:31:00.374+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.374+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.387+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Warmup took 23.313273ms 2023-10-21T04:31:00.387+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 4 WriterThreads 1 2023-10-21T04:31:00.387+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index Index: 8965923595543532942 Partition: 4 2023-10-21T04:31:00.387+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.388+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 5 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.388+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 238 2023-10-21T04:31:00.388+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 138 metaSz: 2514 rp [1] sn: 15 count: 138 metaSz: 2518 rp [2] sn: 16 count: 138 metaSz: 2520 2023-10-21T04:31:00.388+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 57377 recovery UsedSpace 50312 2023-10-21T04:31:00.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.389+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:00.389+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 138 metaSz: 2514 rp [1] sn: 15 count: 138 metaSz: 2518 rp [2] sn: 16 count: 138 metaSz: 2520 2023-10-21T04:31:00.389+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 60100 recovery UsedSpace 49870 2023-10-21T04:31:00.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:31:00.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.397+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:31:00.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex started 2023-10-21T04:31:00.398+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.398+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:31:00.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.400+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:31:00.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex started 2023-10-21T04:31:00.400+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:31:00.401+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.401+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.443+05:30 [Error] ScanCoordinator: Unable to compute index items_count for bucket_%2/idx_partitioned/8965923595543532942 state 0 (Index not found) 2023-10-21T04:31:00.455+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Warmup took 67.441899ms 2023-10-21T04:31:00.455+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 5 WriterThreads 1 2023-10-21T04:31:00.455+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index Index: 8965923595543532942 Partition: 5 2023-10-21T04:31:00.455+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8965923595543532942, shardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] err: 2023-10-21T04:31:00.456+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8965923595543532942, partnShardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]], err: 2023-10-21T04:31:00.456+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:00.456+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8965923595543532942 2023-10-21T04:31:00.456+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:00.456+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:31:00.456+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842860456046420 bucket_1:1697842860022393050] 2023-10-21T04:31:00.456+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:00.456+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:00.457+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8965923595543532942 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:31:00.458+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:1 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:31:00.458+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:2 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:31:00.459+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:3 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:31:00.466+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:4 Attempting to open snapshot (SnapshotInfo: count:118 committed:false) 2023-10-21T04:31:00.472+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.034328ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:00.472+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 109.977µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:00.480+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:5 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:31:00.483+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:00.483+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:00.485+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:31:00.487+05:30 [Info] switched currmeta from 31 -> 32 force true 2023-10-21T04:31:00.490+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:7 Attempting to open snapshot (SnapshotInfo: count:131 committed:false) 2023-10-21T04:31:00.504+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:8 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:31:00.651+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8965923595543532942:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:00.652+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 10606939961365971246, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:00.652+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 10606939961365971246, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:00.671+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8732144615635911668,realInstId: 0, indexDefn DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:00.672+05:30 [Info] Indexer::handleRecoverIndex InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:00.672+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 489.6µs 2023-10-21T04:31:00.672+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 538.327µs 2023-10-21T04:31:00.672+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.673+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 1 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:00.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 31507 recovery UsedSpace 8236 2023-10-21T04:31:00.672+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8732144615635911668, indexDefn DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:00.673+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 21.95632ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:00.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.675+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:00.675+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:00.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:00.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.681+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:00.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:31:00.681+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.681+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:00.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.683+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:00.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:31:00.683+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:31:00.683+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:00.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:00.683+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:00.683+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:00.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:00.683+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:00.685+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:00.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:00.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:00.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:00.696+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:00.696+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:00.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:00.713+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:00.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:00.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:31:00.713+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:00.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:00.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:00.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:00.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed 2023-10-21T04:31:00.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:00.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:00.722+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:00.722+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:00.727+05:30 [Info] switched currmeta from 32 -> 33 force true 2023-10-21T04:31:00.727+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:00.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:00.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:31:00.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:00.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.739+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:00.739+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:00.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:00.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.747+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:00.747+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:00.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:00.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.751+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:00.751+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:00.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:31:00.751+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.751+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:00.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.756+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:00.756+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:00.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:31:00.756+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Warmup took 83.250885ms 2023-10-21T04:31:00.756+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 1 WriterThreads 1 2023-10-21T04:31:00.756+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Stopped Billing WUs 2023-10-21T04:31:00.756+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index Index: 8732144615635911668 Partition: 1 2023-10-21T04:31:00.756+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:00.757+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 2 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:00.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:00.758+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:00.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:00.758+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:00.758+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.758+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.759+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 31507 recovery UsedSpace 8236 2023-10-21T04:31:00.761+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:00.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:00.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:00.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:00.768+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:00.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:31:00.769+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.769+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:00.770+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:31:00.770+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:31:00.770+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:00.770+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:00.770+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:00.770+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:00.772+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:00.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:00.772+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:00.772+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:00.772+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:00.773+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:00.773+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:00.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:31:00.773+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:00.774+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:00.774+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:00.775+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:00.787+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:00.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:00.788+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:31:00.789+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:00.789+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:00.790+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:00.791+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:00.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:00.791+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:00.792+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:00.792+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:00.792+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:00.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:31:00.793+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:00.793+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:00.793+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:00.794+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:00.808+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3464125101225762400:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:00.808+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [11276338709617092948] belonging to collectionId: 9, ttid: ShardToken17:ba:f0:81:68:83:f1:eb. Initiating build 2023-10-21T04:31:00.828+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:00.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:00.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:31:00.829+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:00.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:00.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:00.830+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [3464125101225762400] &{2} 2023-10-21T04:31:00.830+05:30 [Info] Indexer::handleBuildRecoveredIndexes [3464125101225762400] 2023-10-21T04:31:00.830+05:30 [Info] bulkUpdateRState: Index instance 3464125101225762400 rstate moved to PENDING 2023-10-21T04:31:00.830+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [3464125101225762400] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:31:00.831+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:00.831+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842860456046420 bucket_1:1697842860022393050] 2023-10-21T04:31:00.831+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:00.833+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:31:00.834+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:00.834+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:00.834+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:31:00.834+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:31:00.834+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 3979 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:31:00.834+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3464125101225762400 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:31:00.834+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:31:00.835+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=23,1=24,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=27,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=20,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=22,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 3979 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:31:00.835+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:31:00.835+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:31:00.836+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:00.836+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 5.47018ms 2023-10-21T04:31:00.836+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 5.510014ms 2023-10-21T04:31:00.836+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [3464125101225762400] 2023-10-21T04:31:00.836+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 25.100333ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:00.838+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[3464125101225762400:true] 2023-10-21T04:31:00.845+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:31:00.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:00.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:00.857+05:30 [Info] switched currmeta from 33 -> 34 force true 2023-10-21T04:31:00.860+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:00.861+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken17:ba:f0:81:68:83:f1:eb panic: TestCode::TestActionAtTag - Inducing artificial panic at tag: 6 as wished goroutine 1103 [running]: github.com/couchbase/indexing/secondary/testcode.TestActionAtTag(0xc007386e08?, 0x6) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/testcode/test_action_ci.go:45 +0x5e5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).waitForIndexState(0xc006ea8000, 0x4, 0xc008187920, {0xc007383854, 0x21}, 0xc0073b3c00) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:2271 +0x58f github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).startShardRecovery(0xc006ea8000, {0xc007383854, 0x21}, 0xc0073b3c00) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1926 +0x1478 created by github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferTokenAsDest /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:1431 +0x28d 2023-10-21T04:31:02.308+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=18194302e3f10df59bd076e1c4142d0b -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:31:02.308+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:31:02.308+05:30 [Info] Setting ipv6=false 2023-10-21T04:31:02.328+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:31:02.351+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:31:02.362+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:31:02.362+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:31:02.362+05:30 [Info] Setting log level to Info 2023-10-21T04:31:02.362+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:31:02.362+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:31:02.363+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:31:02.381+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:31:02.414+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:31:02 audit: created new audit service 2023-10-21T04:31:02.838+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:31:02.838+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:31:02.838+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:31:02.839+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:31:02.839+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:31:02.839+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:31:02.867+05:30 [Info] security context initialized 2023-10-21T04:31:02.867+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:31:02.868+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:31:02.868+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:31:02.868+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:31:02.868+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:31:02.873+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:31:02.873+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:31:02.873+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:31:02.875+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:31:02.879+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:31:02.879+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:31:02.880+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:31:02.881+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:31:02.885+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:31:02.896+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:31:02.896+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:31:02.896+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:02.896+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:31:02.916+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:31:02.916+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:31:02.917+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:31:02.919+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:31:02.919+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:31:02.921+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:31:02.922+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:31:02.922+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:31:02.922+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:31:02.923+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:31:02.923+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 248 us 2023-10-21T04:31:02.925+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:31:02.930+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:31:02.931+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:31:02.947+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:31:02.947+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:31:02.947+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:31:02.947+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:31:02.947+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:31:02.947+05:30 [Info] Indexer Id 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:31:02.947+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:31:02.947+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:31:02.949+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:31:02.950+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:31:02.951+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 211] updating cluster rev epoch 0 -> 1 2023-10-21T04:31:02.952+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 211] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:31:02.960+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 185] bucketNames has been updated (was map[], now map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43]) 2023-10-21T04:31:02.961+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 229] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:31:02.961+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 230] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:31:02.966+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 185] updated cluster rebalancing status to false 2023-10-21T04:31:02.966+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:31:02.966+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:31:02.966+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:31:02.971+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:31:02.972+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:31:02.972+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:31:02.972+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:31:02.972+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:31:02.973+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning true RebalanceToken &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:31:02.973+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:31:02.973+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:31:02.974+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:31:02.989+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:31:02.989+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:31:02.990+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:31:02.993+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:31:02.993+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:31:03.002+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 8732144615635911668 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 , elapsed: 28.070065ms 2023-10-21T04:31:03.002+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:31:03.002+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:31:03.002+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:31:03.002+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager true &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:31:03.002+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:31:03.003+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:31:03.003+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:31:03.003+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:31:03.005+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:31:03.005+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:31:03.005+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:31:03.005+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:31:03.006+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:31:03.006+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:31:03.006+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:31:03.006+05:30 [Warn] Indexer::validateIndexInstMap State INDEX_STATE_CREATED Stream NIL_STREAM Deferred true Found. Updating State to Ready InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:03.006+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:03.007+05:30 [Warn] Indexer::validateIndexInstMap State INDEX_STATE_RECOVERED Not Recoverable. Not Recovering Index InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] [16787445029868804812]}] 1 SINGLE 0} 2023-10-21T04:31:03.007+05:30 [Info] ClustMgr:handleCleanupIndex mType: CLUST_MGR_CLEANUP_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:03.007+05:30 [Warn] Indexer::validateIndexInstMap State INDEX_STATE_RECOVERED Stream NIL_STREAM Deferred false Not Valid For Recovery. Cleanup Index InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] [17406544375239214575 10170029854712240877]} 2:{2 1 [:9123] [17406544375239214575 10170029854712240877]} 3:{3 1 [:9123] [17406544375239214575 10170029854712240877]} 4:{4 1 [:9123] [17406544375239214575 10170029854712240877]} 5:{5 1 [:9123] [17406544375239214575 10170029854712240877]} 6:{6 1 [:9123] [17406544375239214575 10170029854712240877]} 7:{7 1 [:9123] [17406544375239214575 10170029854712240877]} 8:{8 1 [:9123] [17406544375239214575 10170029854712240877]}] 8 KEY 0} 2023-10-21T04:31:03.008+05:30 [Info] ClustMgr:handleCleanupIndex mType: CLUST_MGR_CLEANUP_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:03.008+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:03.008+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index. Scheduling it for cleanup. 2023-10-21T04:31:03.009+05:30 [Info] Found orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index. Scheduling it for cleanup. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.009+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:31:03.010+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:31:03.010+05:30 [Info] : Recover shard shard10170029854712240877 from metadata completed. 2023-10-21T04:31:03.010+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:31:03.010+05:30 [Info] : Recover shard shard16787445029868804812 from metadata completed. 2023-10-21T04:31:03.011+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:31:03.011+05:30 [Info] : Recover shard shard17406544375239214575 from metadata completed. 2023-10-21T04:31:03.011+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:31:03.011+05:30 [Info] : Recover shard shard2925063509317026816 from metadata completed. 2023-10-21T04:31:03.011+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex ... 2023-10-21T04:31:03.011+05:30 [Info] : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex ... 2023-10-21T04:31:03.014+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.014+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.015+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex ... 2023-10-21T04:31:03.015+05:30 [Info] : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex ... 2023-10-21T04:31:03.018+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:31:03.019+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:31:03.021+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.315831ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:03.021+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 11276338709617092948 instance id 3464125101225762400 2023-10-21T04:31:03.021+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 11276338709617092948 2023-10-21T04:31:03.022+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:31:03.022+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:false Messages:[]} 2023-10-21T04:31:03.025+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.025+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index destroyed 2023-10-21T04:31:03.025+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.025+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex ... 2023-10-21T04:31:03.025+05:30 [Info] : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex ... 2023-10-21T04:31:03.026+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:31:03.026+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:31:03.030+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.031+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.031+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex ... 2023-10-21T04:31:03.031+05:30 [Info] : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex ... 2023-10-21T04:31:03.035+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.035+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index destroyed 2023-10-21T04:31:03.035+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.035+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex ... 2023-10-21T04:31:03.035+05:30 [Info] : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex ... 2023-10-21T04:31:03.038+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.038+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.038+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex ... 2023-10-21T04:31:03.038+05:30 [Info] : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex ... 2023-10-21T04:31:03.048+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLEANUP_INDEX elapsed 27.582715ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:03.049+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 8910459001587546151 instance id 8965923595543532942 2023-10-21T04:31:03.049+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 8910459001587546151 2023-10-21T04:31:03.050+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.050+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index destroyed 2023-10-21T04:31:03.050+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.050+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex ... 2023-10-21T04:31:03.050+05:30 [Info] : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex ... 2023-10-21T04:31:03.058+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.058+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.058+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex ... 2023-10-21T04:31:03.058+05:30 [Info] : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex ... 2023-10-21T04:31:03.061+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:31:03.062+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.062+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index destroyed 2023-10-21T04:31:03.062+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.062+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex ... 2023-10-21T04:31:03.062+05:30 [Info] : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex ... 2023-10-21T04:31:03.065+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.065+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.065+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex ... 2023-10-21T04:31:03.065+05:30 [Info] : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex ... 2023-10-21T04:31:03.070+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.070+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index destroyed 2023-10-21T04:31:03.070+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.070+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex ... 2023-10-21T04:31:03.070+05:30 [Info] : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex ... 2023-10-21T04:31:03.074+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.074+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.074+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex ... 2023-10-21T04:31:03.074+05:30 [Info] : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex ... 2023-10-21T04:31:03.075+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLEANUP_INDEX elapsed 26.765146ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:03.079+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.079+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index destroyed 2023-10-21T04:31:03.079+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.079+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:31:03.079+05:30 [Info] : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:31:03.086+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.086+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.086+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:31:03.086+05:30 [Info] : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:31:03.095+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.095+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index destroyed 2023-10-21T04:31:03.095+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.095+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex ... 2023-10-21T04:31:03.095+05:30 [Info] : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex ... 2023-10-21T04:31:03.102+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.102+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.102+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex ... 2023-10-21T04:31:03.102+05:30 [Info] : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex ... 2023-10-21T04:31:03.107+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.107+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index destroyed 2023-10-21T04:31:03.107+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index. Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.107+05:30 [Info] : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:31:03.107+05:30 [Info] : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:31:03.109+05:30 [Info] : metadata saved successfully 2023-10-21T04:31:03.109+05:30 [Info] : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index destroyed 2023-10-21T04:31:03.109+05:30 [Info] Cleaned up the orphan index slice /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index. 2023-10-21T04:31:03.111+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:31:03.112+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:31:03.112+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.113+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': added 2023-10-21T04:31:03.113+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:31:03.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner initialized 2023-10-21T04:31:03.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:03.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner initialized 2023-10-21T04:31:03.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:03.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCleaner initialized for recovery 2023-10-21T04:31:03.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:03.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery 2023-10-21T04:31:03.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true] 2023-10-21T04:31:03.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[892928] Data log Head [0] Tail [942080] 2023-10-21T04:31:03.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [892928] 2023-10-21T04:31:03.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCleaner initialized for recovery 2023-10-21T04:31:03.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:03.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery 2023-10-21T04:31:03.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true] 2023-10-21T04:31:03.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[778240] Data log Head [0] Tail [835584] 2023-10-21T04:31:03.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [778240] 2023-10-21T04:31:03.119+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.120+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:31:03.120+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:31:03.120+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.120+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:31:03.121+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.121+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:31:03.121+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.121+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.122+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.122+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.122+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.122+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.123+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.122+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.123+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.123+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.123+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.123+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.123+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.124+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.124+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.124+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.124+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.124+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.124+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.125+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.125+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : replayOffset: 831488 (RP:831488) (CP:294912) 2023-10-21T04:31:03.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [831488] took [7.210362ms] 2023-10-21T04:31:03.125+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 22629399 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 23677975 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 23678045 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 23678045 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 23678045 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1056128, resident 1, memSz 67959, memSzIndex 841, numRecords 1000 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 24726621 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 24726621 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 24726621 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 24726621 controller mem 3118 2023-10-21T04:31:03.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [835584] replayOffset [831488] 2023-10-21T04:31:03.125+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.126+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.126+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.126+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.126+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : replayOffset: 937984 (RP:937984) (CP:323584) 2023-10-21T04:31:03.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [937984] took [9.470772ms] 2023-10-21T04:31:03.126+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.126+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.126+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.126+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1055176, resident 1, memSz 938, memSzIndex 0, numRecords 8 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1055942, resident 1, memSz 70473, memSzIndex 354, numRecords 1000 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920925 controller mem 3118 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920995 controller mem 3182 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:03.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 28920995 controller mem 3182 2023-10-21T04:31:03.127+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [942080] replayOffset [937984] 2023-10-21T04:31:03.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [10.720995ms] 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true]. Built [10] plasmas, took [11.524641ms] 2023-10-21T04:31:03.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [11.509972ms] 2023-10-21T04:31:03.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:03.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true]. Built [9] plasmas, took [12.134534ms] 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.140+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper started 2023-10-21T04:31:03.140+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:31:03.140+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.140+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:31:03.143+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:31:03.143+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:03.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:03.143+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.144+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.144+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:03.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:31:03.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:03.150+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.150+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:03.241+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:03.241+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:31:03.241+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.241+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.241+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.241+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.249+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:03.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:31:03.249+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:03.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:03.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:31:03.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:31:03.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:31:03.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.261+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.261+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.265+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.266+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.268+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.268+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:03.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:31:03.268+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.268+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.271+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.271+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:03.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:31:03.271+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.271+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Warmup took 158.674276ms 2023-10-21T04:31:03.272+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 6 WriterThreads 1 2023-10-21T04:31:03.272+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index Index: 8732144615635911668 Partition: 6 2023-10-21T04:31:03.272+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.271+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.273+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.273+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.273+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.274+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.280+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:31:03.280+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.280+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.282+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:31:03.282+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:31:03.282+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:03.282+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:03.282+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:03.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:03.290+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:03.290+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.306+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 52] will report aggregate recorder stats every 5m0s 2023-10-21T04:31:03.317+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:03.320+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:03.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:31:03.321+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:03.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:03.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:31:03.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:31:03.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:31:03.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.340+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.340+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.346+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.346+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.349+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.349+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:03.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:31:03.349+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.349+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:03.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:31:03.358+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Warmup took 85.104267ms 2023-10-21T04:31:03.358+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.358+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.358+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 7 WriterThreads 1 2023-10-21T04:31:03.358+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index Index: 8732144615635911668 Partition: 7 2023-10-21T04:31:03.358+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.359+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.359+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.360+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.360+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.372+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:31:03.372+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.372+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.373+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.373+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:31:03.373+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:03.373+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:03.373+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.373+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:03.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:03.378+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:03.378+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.378+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.383+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:03.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:31:03.383+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:03.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:03.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:03.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:03.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:31:03.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.393+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.393+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.397+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.397+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.399+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.399+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:03.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:31:03.399+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.399+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.402+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.402+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:03.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:31:03.402+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Warmup took 43.479577ms 2023-10-21T04:31:03.402+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.402+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.402+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 8 WriterThreads 1 2023-10-21T04:31:03.402+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index Index: 8732144615635911668 Partition: 8 2023-10-21T04:31:03.403+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.403+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.410+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.410+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:31:03.410+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.410+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:03.410+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:03.410+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:31:03.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:03.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:31:03.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:03.417+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.417+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:03.418+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:03.418+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:31:03.418+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.418+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.418+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.418+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.430+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:03.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:31:03.430+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:03.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:03.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:03.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:03.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:31:03.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.476+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.477+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.489+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.489+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.497+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.497+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:03.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:31:03.499+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.499+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.506+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.506+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:03.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:31:03.506+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Warmup took 102.928483ms 2023-10-21T04:31:03.506+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 1 WriterThreads 1 2023-10-21T04:31:03.506+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index Index: 8732144615635911668 Partition: 1 2023-10-21T04:31:03.506+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:31:03.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74636 recovery UsedSpace 16530 2023-10-21T04:31:03.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:31:03.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74636 recovery UsedSpace 16530 2023-10-21T04:31:03.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.530+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:31:03.530+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.530+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:31:03.530+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:31:03.530+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.530+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.530+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.530+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:03.531+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:03.531+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.530+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.531+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:03.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:03.536+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:03.536+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:03.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:31:03.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:03.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:03.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:03.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:03.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:31:03.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.558+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.558+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.558+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.564+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.564+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.571+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.571+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:03.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:31:03.571+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.571+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.580+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.580+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:03.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:31:03.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.580+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Warmup took 73.725006ms 2023-10-21T04:31:03.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.580+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 2 WriterThreads 1 2023-10-21T04:31:03.580+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index Index: 8732144615635911668 Partition: 2 2023-10-21T04:31:03.580+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.581+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.588+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:31:03.588+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:31:03.588+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.589+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.589+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.589+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:03.589+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:03.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:03.600+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:03.600+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.601+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.601+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.612+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:03.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:31:03.613+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:03.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:03.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:03.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:03.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:31:03.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.626+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.626+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.629+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.629+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.632+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.633+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:03.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:31:03.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.635+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.635+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:03.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:31:03.635+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Warmup took 54.23291ms 2023-10-21T04:31:03.635+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 3 WriterThreads 1 2023-10-21T04:31:03.635+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index Index: 8732144615635911668 Partition: 3 2023-10-21T04:31:03.635+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.636+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.636+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:31:03.636+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.636+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.637+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.637+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:31:03.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.648+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:31:03.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.649+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:31:03.649+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:31:03.649+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:03.649+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:03.649+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.649+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.654+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:03.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:31:03.654+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.654+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:03.655+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:03.655+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.655+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.660+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:03.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:31:03.661+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:03.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:03.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:03.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:03.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:31:03.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.671+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.671+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.677+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.677+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.679+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.679+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:03.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:31:03.679+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.679+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.684+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.684+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:03.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:31:03.684+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Warmup took 48.93944ms 2023-10-21T04:31:03.685+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.685+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.685+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 4 WriterThreads 1 2023-10-21T04:31:03.685+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index Index: 8732144615635911668 Partition: 4 2023-10-21T04:31:03.685+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:31:03.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.685+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.686+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:03.686+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 4118 2023-10-21T04:31:03.686+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 4118 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.695+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:31:03.695+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.695+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.695+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:31:03.695+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:31:03.695+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:03.695+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:03.696+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:03.696+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.696+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:03.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:03.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:03.708+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:03.708+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:31:03.708+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:03.709+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:03.709+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:03.709+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:03.721+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:03.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:03.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:31:03.721+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:03.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:03.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:03.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:31:03.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:03.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:03.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:31:03.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:03.736+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.736+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:03.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:03.745+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:03.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:03.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:03.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:03.752+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.752+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:03.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:31:03.752+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.752+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:03.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:03.755+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.755+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:03.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:31:03.756+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Warmup took 70.386722ms 2023-10-21T04:31:03.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:03.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:03.756+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 5 WriterThreads 1 2023-10-21T04:31:03.756+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index Index: 8732144615635911668 Partition: 5 2023-10-21T04:31:03.756+05:30 [Warn] Indexer::updateTopologyOnShardIdChange. Persisted shardIds: map[1:[] 2:[] 3:[] 4:[] 5:[] 6:[] 7:[] 8:[]] are different from currShardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] for index inst: 8732144615635911668. Updating topology with new shardIds 2023-10-21T04:31:03.756+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005dc9b00 2023-10-21T04:31:03.757+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8732144615635911668 2023-10-21T04:31:03.757+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Partitions [4 5 6 7 8 1 2 3] 2023-10-21T04:31:03.757+05:30 [Info] StorageMgr::openSnapshot IndexInst:8732144615635911668 Partition:4 No Snapshot Found. 2023-10-21T04:31:03.757+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Adding Nil Snapshot. 2023-10-21T04:31:03.757+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00010e840 ScanCoordinator {8732144615635911668 {10606939961365971246 idx_partitioned_defer plasma bucket_%2 b9538b29d7e025c819e734b646db039c false [`balance`] N1QL KEY [false] true true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 0 1 0 0xc005dbff50 [] 1 0 false plasma 0} map[1:{{1 1 [:9123] []} 0xc005077870} 2:{{2 1 [:9123] []} 0xc005bf4260} 3:{{3 1 [:9123] []} 0xc005bf4620} 4:{{4 1 [:9123] []} 0xc005d1d650} 5:{{5 1 [:9123] []} 0xc005c3e940} 6:{{6 1 [:9123] []} 0xc0050811e0} 7:{{7 1 [:9123] []} 0xc004f31960} 8:{{8 1 [:9123] []} 0xc005a1d080}] 0xc0050b3200 0xc005dc9c80} 2023-10-21T04:31:03.759+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:31:03.759+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:03.759+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:31:03.759+05:30 [Info] RebalanceServiceManager::recoverRebalance: Init Pending Cleanup 2023-10-21T04:31:03.759+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:31:03.759+05:30 [Info] No instances in shard being recovered. Destroy shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:03.759+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:31:03.759+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:31:03.759+05:30 [Info] : All instance closed 2023-10-21T04:31:03.759+05:30 [Info] : All instance destroyed 2023-10-21T04:31:03.760+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:31:03.761+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:31:03.761+05:30 [Info] No instances in shard being recovered. Destroy shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:03.761+05:30 [Info] : shard removed from ShardMgr 2023-10-21T04:31:03.761+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:31:03.761+05:30 [Info] : All instance closed 2023-10-21T04:31:03.761+05:30 [Info] : All instance destroyed 2023-10-21T04:31:03.762+05:30 [Info] : Shard Destroyed Successfully 2023-10-21T04:31:03.762+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:31:03.762+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : all deamons started 2023-10-21T04:31:03.762+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : all deamons started 2023-10-21T04:31:03.762+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:31:03.762+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:31:03.762+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : all deamons started 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : all deamons started 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:31:03.763+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:31:03.766+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:31:03.767+05:30 [Info] RebalanceServiceManager::doRecoverRebalance Found Global Rebalance Token &{18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:31:03.767+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:31:03.768+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:31:03.768+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:31:03.769+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {18194302e3f10df59bd076e1c4142d0b 098bb6b302f203e2ac2093f52370af18 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:31:03.771+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:31:03.771+05:30 [Info] Setting log level to Info 2023-10-21T04:31:03.771+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:31:03.794+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:31:03.794+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:31:03.803+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 46.321342ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:03.812+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:31:03.812+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:31:03.813+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:31:03.813+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:31:03.813+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:31:03.825+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:31:03.825+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:31:03.825+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:31:03.861+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:31:03.879+05:30 [Info] security context initialized 2023-10-21T04:31:03.879+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:31:03.881+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:31:03.897+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:31:03.897+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:03.898+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:03.898+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:31:03.906+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:31:03.910+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:31:03.910+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:31:03.911+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:31:03.913+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:31:03.914+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:31:03.928+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:31:03.929+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:31:03.935+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:31:03.953+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:31:03.954+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:31:03.959+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55970 2023-10-21T04:31:03.960+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:31:03.973+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 9.77547ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:03.973+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:31:03.973+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:31:03.974+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 31.333µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:03.980+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 252] aggregate storage info: map[bucket_%2:map[kv:2304192 index:4350220] bucket_1:map[kv:2369728 index:4358328]] 2023-10-21T04:31:03.980+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:31:03.997+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:31:03.999+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:31:03.999+05:30 [Info] initialized currmeta 26 force true 2023-10-21T04:31:04.005+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:31:04.005+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:31:04.005+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:31:04.006+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:31:04.013+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:31:04.013+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:31:04.014+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:31:04.015+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:31:04.033+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:39324" doAuth() ... 2023-10-21T04:31:04.033+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:39324" doReceive() ... 2023-10-21T04:31:04.038+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:31:04.038+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:31:04.038+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:31:04.039+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:31:04.046+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:31:04.046+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:31:04.046+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:31:04.048+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:31:04.051+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:31:04.051+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:31:04.051+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:31:04.051+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:31:04.051+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:31:04.051+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:31:04.052+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:31:04.053+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:31:04.053+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:31:04.056+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken17:ba:f0:81:68:83:f1:eb MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf9:5d:ac:9c:32:8:f6:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15448912456688985852 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5684298908800426981 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:04.057+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 098bb6b302f203e2ac2093f52370af18 TransferTokenId: ShardToken17:ba:f0:81:68:83:f1:eb Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:31:04.057+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000257149 2023-10-21T04:31:04.057+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken17:ba:f0:81:68:83:f1:eb, dropIndexes: true 2023-10-21T04:31:04.057+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb, shardId :0 2023-10-21T04:31:04.058+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken17_ba_f0_81_68_83_f1_eb, shardId :0 2023-10-21T04:31:04.061+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 6.372077ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.066+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16337157693634799600 instance id 15448912456688985852 real instance id 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:04.067+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 364.573µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.069+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16961575949607441659 instance id 5684298908800426981 real instance id 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:04.069+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 823.644µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.071+05:30 [Info] LifecycleMgr.DeleteIndex() : index 17797267771846799851 does not exist. 2023-10-21T04:31:04.072+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 796.824µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.074+05:30 [Info] LifecycleMgr.DeleteIndex() : index 11276338709617092948 does not exist. 2023-10-21T04:31:04.074+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 177.25µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.074+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:31:04.108+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno cache..., established connections: map[172.31.5.112:45436:172.31.5.112:12000] 2023-10-21T04:31:04.113+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 16787445029868804812 2023-10-21T04:31:04.114+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2925063509317026816 2023-10-21T04:31:04.113+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 16787445029868804812 2023-10-21T04:31:04.114+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2925063509317026816 2023-10-21T04:31:04.114+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [16787445029868804812 2925063509317026816], elapsed: 40.300963ms 2023-10-21T04:31:04.114+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 16787445029868804812 from local file system, err: error destroy shard, shardId 16787445029868804812 not found 2023-10-21T04:31:04.114+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 2925063509317026816 from local file system, err: error destroy shard, shardId 2925063509317026816 not found 2023-10-21T04:31:04.114+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [16787445029868804812 2925063509317026816], elapsed(sec): 0.040394863 2023-10-21T04:31:04.127+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken77:a6:d4:cd:41:6c:45:bd MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 098bb6b302f203e2ac2093f52370af18 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2f:a7:bd:b5:1a:16:61:f6 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7832199063701440128 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16895258949616040095 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:04.128+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 098bb6b302f203e2ac2093f52370af18 TransferTokenId: ShardToken77:a6:d4:cd:41:6c:45:bd Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:31:04.129+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd, shardId :0 2023-10-21T04:31:04.130+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/098bb6b302f203e2ac2093f52370af18_ShardToken77_a6_d4_cd_41_6c_45_bd, shardId :0 2023-10-21T04:31:04.130+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.002095787 2023-10-21T04:31:04.130+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken77:a6:d4:cd:41:6c:45:bd, dropIndexes: true 2023-10-21T04:31:04.137+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 8910459001587546151 instance id 7832199063701440128 real instance id 8965923595543532942 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:04.137+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 178.667µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.146+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 10606939961365971246 instance id 16895258949616040095 real instance id 8732144615635911668 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:04.146+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 10606939961365971246 instance 8732144615635911668 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:04.146+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 10606939961365971246 instance id 8732144615635911668 2023-10-21T04:31:04.146+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 10606939961365971246 2023-10-21T04:31:04.151+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:31:04.151+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:31:04.151+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0] 15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:31:04.151+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:04.151+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:04.151+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 3464125101225762400:[0] 15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[8965923595543532942:[7 8 1 2 3 4 5 6] 12266944006962282785:[0] 17434304260397056806:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:04.155+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 8732144615635911668 &{1} 2023-10-21T04:31:04.155+05:30 [Info] Indexer::handleDropIndex - IndexInstId 8732144615635911668 2023-10-21T04:31:04.155+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 10606939961365971246, instId: 8732144615635911668 2023-10-21T04:31:04.155+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [8732144615635911668] 2023-10-21T04:31:04.155+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [8732144615635911668] 2023-10-21T04:31:04.155+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [8732144615635911668] 2023-10-21T04:31:04.155+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 4, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:04.155+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:04.155+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:04.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.155+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.156+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 5, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.156+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.156+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.156+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:04.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:04.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:04.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.157+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 7, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:04.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:04.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:04.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.157+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.157+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 8, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.162+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.162+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.162+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.162+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:04.162+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:04.162+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:04.162+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:04.162+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:04.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:31:04.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.158+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 6, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:04.164+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:04.164+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:04.164+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:04.165+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:04.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:31:04.165+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.165+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.165+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.165+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.159+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 1, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.166+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.166+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.166+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:04.166+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:04.167+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:04.167+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:04.167+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:04.167+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:31:04.167+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.167+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.168+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.168+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.159+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:04.168+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 13.566802ms 2023-10-21T04:31:04.168+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 13.773174ms 2023-10-21T04:31:04.159+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 2, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.169+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.169+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.169+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:04.170+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:04.170+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:04.170+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:04.170+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:04.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:31:04.170+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.170+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.170+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.170+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.169+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 8732144615635911668 2023-10-21T04:31:04.160+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:04.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:04.171+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:04.171+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:04.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.159+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 3, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:04.172+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:04.172+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:04.173+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:04.173+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:04.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:31:04.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.169+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:04.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:31:04.174+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.174+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.174+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.174+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:04.175+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:04.175+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:04.175+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:04.175+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:04.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:31:04.175+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.175+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.175+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.175+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:04.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:31:04.176+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.176+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.176+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.176+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:04.176+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:04.176+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:04.176+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:04.177+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:04.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:31:04.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.178+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:04.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:31:04.178+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.178+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:04.179+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:04.179+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:04.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.179+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.202+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:04.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:31:04.202+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.202+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:04.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:04.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:04.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:31:04.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:04.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:04.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:04.214+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:04.214+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:04.214+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:04.214+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:04.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:31:04.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.215+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:04.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:31:04.215+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.215+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:04.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:04.217+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.212+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:31:04.229+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 4, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.229+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 4 Destroy Done 2023-10-21T04:31:04.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:04.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:04.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:04.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:04.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:04.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:04.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:04.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:04.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:04.236+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:04.233+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 86.455246ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:04.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:31:04.237+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.237+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:04.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:04.238+05:30 [Info] LifecycleMgr.DeleteIndex() : index 12824340686309573428 does not exist. 2023-10-21T04:31:04.238+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 73.203µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.241+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:31:04.243+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 5, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.243+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 5 Destroy Done 2023-10-21T04:31:04.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.243+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:04.243+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:04.243+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:04.243+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:04.243+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:04.243+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:04.243+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:04.243+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:04.243+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:04.243+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:04.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:31:04.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:31:04.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:31:04.244+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.244+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.244+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.244+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:31:04.245+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.245+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.245+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.245+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:31:04.246+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 7, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.246+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 7 Destroy Done 2023-10-21T04:31:04.257+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.258+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.260+05:30 [Info] LifecycleMgr.DeleteIndex() : index 3381524789655566900 does not exist. 2023-10-21T04:31:04.260+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 121.901µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:04.260+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17406544375239214575 10170029854712240877] 2023-10-21T04:31:04.276+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:04.276+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:04.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:31:04.276+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.277+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:04.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:04.277+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.277+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:31:04.277+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:04.277+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:04.277+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:04.277+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:04.277+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:04.277+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:04.277+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.278+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.278+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.278+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.281+05:30 [Info] switched currmeta from 26 -> 28 force true 2023-10-21T04:31:04.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:04.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:04.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:04.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:04.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:31:04.285+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:04.286+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 3, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.286+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 3 Destroy Done 2023-10-21T04:31:04.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:04.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:31:04.286+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:04.286+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:04.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:04.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:04.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:31:04.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:04.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:04.290+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 6, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.290+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 6 Destroy Done 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:31:04.295+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 8, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.295+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 8 Destroy Done 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:04.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:04.295+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:04.295+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:04.298+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:31:04.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:31:04.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:31:04.299+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 1, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.299+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 1 Destroy Done 2023-10-21T04:31:04.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:04.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:04.302+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45470:172.31.5.112:12000] 2023-10-21T04:31:04.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:04.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:31:04.303+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 2, IndexDefnId 10606939961365971246. 2023-10-21T04:31:04.303+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 2 Destroy Done 2023-10-21T04:31:04.335+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45472:172.31.5.112:12000] 2023-10-21T04:31:04.350+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45474:172.31.5.112:12000] 2023-10-21T04:31:04.366+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45476:172.31.5.112:12000] 2023-10-21T04:31:04.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:31:04.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:04.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[7 8 1 2 3 4 5 6] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:31:04.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[7 8 1 2 3 4 5 6] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:04.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[6 7 8 1 2 3 4 5] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[7 8 1 2 3 4 5 6] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:04.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[6 7 8 1 2 3 4 5] 14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 {Index inst : [partitions]}: map[3345094951175719474:[7 8 1 2 3 4 5 6] 10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:04.387+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45478:172.31.5.112:12000] 2023-10-21T04:31:04.419+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45482:172.31.5.112:12000] 2023-10-21T04:31:04.427+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45484:172.31.5.112:12000] 2023-10-21T04:31:04.446+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45486:172.31.5.112:12000] 2023-10-21T04:31:04.456+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45488:172.31.5.112:12000] 2023-10-21T04:31:04.472+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45490:172.31.5.112:12000] 2023-10-21T04:31:04.473+05:30 [Info] progress stats collection done. 2023-10-21T04:31:04.473+05:30 [Info] send progress stats to clients 2023-10-21T04:31:04.474+05:30 [Info] Indexer::ReadMemstats Time Taken 215.867µs 2023-10-21T04:31:04.974+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:31:05.113+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 10170029854712240877 2023-10-21T04:31:05.113+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17406544375239214575 2023-10-21T04:31:05.113+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17406544375239214575 2023-10-21T04:31:05.113+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 10170029854712240877 2023-10-21T04:31:05.113+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17406544375239214575 10170029854712240877], elapsed: 852.80811ms 2023-10-21T04:31:05.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:05.113+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) removed 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper stopped 2023-10-21T04:31:05.113+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': stopped 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:05.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All daemons stopped 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:05.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.Close: Begin close 2023-10-21T04:31:05.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.Close: Done close 2023-10-21T04:31:05.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:05.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:05.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:05.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:05.115+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.Close: Begin close 2023-10-21T04:31:05.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.Close: Done close 2023-10-21T04:31:05.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:05.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:05.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:05.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All instance closed 2023-10-21T04:31:05.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shutdown completed 2023-10-21T04:31:05.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:05.116+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) removed 2023-10-21T04:31:05.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All daemons stopped 2023-10-21T04:31:05.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All instance closed 2023-10-21T04:31:05.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All instance destroyed 2023-10-21T04:31:05.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:31:05.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:05.117+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) removed 2023-10-21T04:31:05.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:05.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All daemons stopped 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:05.118+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.Close: Begin close 2023-10-21T04:31:05.120+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.Close: Done close 2023-10-21T04:31:05.120+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:05.120+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:05.120+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:05.120+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:05.120+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.Close: Begin close 2023-10-21T04:31:05.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.Close: Done close 2023-10-21T04:31:05.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:05.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:05.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:05.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All instance closed 2023-10-21T04:31:05.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shutdown completed 2023-10-21T04:31:05.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:05.122+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) removed 2023-10-21T04:31:05.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All daemons stopped 2023-10-21T04:31:05.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All instance closed 2023-10-21T04:31:05.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All instance destroyed 2023-10-21T04:31:05.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:31:05.125+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': Cleaned up Tenant with name:bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:31:05.125+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17406544375239214575 10170029854712240877], elapsed(sec): 0.865037544 2023-10-21T04:31:05.135+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:31:05.135+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:31:05.135+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.619µs 2023-10-21T04:31:05.135+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:31:05.135+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:31:05.135+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:31:05.137+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:31:05.137+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:31:05.137+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:31:05.138+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:31:05.138+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 35.335µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:05.139+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:31:05.139+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:31:05.139+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{18194302e3f10df59bd076e1c4142d0b 7060000 } 2023-10-21T04:31:05.140+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.38056816s in RebalanceServiceManager::recoverRebalance: 2023-10-21T04:31:06.946+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56100 2023-10-21T04:31:06.946+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56098 2023-10-21T04:31:06.948+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56102 2023-10-21T04:31:06.948+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56106 2023-10-21T04:31:06.948+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56104 2023-10-21T04:31:06.949+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56108 2023-10-21T04:31:08.003+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571] 2023-10-21T04:31:08.004+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571] 2023-10-21T04:31:08.004+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 4 NodeUUIDs: [964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571] 2023-10-21T04:31:08.004+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee 7f731cd927c72e28ff24e69497b93571] IsBalanced:false Messages:[]} 2023-10-21T04:31:08.004+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:31:08.007+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:31:08.007+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:31:08.344+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:08.351+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:08.351+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:08.354+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:31:08.822+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:08.850+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:19.368+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:31:19.411+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:19.411+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:19.417+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:31:19.484+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:19.494+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:19.494+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:19.496+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:31:19.523+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 185] updated cluster rebalancing status to true 2023-10-21T04:31:19.523+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 185] updating rebalancing status to true 2023-10-21T04:31:19.672+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:31:19.800+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:19.800+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:19.804+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:31:22.341+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {c596d1e7da6bfc94cbe77df771748cc5 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:31:22.346+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.504µs 2023-10-21T04:31:22.346+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:31:22.346+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:31:22.346+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:31:22.349+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:31:22.349+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:22.349+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.693757ms 2023-10-21T04:31:22.349+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:31:22.349+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:31:22.349+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:31:22.349+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.439µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:22.349+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/c596d1e7da6bfc94cbe77df771748cc5 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:c596d1e7da6bfc94cbe77df771748cc5]}]} 2023-10-21T04:31:22.351+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:31:22.351+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:31:23.458+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {8806f5706747af4e635b8da5fcd8aeee c596d1e7da6bfc94cbe77df771748cc5 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:31:23.460+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {8806f5706747af4e635b8da5fcd8aeee c596d1e7da6bfc94cbe77df771748cc5 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:31:23.460+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"c596d1e7da6bfc94cbe77df771748cc5","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:31:23.462+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{8806f5706747af4e635b8da5fcd8aeee c596d1e7da6bfc94cbe77df771748cc5 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:31:23.462+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{8806f5706747af4e635b8da5fcd8aeee c596d1e7da6bfc94cbe77df771748cc5 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:31:23.462+05:30 [Info] ShardRebalancer::observeRebalance &{8806f5706747af4e635b8da5fcd8aeee c596d1e7da6bfc94cbe77df771748cc5 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:31:23.466+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"c596d1e7da6bfc94cbe77df771748cc5","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:31:23.466+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:31:23.587+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40916 2023-10-21T04:31:23.595+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40916 2023-10-21T04:31:23.620+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40916 2023-10-21T04:31:23.712+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2e:2c:44:28:da:2:14:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenae:b8:e2:e7:52:ef:e5:ce Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9447329072527540494 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5145547347365683414 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.744+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2e:2c:44:28:da:2:14:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenae:b8:e2:e7:52:ef:e5:ce Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9447329072527540494 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5145547347365683414 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.745+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:31:23.745+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:31:23.745+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:31:23.745+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 24.055µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:23.790+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:7c:cc:96:58:f4:7c:a8 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7309083551510465457 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14527226347372019068 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.791+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.792+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2e:2c:44:28:da:2:14:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenae:b8:e2:e7:52:ef:e5:ce Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9447329072527540494 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5145547347365683414 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.862+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:7c:cc:96:58:f4:7c:a8 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7309083551510465457 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14527226347372019068 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.863+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:31:23.863+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:31:23.863+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:31:23.863+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.39µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:23.891+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.892+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.895+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:7c:cc:96:58:f4:7c:a8 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7309083551510465457 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14527226347372019068 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:23.944+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.028+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.034+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.086+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"c596d1e7da6bfc94cbe77df771748cc5","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:31:24.086+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:31:24.086+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:31:24.086+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:31:24.086+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:31:24.086+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:31:24.087+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 18.756µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:24.098+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2e:2c:44:28:da:2:14:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenae:b8:e2:e7:52:ef:e5:ce Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9447329072527540494 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5145547347365683414 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.108+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.208+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:7c:cc:96:58:f4:7c:a8 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7309083551510465457 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14527226347372019068 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.219+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.272+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.346+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:7c:cc:96:58:f4:7c:a8 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7309083551510465457 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14527226347372019068 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.346+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:31:24.346+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:31:24.346+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:31:24.346+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.94µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:24.346+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[10170029854712240877:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/10170029854712240877/shards/shard10170029854712240877 17406544375239214575:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/17406544375239214575/shards/shard17406544375239214575] c596d1e7da6bfc94cbe77df771748cc5 ShardToken75:7c:cc:96:58:f4:7c:a8 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0084ae240 0xc0084ae2a0 map[] 0xc00834b260 0xc00853a900 false} 2023-10-21T04:31:24.347+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/17406544375239214575/shards/shard17406544375239214575 2023-10-21T04:31:24.347+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/10170029854712240877/shards/shard10170029854712240877 2023-10-21T04:31:24.349+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/17406544375239214575/shards/shard17406544375239214575/data: super block snapshot head offset :0 tail offset:757760 sbgen :136 2023-10-21T04:31:24.350+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/10170029854712240877/shards/shard10170029854712240877/data: super block snapshot head offset :0 tail offset:651264 sbgen :126 2023-10-21T04:31:24.350+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/17406544375239214575/shards/shard17406544375239214575/data/recovery: super block snapshot head offset :0 tail offset:708608 sbgen :136 2023-10-21T04:31:24.350+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/10170029854712240877/shards/shard10170029854712240877/data/recovery: super block snapshot head offset :0 tail offset:593920 sbgen :126 2023-10-21T04:31:24.351+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{8806f5706747af4e635b8da5fcd8aeee c596d1e7da6bfc94cbe77df771748cc5 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:31:24.353+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/10170029854712240877/shards/shard10170029854712240877: metadata saved successfully 2023-10-21T04:31:24.357+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:31:24.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:31:24.357+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 126429000 totalBytes: 1264290 numShards: 2 duration: 10.454335ms 2023-10-21T04:31:24.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/10170029854712240877/shards/shard10170029854712240877, newshardId 10170029854712240877, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, shardsize 1264290, duration 9.742059ms 2023-10-21T04:31:24.357+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10170029854712240877, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, err: , elapsed(sec): 0.010677013 2023-10-21T04:31:24.357+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10170029854712240877 bytesWritten: 1264290, totalBytes: 1264290, transferRate: 1.4047666666666666e+08 2023-10-21T04:31:24.357+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/17406544375239214575/shards/shard17406544375239214575: metadata saved successfully 2023-10-21T04:31:24.358+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:31:24.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:31:24.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8/17406544375239214575/shards/shard17406544375239214575, newshardId 17406544375239214575, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, shardsize 1485636, duration 17.042133ms 2023-10-21T04:31:24.364+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17406544375239214575, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, err: , elapsed(sec): 0.017480113 2023-10-21T04:31:24.364+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17406544375239214575 bytesWritten: 1485636, totalBytes: 1485636, transferRate: 8.739035294117647e+07 2023-10-21T04:31:24.364+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: c596d1e7da6bfc94cbe77df771748cc5, transferId: ShardToken75:7c:cc:96:58:f4:7c:a8 2023-10-21T04:31:24.364+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: c596d1e7da6bfc94cbe77df771748cc5, transferId: ShardToken75:7c:cc:96:58:f4:7c:a8, elapsed(sec): 0.000134429 2023-10-21T04:31:24.364+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[10170029854712240877: 17406544375239214575:], shardPaths: map[10170029854712240877:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 17406544375239214575:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.017829417 2023-10-21T04:31:24.364+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17406544375239214575 10170029854712240877], ttid: ShardToken75:7c:cc:96:58:f4:7c:a8, elapsed(sec): 0.01842998 2023-10-21T04:31:24.394+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:24.444+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2e:2c:44:28:da:2:14:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenae:b8:e2:e7:52:ef:e5:ce Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9447329072527540494 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5145547347365683414 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.446+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:31:24.447+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:31:24.448+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:31:24.449+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2925063509317026816:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/2925063509317026816/shards/shard2925063509317026816 16787445029868804812:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/16787445029868804812/shards/shard16787445029868804812] c596d1e7da6bfc94cbe77df771748cc5 ShardToken2e:2c:44:28:da:2:14:94 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc0084ae240 0xc0084ae2a0 map[] 0xc0085447e0 0xc0084aef00 false} 2023-10-21T04:31:24.450+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/2925063509317026816/shards/shard2925063509317026816 2023-10-21T04:31:24.450+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/16787445029868804812/shards/shard16787445029868804812 2023-10-21T04:31:24.449+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.727µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:24.453+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/16787445029868804812/shards/shard16787445029868804812/data: super block snapshot head offset :0 tail offset:921600 sbgen :156 2023-10-21T04:31:24.453+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/16787445029868804812/shards/shard16787445029868804812/data/recovery: super block snapshot head offset :0 tail offset:876544 sbgen :156 2023-10-21T04:31:24.454+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/2925063509317026816/shards/shard2925063509317026816/data: super block snapshot head offset :0 tail offset:794624 sbgen :144 2023-10-21T04:31:24.455+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/2925063509317026816/shards/shard2925063509317026816/data/recovery: super block snapshot head offset :0 tail offset:729088 sbgen :144 2023-10-21T04:31:24.457+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/16787445029868804812/shards/shard16787445029868804812: metadata saved successfully 2023-10-21T04:31:24.457+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:31:24.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:31:24.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/16787445029868804812/shards/shard16787445029868804812, newshardId 16787445029868804812, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, shardsize 1817403, duration 7.252343ms 2023-10-21T04:31:24.458+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 16787445029868804812, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, err: , elapsed(sec): 0.008371451 2023-10-21T04:31:24.458+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 16787445029868804812 bytesWritten: 1817403, totalBytes: 1817403, transferRate: 2.59629e+08 2023-10-21T04:31:24.459+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/2925063509317026816/shards/shard2925063509317026816: metadata saved successfully 2023-10-21T04:31:24.459+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:31:24.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:31:24.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94/2925063509317026816/shards/shard2925063509317026816, newshardId 2925063509317026816, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, shardsize 1542812, duration 10.01191ms 2023-10-21T04:31:24.460+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2925063509317026816, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, err: , elapsed(sec): 0.010390546 2023-10-21T04:31:24.460+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: c596d1e7da6bfc94cbe77df771748cc5, transferId: ShardToken2e:2c:44:28:da:2:14:94 2023-10-21T04:31:24.460+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: c596d1e7da6bfc94cbe77df771748cc5, transferId: ShardToken2e:2c:44:28:da:2:14:94, elapsed(sec): 7.6173e-05 2023-10-21T04:31:24.460+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2925063509317026816: 16787445029868804812:], shardPaths: map[2925063509317026816:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 16787445029868804812:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.010543617 2023-10-21T04:31:24.460+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2925063509317026816 bytesWritten: 1542812, totalBytes: 1542812, transferRate: 1.542812e+08 2023-10-21T04:31:24.460+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [16787445029868804812 2925063509317026816], ttid: ShardToken2e:2c:44:28:da:2:14:94, elapsed(sec): 0.014241317 2023-10-21T04:31:24.496+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:7c:cc:96:58:f4:7c:a8 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7309083551510465457 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14527226347372019068 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.504+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken75:7c:cc:96:58:f4:7c:a8 2023-10-21T04:31:24.504+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[10170029854712240877:true 17406544375239214575:true] 2023-10-21T04:31:24.504+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.617µs 2023-10-21T04:31:24.504+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [17406544375239214575 10170029854712240877], isLockedForRecovery: true 2023-10-21T04:31:24.504+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [17406544375239214575 10170029854712240877], errMap: map[10170029854712240877: 17406544375239214575:], elapsed: 10.831µs 2023-10-21T04:31:24.550+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.551+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:31:24.551+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:31:24.552+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:24.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.595+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.596+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.596+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.596+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.596+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.596+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.596+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.596+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:24.601+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.605+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken2e:2c:44:28:da:2:14:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenae:b8:e2:e7:52:ef:e5:ce Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9447329072527540494 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5145547347365683414 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.607+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken2e:2c:44:28:da:2:14:94 2023-10-21T04:31:24.615+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8965923595543532942,realInstId: 0, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:24.620+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2925063509317026816:true 16787445029868804812:true] 2023-10-21T04:31:24.620+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.579µs 2023-10-21T04:31:24.621+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [16787445029868804812 2925063509317026816], isLockedForRecovery: true 2023-10-21T04:31:24.622+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [16787445029868804812 2925063509317026816], errMap: map[2925063509317026816: 16787445029868804812:], elapsed: 262.296µs 2023-10-21T04:31:24.621+05:30 [Info] Indexer::handleRecoverIndex InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:24.630+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 8.674648ms 2023-10-21T04:31:24.632+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:24.633+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 16.23987ms 2023-10-21T04:31:24.633+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8965923595543532942, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:24.649+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 98.389883ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:24.635+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 3 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:24.650+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': added 2023-10-21T04:31:24.680+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:24.696+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:31:24.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner initialized 2023-10-21T04:31:24.709+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.708+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner initialized 2023-10-21T04:31:24.711+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:24.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:24.722+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCleaner initialized for recovery 2023-10-21T04:31:24.744+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.730+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:24.739+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCleaner initialized for recovery 2023-10-21T04:31:24.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery 2023-10-21T04:31:24.748+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:24.749+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:24.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true] 2023-10-21T04:31:24.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery 2023-10-21T04:31:24.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:24.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true] 2023-10-21T04:31:24.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[593920] Data log Head [0] Tail [651264] 2023-10-21T04:31:24.759+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [593920] 2023-10-21T04:31:24.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[708608] Data log Head [0] Tail [757760] 2023-10-21T04:31:24.772+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [708608] 2023-10-21T04:31:24.771+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.775+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:31:24.776+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.777+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:31:24.777+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.779+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:31:24.779+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.781+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:31:24.781+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.782+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:31:24.784+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.785+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.792+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:31:24.793+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.793+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:31:24.800+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.800+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:31:24.805+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.804+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:24.806+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:31:24.808+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:31:24.808+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.808+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:31:24.809+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.810+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:31:24.810+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.811+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:31:24.810+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.812+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:31:24.812+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.818+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.818+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:31:24.820+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.820+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10032435514095102923,realInstId: 0, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:24.835+05:30 [Info] Indexer::handleRecoverIndex InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:24.820+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:31:24.821+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:24.839+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:24.840+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 5.033449ms 2023-10-21T04:31:24.840+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 5.241988ms 2023-10-21T04:31:24.840+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.841+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:31:24.841+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.841+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:31:24.842+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:24.842+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 6 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:24.842+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.843+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:31:24.843+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:31:24.843+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:31:24.845+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.845+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:24.845+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.845+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:24.846+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.846+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:24.846+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.846+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:24.846+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.847+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:24.847+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.847+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:24.847+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.847+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:24.848+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.848+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:24.848+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : replayOffset: 745472 (RP:745472) (CP:446464) 2023-10-21T04:31:24.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [745472] took [75.897991ms] 2023-10-21T04:31:24.848+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057426, resident 1, memSz 12313, memSzIndex 0, numRecords 123 2023-10-21T04:31:24.848+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37413619 controller mem 4702 2023-10-21T04:31:24.849+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.849+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37413619 controller mem 4702 2023-10-21T04:31:24.849+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.849+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37413619 controller mem 4702 2023-10-21T04:31:24.871+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057276, resident 1, memSz 12667, memSzIndex 0, numRecords 127 2023-10-21T04:31:24.849+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:24.850+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner initialized 2023-10-21T04:31:24.849+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner initialized 2023-10-21T04:31:24.849+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10032435514095102923, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:24.853+05:30 [Info] switched currmeta from 28 -> 33 force true 2023-10-21T04:31:24.872+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 37413619 controller mem 4702 2023-10-21T04:31:24.872+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057276, resident 1, memSz 13063, memSzIndex 0, numRecords 131 2023-10-21T04:31:24.872+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 38462195 controller mem 4702 2023-10-21T04:31:24.873+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057276, resident 1, memSz 11171, memSzIndex 0, numRecords 112 2023-10-21T04:31:24.873+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 38462195 controller mem 4702 2023-10-21T04:31:24.873+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057426, resident 1, memSz 12294, memSzIndex 0, numRecords 123 2023-10-21T04:31:24.873+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 38462195 controller mem 4702 2023-10-21T04:31:24.872+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.874+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:24.874+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.874+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:24.874+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.874+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:24.874+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.874+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:24.875+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.875+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:24.875+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.875+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:24.875+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.875+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:24.875+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : replayOffset: 638976 (RP:638976) (CP:360448) 2023-10-21T04:31:24.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [638976] took [116.333057ms] 2023-10-21T04:31:24.875+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053288, resident 1, memSz 9511, memSzIndex 76, numRecords 127 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053288, resident 1, memSz 9997, memSzIndex 77, numRecords 131 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053364, resident 1, memSz 9834, memSzIndex 0, numRecords 123 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053364, resident 1, memSz 9853, memSzIndex 0, numRecords 123 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1056128, resident 1, memSz 67959, memSzIndex 841, numRecords 1000 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053288, resident 1, memSz 9669, memSzIndex 76, numRecords 128 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053288, resident 1, memSz 9287, memSzIndex 0, numRecords 118 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053288, resident 1, memSz 10687, memSzIndex 75, numRecords 138 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.876+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.877+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.877+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.877+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.877+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.877+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1053288, resident 1, memSz 8667, memSzIndex 0, numRecords 112 2023-10-21T04:31:24.878+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.872+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.876+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:24.872+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.872+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 142.765097ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:24.873+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.884+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:24.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:24.890+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.891+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.892+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.890+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.893+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.893+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.894+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.894+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.894+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [651264] replayOffset [638976] 2023-10-21T04:31:24.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [135.732779ms] 2023-10-21T04:31:24.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.896+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.896+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.896+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.896+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.897+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.897+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.897+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.897+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.897+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.898+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.898+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.893+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.898+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.899+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.899+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.899+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057276, resident 1, memSz 12749, memSzIndex 0, numRecords 128 2023-10-21T04:31:24.899+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.898+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.900+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057276, resident 1, memSz 11767, memSzIndex 0, numRecords 118 2023-10-21T04:31:24.900+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.900+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1057276, resident 1, memSz 13729, memSzIndex 0, numRecords 138 2023-10-21T04:31:24.900+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.900+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1055942, resident 1, memSz 70473, memSzIndex 354, numRecords 1000 2023-10-21T04:31:24.900+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.900+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.901+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.901+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1055176, resident 1, memSz 938, memSzIndex 0, numRecords 8 2023-10-21T04:31:24.901+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.901+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:24.901+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802227 controller mem 4702 2023-10-21T04:31:24.901+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [757760] replayOffset [745472] 2023-10-21T04:31:24.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [129.575514ms] 2023-10-21T04:31:24.902+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.902+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.902+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.902+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.902+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.902+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.902+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true]. Built [18] plasmas, took [130.971487ms] 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 14 count: 123 metaSz: 2517 rp [1] sn: 15 count: 123 metaSz: 2520 2023-10-21T04:31:24.903+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 51003 recovery UsedSpace 58084 2023-10-21T04:31:24.904+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.904+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.904+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:24.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true]. Built [17] plasmas, took [145.072146ms] 2023-10-21T04:31:24.904+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:31:24.904+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 13 count: 123 metaSz: 2514 rp [1] sn: 14 count: 123 metaSz: 2517 rp [2] sn: 15 count: 123 metaSz: 2520 2023-10-21T04:31:24.904+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 44787 recovery UsedSpace 49870 2023-10-21T04:31:24.906+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCleaner initialized for recovery 2023-10-21T04:31:24.906+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery 2023-10-21T04:31:24.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:24.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true] 2023-10-21T04:31:24.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[729088] Data log Head [0] Tail [794624] 2023-10-21T04:31:24.906+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [729088] 2023-10-21T04:31:24.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:31:24.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:24.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:24.912+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper started 2023-10-21T04:31:24.912+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:31:24.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex started 2023-10-21T04:31:24.912+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:24.912+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:24.917+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCleaner initialized for recovery 2023-10-21T04:31:24.919+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:31:24.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery 2023-10-21T04:31:24.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:24.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true] 2023-10-21T04:31:24.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[876544] Data log Head [0] Tail [921600] 2023-10-21T04:31:24.922+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [876544] 2023-10-21T04:31:24.925+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.927+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:31:24.934+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.934+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:31:24.935+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.935+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:31:24.936+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.940+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:31:24.943+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.944+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:31:24.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:31:24.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:24.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:24.978+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:31:24.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex started 2023-10-21T04:31:24.978+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:31:24.979+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.970+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:24.980+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:24.980+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:24.982+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:31:24.982+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.984+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.984+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:31:24.984+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.984+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:31:24.984+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:31:24.985+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:24.985+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.987+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:31:24.986+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.988+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:31:24.989+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.989+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:24.989+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.990+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:24.990+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.990+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.991+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:24.991+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.991+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:31:24.996+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:24.996+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.996+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:24.997+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.997+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:24.998+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:24.998+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.003+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.003+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:31:25.004+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.004+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.006+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Warmup took 356.483579ms 2023-10-21T04:31:25.006+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 3 WriterThreads 1 2023-10-21T04:31:25.007+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index Index: 8965923595543532942 Partition: 3 2023-10-21T04:31:25.007+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.007+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 4 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.008+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:31:25.009+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 118 metaSz: 2514 rp [1] sn: 15 count: 118 metaSz: 2517 rp [2] sn: 16 count: 118 metaSz: 2520 2023-10-21T04:31:25.009+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 45862 recovery UsedSpace 49870 2023-10-21T04:31:25.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.010+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:25.010+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 118 metaSz: 2517 rp [1] sn: 16 count: 118 metaSz: 2520 2023-10-21T04:31:25.010+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 50672 recovery UsedSpace 58084 2023-10-21T04:31:25.006+05:30 [Info] switched currmeta from 33 -> 34 force true 2023-10-21T04:31:25.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:31:25.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.016+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:31:25.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex started 2023-10-21T04:31:25.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.017+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:31:25.017+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.017+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:31:25.017+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.017+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:31:25.017+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.017+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:31:25.017+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.017+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:31:25.019+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.019+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:31:25.019+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.019+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.019+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.019+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.020+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.020+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.020+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.020+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.020+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.020+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.022+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.023+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.023+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.023+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.024+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.024+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.032+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : replayOffset: 913408 (RP:905216) (CP:913408) 2023-10-21T04:31:25.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [913408] took [110.574858ms] 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1055176, resident 0.12, memSz 347, memSzIndex 0, numRecords 8 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12169, memSzIndex 0, numRecords 122 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 10600, memSzIndex 0, numRecords 106 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1055988, resident 1, memSz 70520, memSzIndex 401, numRecords 1000 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13546, memSzIndex 0, numRecords 136 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12489, memSzIndex 0, numRecords 125 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12281, memSzIndex 0, numRecords 123 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12657, memSzIndex 0, numRecords 127 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12491, memSzIndex 0, numRecords 125 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13565, memSzIndex 0, numRecords 136 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802801 controller mem 5378 2023-10-21T04:31:25.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [921600] replayOffset [913408] 2023-10-21T04:31:25.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [111.632207ms] 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true]. Built [18] plasmas, took [112.661919ms] 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 32768 recovery UsedSpace 8236 2023-10-21T04:31:25.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:31:25.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.036+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:31:25.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex started 2023-10-21T04:31:25.036+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:31:25.037+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.038+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.043+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : replayOffset: 782336 (RP:736722) (CP:782336) 2023-10-21T04:31:25.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [782336] took [136.525389ms] 2023-10-21T04:31:25.046+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.046+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9269, memSzIndex 92, numRecords 125 2023-10-21T04:31:25.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1056202, resident 1, memSz 66746, memSzIndex 1058, numRecords 1000 2023-10-21T04:31:25.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.049+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:31:25.049+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:31:25.049+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.049+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.049+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.051+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.051+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.051+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.051+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:31:25.051+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.052+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.053+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.053+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.053+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.053+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.054+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.054+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:31:25.054+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.054+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:31:25.054+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.054+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:31:25.054+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Warmup took 46.494084ms 2023-10-21T04:31:25.054+05:30 [Info] switched currmeta from 34 -> 34 force true 2023-10-21T04:31:25.054+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 4 WriterThreads 1 2023-10-21T04:31:25.055+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index Index: 8965923595543532942 Partition: 4 2023-10-21T04:31:25.055+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.055+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 5 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.055+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 238 2023-10-21T04:31:25.055+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 138 metaSz: 2514 rp [1] sn: 15 count: 138 metaSz: 2518 rp [2] sn: 16 count: 138 metaSz: 2520 2023-10-21T04:31:25.055+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 49974 recovery UsedSpace 50312 2023-10-21T04:31:25.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.056+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:25.056+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 138 metaSz: 2514 rp [1] sn: 15 count: 138 metaSz: 2518 rp [2] sn: 16 count: 138 metaSz: 2520 2023-10-21T04:31:25.056+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 53660 recovery UsedSpace 49870 2023-10-21T04:31:25.054+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45802799 controller mem 5378 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754259 controller mem 5410 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 10500, memSzIndex 77, numRecords 136 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754259 controller mem 5410 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9501, memSzIndex 76, numRecords 127 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754259 controller mem 5410 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754259 controller mem 5410 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:31:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754259 controller mem 5410 2023-10-21T04:31:25.069+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [794624] replayOffset [782336] 2023-10-21T04:31:25.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [163.729493ms] 2023-10-21T04:31:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:31:25.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true]. Built [17] plasmas, took [165.668771ms] 2023-10-21T04:31:25.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 26425 recovery UsedSpace 8236 2023-10-21T04:31:25.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:31:25.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.076+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:31:25.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex started 2023-10-21T04:31:25.076+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.077+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:25.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.079+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:25.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:31:25.080+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:31:25.080+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.080+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.080+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.081+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:31:25.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.083+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:31:25.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex started 2023-10-21T04:31:25.083+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:31:25.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.086+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.086+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:25.086+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.086+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:25.086+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:25.086+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:31:25.086+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.086+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.086+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.086+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.092+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:25.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:31:25.093+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.093+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.093+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.093+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:25.093+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:25.093+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:25.133+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Warmup took 78.132242ms 2023-10-21T04:31:25.133+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 5 WriterThreads 1 2023-10-21T04:31:25.133+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index Index: 8965923595543532942 Partition: 5 2023-10-21T04:31:25.133+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.134+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 6 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.134+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:31:25.134+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 127 metaSz: 2514 rp [1] sn: 15 count: 127 metaSz: 2518 rp [2] sn: 16 count: 127 metaSz: 2520 2023-10-21T04:31:25.134+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 50596 recovery UsedSpace 50224 2023-10-21T04:31:25.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.135+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:25.135+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 127 metaSz: 2518 rp [1] sn: 16 count: 127 metaSz: 2520 2023-10-21T04:31:25.135+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 54914 recovery UsedSpace 58084 2023-10-21T04:31:25.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:31:25.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.140+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:31:25.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex started 2023-10-21T04:31:25.140+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.140+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:31:25.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.141+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:31:25.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex started 2023-10-21T04:31:25.142+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:31:25.142+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.142+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.149+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:25.149+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:25.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:31:25.149+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.149+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.150+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.150+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.156+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Warmup took 22.421661ms 2023-10-21T04:31:25.157+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 6 WriterThreads 1 2023-10-21T04:31:25.157+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:25.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:31:25.157+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:31:25.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:31:25.157+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index Index: 8965923595543532942 Partition: 6 2023-10-21T04:31:25.158+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.159+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 7 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:31:25.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:31:25.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:31:25.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:31:25.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 131 metaSz: 2514 rp [1] sn: 15 count: 131 metaSz: 2518 rp [2] sn: 16 count: 131 metaSz: 2520 2023-10-21T04:31:25.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 52917 recovery UsedSpace 50324 2023-10-21T04:31:25.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.165+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:25.165+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 131 metaSz: 2518 rp [1] sn: 16 count: 131 metaSz: 2520 2023-10-21T04:31:25.165+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 55359 recovery UsedSpace 58084 2023-10-21T04:31:25.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.168+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.168+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:31:25.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.171+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:31:25.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex started 2023-10-21T04:31:25.171+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.172+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.173+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:31:25.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.173+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:31:25.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex started 2023-10-21T04:31:25.173+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:31:25.174+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.174+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.177+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.177+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:25.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:31:25.177+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.177+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:25.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.179+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:25.179+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:25.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:31:25.179+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Warmup took 336.648014ms 2023-10-21T04:31:25.180+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.180+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 6 WriterThreads 1 2023-10-21T04:31:25.180+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Stopped Billing WUs 2023-10-21T04:31:25.180+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index Index: 10032435514095102923 Partition: 6 2023-10-21T04:31:25.180+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.180+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.181+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 7 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.182+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.183+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:25.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.183+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.184+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 27306 recovery UsedSpace 8236 2023-10-21T04:31:25.185+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Warmup took 25.822912ms 2023-10-21T04:31:25.185+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 7 WriterThreads 1 2023-10-21T04:31:25.185+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index Index: 8965923595543532942 Partition: 7 2023-10-21T04:31:25.185+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.186+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 8 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.186+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:31:25.186+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 112 metaSz: 2514 rp [1] sn: 15 count: 112 metaSz: 2518 rp [2] sn: 16 count: 112 metaSz: 2520 2023-10-21T04:31:25.186+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 51982 recovery UsedSpace 49870 2023-10-21T04:31:25.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.186+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:25.186+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 112 metaSz: 2518 rp [1] sn: 16 count: 112 metaSz: 2520 2023-10-21T04:31:25.186+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 56042 recovery UsedSpace 58084 2023-10-21T04:31:25.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.207+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:31:25.207+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.207+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:25.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.208+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:25.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:31:25.208+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:31:25.208+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.208+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.208+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.208+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:25.208+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.208+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:25.208+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:25.208+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:31:25.208+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.208+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.208+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.208+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.209+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.209+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:31:25.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.211+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:31:25.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex started 2023-10-21T04:31:25.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:31:25.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.212+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:31:25.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex started 2023-10-21T04:31:25.212+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:31:25.213+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.213+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:25.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:31:25.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.215+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.215+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.215+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:25.215+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:25.215+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:25.215+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:25.215+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:25.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:31:25.216+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.216+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.216+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.216+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.232+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:25.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:31:25.233+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:31:25.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:31:25.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:31:25.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:31:25.243+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Warmup took 57.764839ms 2023-10-21T04:31:25.244+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 8 WriterThreads 1 2023-10-21T04:31:25.244+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index Index: 8965923595543532942 Partition: 8 2023-10-21T04:31:25.244+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.244+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 1 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:31:25.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.246+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:31:25.246+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 13 count: 128 metaSz: 2514 rp [1] sn: 14 count: 128 metaSz: 2518 rp [2] sn: 15 count: 128 metaSz: 2520 2023-10-21T04:31:25.246+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 54603 recovery UsedSpace 50318 2023-10-21T04:31:25.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.247+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:25.247+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 14 count: 128 metaSz: 2518 rp [1] sn: 15 count: 128 metaSz: 2520 2023-10-21T04:31:25.247+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 59391 recovery UsedSpace 58084 2023-10-21T04:31:25.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.249+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:31:25.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.250+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:31:25.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex started 2023-10-21T04:31:25.250+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.250+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.252+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.252+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:31:25.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.255+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:31:25.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex started 2023-10-21T04:31:25.255+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:31:25.256+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.256+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.257+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.257+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:25.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:31:25.257+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.257+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:25.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.259+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:25.259+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:25.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:31:25.259+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Warmup took 77.112393ms 2023-10-21T04:31:25.259+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 7 WriterThreads 1 2023-10-21T04:31:25.259+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Stopped Billing WUs 2023-10-21T04:31:25.259+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index Index: 10032435514095102923 Partition: 7 2023-10-21T04:31:25.259+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.260+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 8 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.260+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.260+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 27306 recovery UsedSpace 8236 2023-10-21T04:31:25.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:25.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.271+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:31:25.271+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.271+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.272+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:31:25.272+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:31:25.272+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:25.272+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.272+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:25.272+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:25.272+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.272+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.273+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.281+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Warmup took 36.908181ms 2023-10-21T04:31:25.281+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 1 WriterThreads 1 2023-10-21T04:31:25.281+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index Index: 8965923595543532942 Partition: 1 2023-10-21T04:31:25.281+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.284+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 2 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.284+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:31:25.284+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 6 numRPs: 3 rp [0] sn: 14 count: 123 metaSz: 2514 rp [1] sn: 15 count: 123 metaSz: 2517 rp [2] sn: 16 count: 123 metaSz: 2520 2023-10-21T04:31:25.284+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 54877 recovery UsedSpace 49870 2023-10-21T04:31:25.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.284+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:25.284+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 15 count: 123 metaSz: 2517 rp [1] sn: 16 count: 123 metaSz: 2520 2023-10-21T04:31:25.285+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 59140 recovery UsedSpace 58084 2023-10-21T04:31:25.290+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:25.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:31:25.290+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.290+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.290+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.291+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:25.291+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:25.291+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:25.291+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:25.291+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:25.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:31:25.291+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.291+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.291+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.291+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:31:25.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.302+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:31:25.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex started 2023-10-21T04:31:25.302+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.302+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:31:25.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.302+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:31:25.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex started 2023-10-21T04:31:25.303+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:31:25.304+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.304+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.313+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:25.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:31:25.314+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:31:25.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:31:25.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:31:25.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:31:25.343+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.343+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:31:25.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.351+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.351+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.352+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Warmup took 68.775438ms 2023-10-21T04:31:25.353+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 2 WriterThreads 1 2023-10-21T04:31:25.353+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index Index: 8965923595543532942 Partition: 2 2023-10-21T04:31:25.353+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8965923595543532942, shardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] err: 2023-10-21T04:31:25.353+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8965923595543532942, partnShardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]], err: 2023-10-21T04:31:25.353+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:25.353+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8965923595543532942 2023-10-21T04:31:25.353+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:25.353+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:31:25.354+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281] 2023-10-21T04:31:25.354+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:25.354+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:25.354+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8965923595543532942 Partitions [5 6 7 8 1 2 3 4] 2023-10-21T04:31:25.355+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:31:25.356+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:7 Attempting to open snapshot (SnapshotInfo: count:131 committed:false) 2023-10-21T04:31:25.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.362+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.362+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:25.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.367+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:25.367+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:25.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:31:25.367+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.367+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.367+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.59212ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:25.367+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 121.557µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:25.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.372+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.372+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:25.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:31:25.372+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Warmup took 112.24474ms 2023-10-21T04:31:25.372+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 8 WriterThreads 1 2023-10-21T04:31:25.372+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Stopped Billing WUs 2023-10-21T04:31:25.372+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index Index: 10032435514095102923 Partition: 8 2023-10-21T04:31:25.372+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.372+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 1 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.373+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.373+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 28248 recovery UsedSpace 8236 2023-10-21T04:31:25.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:25.374+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:8 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:31:25.377+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:1 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:31:25.379+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:2 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:31:25.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:25.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.381+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:25.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:31:25.383+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.382+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:3 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:31:25.384+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.385+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:4 Attempting to open snapshot (SnapshotInfo: count:118 committed:false) 2023-10-21T04:31:25.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:31:25.386+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:31:25.386+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:25.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.386+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:25.386+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:25.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.387+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.387+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.387+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.390+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:25.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:31:25.390+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.390+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.390+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.390+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:25.390+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:25.390+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:25.390+05:30 [Info] switched currmeta from 34 -> 35 force true 2023-10-21T04:31:25.390+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:25.391+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:25.391+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:31:25.391+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.391+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.391+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.391+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.394+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:5 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:31:25.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:25.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:31:25.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:31:25.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:31:25.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:31:25.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:31:25.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.403+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:31:25.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.411+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.411+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.427+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.427+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.427+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.427+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:25.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:31:25.427+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.428+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:25.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:25.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:25.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:31:25.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.450+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Warmup took 77.696528ms 2023-10-21T04:31:25.451+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 1 WriterThreads 1 2023-10-21T04:31:25.451+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Stopped Billing WUs 2023-10-21T04:31:25.451+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index Index: 10032435514095102923 Partition: 1 2023-10-21T04:31:25.451+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.451+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 2 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.452+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.452+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 28248 recovery UsedSpace 8236 2023-10-21T04:31:25.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.452+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.453+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:25.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:25.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.472+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:25.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:31:25.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.473+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.474+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.474+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.475+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:31:25.475+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:31:25.475+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.475+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.475+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:25.476+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.476+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:25.476+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:25.476+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:31:25.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.476+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.477+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.486+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:25.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:31:25.486+05:30 [Info] switched currmeta from 35 -> 36 force true 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:25.487+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:25.487+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:25.487+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:25.487+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:25.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.491+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:25.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:31:25.491+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:31:25.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:31:25.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:31:25.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:31:25.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:31:25.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.501+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.501+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.505+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.505+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:25.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.519+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:25.519+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:25.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:31:25.519+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.520+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.529+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.529+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:25.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:31:25.530+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.530+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Warmup took 78.416016ms 2023-10-21T04:31:25.530+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.531+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 2 WriterThreads 1 2023-10-21T04:31:25.531+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Stopped Billing WUs 2023-10-21T04:31:25.531+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index Index: 10032435514095102923 Partition: 2 2023-10-21T04:31:25.531+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.532+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 3 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.533+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.534+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:31:25.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.535+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 29257 recovery UsedSpace 8236 2023-10-21T04:31:25.540+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.540+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.542+05:30 [Info] switched currmeta from 36 -> 37 force true 2023-10-21T04:31:25.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.550+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:31:25.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.551+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.551+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.551+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:25.551+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.551+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:25.551+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:25.551+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.551+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:25.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:31:25.564+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.564+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.564+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.565+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:25.565+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.565+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:25.565+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:25.565+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:31:25.565+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.565+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.565+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.565+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.587+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:25.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:31:25.587+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:31:25.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:31:25.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:31:25.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:31:25.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:31:25.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.606+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.606+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.621+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.621+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.626+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.626+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:25.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:31:25.626+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.626+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.630+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.630+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:25.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:31:25.630+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.630+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Warmup took 97.511533ms 2023-10-21T04:31:25.630+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.630+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 3 WriterThreads 1 2023-10-21T04:31:25.630+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Stopped Billing WUs 2023-10-21T04:31:25.630+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index Index: 10032435514095102923 Partition: 3 2023-10-21T04:31:25.631+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.631+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 4 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.631+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.631+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 29257 recovery UsedSpace 8236 2023-10-21T04:31:25.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.632+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.632+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:31:25.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.650+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:31:25.650+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.650+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.651+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:31:25.651+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:31:25.651+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:25.651+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.651+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:25.651+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:25.651+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.651+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.655+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:25.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:31:25.655+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.655+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.656+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.656+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:25.656+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.656+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:25.656+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:25.656+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:31:25.656+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.656+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.656+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.656+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.660+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:25.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:31:25.660+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:31:25.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:31:25.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:31:25.663+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:31:25.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.666+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:31:25.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.669+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.669+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.671+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.671+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.673+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.673+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:25.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:31:25.673+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.673+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.675+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.675+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:25.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:31:25.675+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Warmup took 44.287426ms 2023-10-21T04:31:25.675+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 4 WriterThreads 1 2023-10-21T04:31:25.675+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Stopped Billing WUs 2023-10-21T04:31:25.675+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index Index: 10032435514095102923 Partition: 4 2023-10-21T04:31:25.675+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.676+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 5 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.676+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 29257 recovery UsedSpace 8236 2023-10-21T04:31:25.676+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.676+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.677+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.677+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.685+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:31:25.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.685+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:31:25.685+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:31:25.685+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:25.685+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.685+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:25.685+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:25.685+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:31:25.686+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.686+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.686+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.686+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.687+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8965923595543532942:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:25.687+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 10606939961365971246, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:25.687+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 10606939961365971246, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:25.688+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.688+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.695+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:25.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:25.696+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:25.696+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:25.696+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:25.696+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:25.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.696+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.706+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:25.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:31:25.706+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:31:25.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:31:25.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:31:25.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:31:25.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:31:25.716+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8732144615635911668,realInstId: 0, indexDefn DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:25.716+05:30 [Info] Indexer::handleRecoverIndex InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:25.716+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 434.382µs 2023-10-21T04:31:25.717+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 500.771µs 2023-10-21T04:31:25.717+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.717+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8732144615635911668, indexDefn DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:25.717+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.528612ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:25.717+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 1 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.720+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:25.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:31:25.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.725+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.725+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 31507 recovery UsedSpace 8236 2023-10-21T04:31:25.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.731+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.731+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:25.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.736+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.736+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:25.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.737+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:25.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:31:25.737+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.737+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.740+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.740+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.742+05:30 [Info] switched currmeta from 37 -> 38 force true 2023-10-21T04:31:25.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:25.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.744+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:25.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:31:25.744+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.745+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:31:25.745+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:25.745+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:25.745+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:25.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.745+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.746+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.746+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.750+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.750+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:25.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:31:25.750+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.750+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:25.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:25.756+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:25.756+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:25.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:25.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.759+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:25.759+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:25.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:31:25.760+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Warmup took 83.888757ms 2023-10-21T04:31:25.760+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.760+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.760+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 5 WriterThreads 1 2023-10-21T04:31:25.760+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Stopped Billing WUs 2023-10-21T04:31:25.760+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index Index: 10032435514095102923 Partition: 5 2023-10-21T04:31:25.760+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 10032435514095102923, shardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:31:25.760+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 10032435514095102923, partnShardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:31:25.760+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:25.760+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10032435514095102923 2023-10-21T04:31:25.761+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:25.761+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:31:25.761+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:25.761+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:25.761+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:25.761+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:31:25.761+05:30 [Info] StorageMgr::openSnapshot IndexInst:10032435514095102923 Partition:2 No Snapshot Found. 2023-10-21T04:31:25.761+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Adding Nil Snapshot. 2023-10-21T04:31:25.771+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:25.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:31:25.771+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:25.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:25.772+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.74109ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:25.772+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 133.925µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:25.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:31:25.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:25.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:25.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:31:25.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.778+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:25.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.783+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:25.783+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:25.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:31:25.784+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.784+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.786+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.786+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:25.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.791+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:25.791+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:25.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:31:25.791+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Warmup took 74.405006ms 2023-10-21T04:31:25.791+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.791+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.792+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 1 WriterThreads 1 2023-10-21T04:31:25.792+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Stopped Billing WUs 2023-10-21T04:31:25.792+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index Index: 8732144615635911668 Partition: 1 2023-10-21T04:31:25.792+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.792+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 2 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.792+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.792+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.792+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:25.793+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 31507 recovery UsedSpace 8236 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.797+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.797+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:31:25.797+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:31:25.797+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:25.797+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:25.797+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:25.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.798+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.804+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.804+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.805+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:25.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:25.806+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:25.806+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:25.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.806+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.811+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:25.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:31:25.811+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:25.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:25.813+05:30 [Info] switched currmeta from 38 -> 40 force true 2023-10-21T04:31:25.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:31:25.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:25.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:25.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:31:25.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.835+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.835+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.846+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.846+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.848+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.848+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:25.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.852+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:25.852+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:25.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:31:25.852+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.852+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.857+05:30 [Info] switched currmeta from 40 -> 40 force true 2023-10-21T04:31:25.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:25.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.874+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:25.874+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:25.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:31:25.874+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.874+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.874+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Warmup took 82.344561ms 2023-10-21T04:31:25.875+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 2 WriterThreads 1 2023-10-21T04:31:25.875+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Stopped Billing WUs 2023-10-21T04:31:25.875+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index Index: 8732144615635911668 Partition: 2 2023-10-21T04:31:25.876+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.878+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 3 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.879+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.879+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:25.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.885+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.889+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 31507 recovery UsedSpace 8236 2023-10-21T04:31:25.891+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[10032435514095102923:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:31:25.891+05:30 [Info] Indexer::handleMergePartition Source 9447329072527540494 Target 10032435514095102923 2023-10-21T04:31:25.891+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006fb5c20 2023-10-21T04:31:25.891+05:30 [Info] MergePartition: sent async request to update index instance 10032435514095102923 rstate moved to ACTIVE 2023-10-21T04:31:25.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:25.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.894+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:25.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:31:25.894+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.894+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.902+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.077732ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:25.902+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 9447329072527540494, realInst: 10032435514095102923 2023-10-21T04:31:25.904+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:25.904+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:25.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:25.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.916+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:25.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:31:25.917+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:31:25.917+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:25.917+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.917+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.917+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:25.917+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:25.917+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:25.917+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:25.917+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:25.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:31:25.917+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.918+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.918+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.918+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.920+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.920+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.932+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:25.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:31:25.933+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:25.933+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:25.933+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:25.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:25.933+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:25.933+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:25.933+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:25.933+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:25.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:31:25.935+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:25.933+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:25.936+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:25.936+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:25.937+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:25.950+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15864358084358521096,realInstId: 0, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:25.950+05:30 [Info] Indexer::handleRecoverIndex InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:25.950+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 336.78µs 2023-10-21T04:31:25.950+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 376.451µs 2023-10-21T04:31:25.950+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:25.951+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 1 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:25.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:25.951+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:31:25.951+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 11 count: 127 metaSz: 2508 rp [1] sn: 12 count: 127 metaSz: 2513 rp [2] sn: 13 count: 127 metaSz: 2512 rp [3] sn: 360005 count: 127 metaSz: 2512 2023-10-21T04:31:25.951+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 71355 recovery UsedSpace 62676 2023-10-21T04:31:25.952+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:25.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:25.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:31:25.952+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:25.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:25.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:25.950+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15864358084358521096, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:25.953+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 49.27203ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:25.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:31:25.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:25.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:25.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:31:25.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:25.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:25.971+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:31:25.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex started 2023-10-21T04:31:25.971+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.971+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:31:25.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:25.974+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:25.974+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 13 count: 127 metaSz: 2512 rp [1] sn: 360005 count: 127 metaSz: 2512 2023-10-21T04:31:25.974+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 67427 recovery UsedSpace 74654 2023-10-21T04:31:25.977+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.977+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.979+05:30 [Info] switched currmeta from 40 -> 42 force true 2023-10-21T04:31:25.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.980+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.980+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:25.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.986+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:25.986+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:25.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:25.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:25.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:25.987+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:25.987+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:25.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:31:25.987+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.987+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.988+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:25.988+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:25.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:31:25.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:25.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:25.989+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:31:25.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex started 2023-10-21T04:31:25.989+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:31:25.989+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.989+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:25.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:25.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:25.993+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:25.993+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:25.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:31:25.993+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:25.993+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Warmup took 115.09817ms 2023-10-21T04:31:25.993+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:25.994+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 3 WriterThreads 1 2023-10-21T04:31:25.994+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Stopped Billing WUs 2023-10-21T04:31:25.994+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index Index: 8732144615635911668 Partition: 3 2023-10-21T04:31:25.994+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:25.994+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 4 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:25.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:25.994+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.995+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:31:25.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:25.996+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:25.996+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 32768 recovery UsedSpace 8236 2023-10-21T04:31:25.998+05:30 [Info] switched currmeta from 42 -> 42 force true 2023-10-21T04:31:26.000+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:26.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:26.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.005+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:26.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:31:26.005+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.005+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:26.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.011+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:26.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:31:26.011+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:31:26.011+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:26.011+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.011+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.011+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.011+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.011+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:26.011+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:26.011+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:26.012+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:26.012+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:26.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:31:26.012+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.012+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.012+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.012+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.030+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:26.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.031+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Warmup took 79.847239ms 2023-10-21T04:31:26.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:26.031+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:26.031+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:26.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.031+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.031+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 1 WriterThreads 1 2023-10-21T04:31:26.031+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Stopped Billing WUs 2023-10-21T04:31:26.031+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index Index: 15864358084358521096 Partition: 1 2023-10-21T04:31:26.031+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:26.031+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 2 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:26.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:26.032+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 258 2023-10-21T04:31:26.032+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 10 count: 136 metaSz: 2508 rp [1] sn: 11 count: 136 metaSz: 2513 rp [2] sn: 12 count: 136 metaSz: 2512 rp [3] sn: 360005 count: 136 metaSz: 2512 2023-10-21T04:31:26.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:26.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:26.032+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 136 metaSz: 2512 rp [1] sn: 360005 count: 136 metaSz: 2512 2023-10-21T04:31:26.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 68159 recovery UsedSpace 74654 2023-10-21T04:31:26.032+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 72514 recovery UsedSpace 62776 2023-10-21T04:31:26.051+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:26.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:31:26.051+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:26.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:26.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:26.055+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex started 2023-10-21T04:31:26.055+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:26.055+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:26.055+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:31:26.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex started 2023-10-21T04:31:26.055+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.055+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:31:26.055+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9865435099635395, throttleDelayMs (new, old, change): (4, 0, 4) 2023-10-21T04:31:26.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:31:26.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:26.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:26.068+05:30 [Error] ScanCoordinator: Unable to compute index items_count for bucket_%2/idx_partitioned_defer/8732144615635911668 state 0 (Index not found) 2023-10-21T04:31:26.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:31:26.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.078+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.078+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.084+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.084+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:26.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.089+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:26.089+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:26.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:31:26.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.092+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Warmup took 60.132539ms 2023-10-21T04:31:26.092+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 2 WriterThreads 1 2023-10-21T04:31:26.092+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Stopped Billing WUs 2023-10-21T04:31:26.092+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index Index: 15864358084358521096 Partition: 2 2023-10-21T04:31:26.092+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:26.092+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 3 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:26.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:26.092+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 274 2023-10-21T04:31:26.092+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 10 count: 125 metaSz: 2508 rp [1] sn: 11 count: 125 metaSz: 2512 rp [2] sn: 12 count: 125 metaSz: 2512 rp [3] sn: 360005 count: 125 metaSz: 2512 2023-10-21T04:31:26.093+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 73842 recovery UsedSpace 62586 2023-10-21T04:31:26.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:26.093+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:26.093+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 125 metaSz: 2512 rp [1] sn: 360005 count: 125 metaSz: 2512 2023-10-21T04:31:26.093+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 67431 recovery UsedSpace 74654 2023-10-21T04:31:26.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:26.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.095+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:26.095+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:26.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:31:26.095+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Warmup took 101.016051ms 2023-10-21T04:31:26.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.095+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 4 WriterThreads 1 2023-10-21T04:31:26.095+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Stopped Billing WUs 2023-10-21T04:31:26.095+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index Index: 8732144615635911668 Partition: 4 2023-10-21T04:31:26.095+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:26.095+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 5 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:26.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.096+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.096+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 16384 recovery UsedSpace 4118 2023-10-21T04:31:26.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.096+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.096+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 17808 recovery UsedSpace 4118 2023-10-21T04:31:26.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:31:26.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:26.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:26.103+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:31:26.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex started 2023-10-21T04:31:26.103+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.103+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:31:26.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:26.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:26.105+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:31:26.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex started 2023-10-21T04:31:26.105+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:31:26.105+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.105+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:26.107+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.107+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:31:26.107+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.108+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.108+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:26.108+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:26.108+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:26.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.108+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:26.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:26.128+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:26.128+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:26.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.139+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:26.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:31:26.139+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:26.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:26.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:26.139+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Warmup took 47.316314ms 2023-10-21T04:31:26.140+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 3 WriterThreads 1 2023-10-21T04:31:26.140+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Stopped Billing WUs 2023-10-21T04:31:26.140+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index Index: 15864358084358521096 Partition: 3 2023-10-21T04:31:26.140+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:26.140+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 4 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:26.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:31:26.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:26.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:26.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:31:26.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:26.159+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 256 2023-10-21T04:31:26.159+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 10 count: 125 metaSz: 2508 rp [1] sn: 11 count: 125 metaSz: 2512 rp [2] sn: 12 count: 125 metaSz: 2512 rp [3] sn: 360005 count: 125 metaSz: 2512 2023-10-21T04:31:26.159+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 76480 recovery UsedSpace 62582 2023-10-21T04:31:26.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:26.160+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:26.160+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 11 count: 125 metaSz: 2512 rp [1] sn: 12 count: 125 metaSz: 2512 rp [2] sn: 360005 count: 125 metaSz: 2512 2023-10-21T04:31:26.160+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 70566 recovery UsedSpace 66440 2023-10-21T04:31:26.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.164+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.164+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:31:26.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:26.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:26.170+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:31:26.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex started 2023-10-21T04:31:26.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.170+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.170+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.170+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.170+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:31:26.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:26.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:26.171+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:31:26.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex started 2023-10-21T04:31:26.171+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:31:26.171+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.171+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:26.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.177+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:26.177+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:26.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:31:26.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:26.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:26.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:26.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:31:26.181+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Warmup took 85.505298ms 2023-10-21T04:31:26.181+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 5 WriterThreads 1 2023-10-21T04:31:26.181+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Stopped Billing WUs 2023-10-21T04:31:26.181+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index Index: 8732144615635911668 Partition: 5 2023-10-21T04:31:26.181+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:26.181+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 6 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:26.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.182+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.182+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 32768 recovery UsedSpace 8236 2023-10-21T04:31:26.182+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.182+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.182+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.182+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:31:26.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:26.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.201+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:26.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:26.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.201+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:26.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:31:26.201+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:31:26.201+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:26.201+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:26.201+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:26.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.202+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.202+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Warmup took 61.500161ms 2023-10-21T04:31:26.202+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 4 WriterThreads 1 2023-10-21T04:31:26.202+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Stopped Billing WUs 2023-10-21T04:31:26.202+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index Index: 15864358084358521096 Partition: 4 2023-10-21T04:31:26.202+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:26.202+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 5 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:26.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:26.203+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:31:26.203+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 123 metaSz: 2512 rp [1] sn: 360005 count: 123 metaSz: 2512 2023-10-21T04:31:26.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:26.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:26.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 123 metaSz: 2512 rp [1] sn: 360005 count: 123 metaSz: 2512 2023-10-21T04:31:26.203+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 61770 recovery UsedSpace 78750 2023-10-21T04:31:26.203+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 70414 recovery UsedSpace 74654 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:26.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:26.211+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:26.211+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:26.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.211+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:31:26.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:26.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:26.215+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:31:26.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex started 2023-10-21T04:31:26.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:31:26.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:26.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:26.216+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:31:26.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex started 2023-10-21T04:31:26.216+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:31:26.216+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.216+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:26.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:31:26.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:26.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:26.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:26.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:31:26.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:31:26.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:31:26.259+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.259+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:31:26.261+05:30 [Info] switched currmeta from 42 -> 43 force true 2023-10-21T04:31:26.263+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Warmup took 60.466746ms 2023-10-21T04:31:26.263+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 5 WriterThreads 1 2023-10-21T04:31:26.263+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Stopped Billing WUs 2023-10-21T04:31:26.263+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index Index: 15864358084358521096 Partition: 5 2023-10-21T04:31:26.263+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:26.264+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 6 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:26.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.267+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.267+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:26.268+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:31:26.268+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 10 count: 122 metaSz: 2508 rp [1] sn: 11 count: 122 metaSz: 2511 rp [2] sn: 12 count: 122 metaSz: 2512 rp [3] sn: 360005 count: 122 metaSz: 2512 2023-10-21T04:31:26.268+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 78541 recovery UsedSpace 62322 2023-10-21T04:31:26.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:26.269+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:26.269+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 122 metaSz: 2512 rp [1] sn: 360005 count: 122 metaSz: 2512 2023-10-21T04:31:26.269+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 73780 recovery UsedSpace 74654 2023-10-21T04:31:26.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.272+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:31:26.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:26.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:26.275+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:31:26.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex started 2023-10-21T04:31:26.275+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.275+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:26.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.275+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:26.275+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:26.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:31:26.276+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.276+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:31:26.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:26.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:26.278+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:31:26.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex started 2023-10-21T04:31:26.279+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:31:26.280+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.280+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:26.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.281+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:26.281+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:26.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:31:26.281+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Warmup took 99.332801ms 2023-10-21T04:31:26.281+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 6 WriterThreads 1 2023-10-21T04:31:26.281+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Stopped Billing WUs 2023-10-21T04:31:26.281+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index Index: 8732144615635911668 Partition: 6 2023-10-21T04:31:26.281+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:26.281+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 7 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:26.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.282+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.282+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.282+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.282+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 32768 recovery UsedSpace 8236 2023-10-21T04:31:26.283+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:31:26.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:26.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.288+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:26.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:31:26.288+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.288+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:26.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.291+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:26.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:31:26.291+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:31:26.291+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:26.291+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:26.291+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:26.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.292+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.292+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.296+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:45958:172.31.5.112:12000] 2023-10-21T04:31:26.300+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Warmup took 36.424821ms 2023-10-21T04:31:26.300+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 6 WriterThreads 1 2023-10-21T04:31:26.300+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Stopped Billing WUs 2023-10-21T04:31:26.300+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index Index: 15864358084358521096 Partition: 6 2023-10-21T04:31:26.301+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:26.301+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 7 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:26.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:26.301+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:31:26.301+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 136 metaSz: 2512 rp [1] sn: 360005 count: 136 metaSz: 2512 2023-10-21T04:31:26.302+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 75030 recovery UsedSpace 74654 2023-10-21T04:31:26.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:26.304+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:31:26.304+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 10 count: 136 metaSz: 2508 rp [1] sn: 11 count: 136 metaSz: 2513 rp [2] sn: 12 count: 136 metaSz: 2512 rp [3] sn: 360005 count: 136 metaSz: 2512 2023-10-21T04:31:26.304+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 84952 recovery UsedSpace 62770 2023-10-21T04:31:26.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:31:26.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:26.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:26.330+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:31:26.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex started 2023-10-21T04:31:26.330+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.330+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.331+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:26.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:31:26.331+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:26.331+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.331+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.331+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:26.331+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:26.331+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:26.332+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:26.332+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:26.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:31:26.332+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.332+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.332+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.332+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:31:26.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:26.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:26.341+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:31:26.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex started 2023-10-21T04:31:26.341+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:31:26.343+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.343+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.350+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:26.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:31:26.350+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:26.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:26.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:26.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:31:26.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:31:26.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:31:26.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:31:26.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.374+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Warmup took 73.644636ms 2023-10-21T04:31:26.375+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 7 WriterThreads 1 2023-10-21T04:31:26.375+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Stopped Billing WUs 2023-10-21T04:31:26.375+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index Index: 15864358084358521096 Partition: 7 2023-10-21T04:31:26.375+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:26.375+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 8 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:26.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.376+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.376+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:26.377+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:31:26.377+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 106 metaSz: 2511 rp [1] sn: 360005 count: 106 metaSz: 2511 2023-10-21T04:31:26.377+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 62856 recovery UsedSpace 78750 2023-10-21T04:31:26.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.378+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.379+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:26.380+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:31:26.380+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 12 count: 106 metaSz: 2511 rp [1] sn: 360005 count: 106 metaSz: 2511 2023-10-21T04:31:26.381+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 72225 recovery UsedSpace 74654 2023-10-21T04:31:26.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:26.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.388+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:26.389+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:26.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:31:26.389+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.389+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:26.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.390+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:26.390+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:26.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:31:26.391+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Warmup took 109.021105ms 2023-10-21T04:31:26.391+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.391+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.391+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 7 WriterThreads 1 2023-10-21T04:31:26.391+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Stopped Billing WUs 2023-10-21T04:31:26.391+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index Index: 8732144615635911668 Partition: 7 2023-10-21T04:31:26.391+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:26.391+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 8 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:26.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.392+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.392+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:31:26.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.392+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:31:26.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:31:26.392+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:31:26.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:26.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:26.392+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:31:26.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex started 2023-10-21T04:31:26.393+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.393+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:26.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.400+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:26.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:31:26.400+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.400+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:26.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.402+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:26.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:31:26.403+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:31:26.405+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:31:26.404+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.404+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.405+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:26.406+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:26.406+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:26.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.406+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:31:26.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:26.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:26.417+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:31:26.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex started 2023-10-21T04:31:26.417+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:31:26.419+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.419+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.422+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.422+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:26.422+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:31:26.423+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:26.423+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:26.423+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:26.424+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:26.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:26.424+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:26.425+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:26.425+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:26.425+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:26.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:31:26.426+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:26.426+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:26.426+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:26.427+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:26.437+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:26.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:26.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:31:26.437+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:26.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:26.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:26.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:31:26.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:26.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:26.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:31:26.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.462+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.462+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.467+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Warmup took 91.55424ms 2023-10-21T04:31:26.467+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 8 WriterThreads 1 2023-10-21T04:31:26.468+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Stopped Billing WUs 2023-10-21T04:31:26.468+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index Index: 15864358084358521096 Partition: 8 2023-10-21T04:31:26.468+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15864358084358521096, shardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:31:26.468+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15864358084358521096, partnShardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:31:26.469+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:26.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:26.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.470+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:31:26.470+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:31:26.470+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15864358084358521096 2023-10-21T04:31:26.471+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:26.471+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:31:26.473+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:26.473+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:26.474+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:26.474+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15864358084358521096 Partitions [5 6 7 8 1 2 3 4] 2023-10-21T04:31:26.476+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:1 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:31:26.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:26.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.478+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:26.478+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:26.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:31:26.478+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.478+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.479+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:2 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:31:26.482+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.155941ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:26.485+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 111.382µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:26.486+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:3 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:31:26.495+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:4 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:31:26.499+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:5 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:31:26.500+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.502+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:26.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.508+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:26.508+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:26.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:31:26.508+05:30 [Info] switched currmeta from 43 -> 45 force true 2023-10-21T04:31:26.509+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Warmup took 117.468209ms 2023-10-21T04:31:26.509+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.509+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.510+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 8 WriterThreads 1 2023-10-21T04:31:26.510+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Stopped Billing WUs 2023-10-21T04:31:26.510+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index Index: 8732144615635911668 Partition: 8 2023-10-21T04:31:26.511+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8732144615635911668, shardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] err: 2023-10-21T04:31:26.510+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:6 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:31:26.524+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:7 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:31:26.534+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.534+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.534+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:8 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:31:26.535+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8732144615635911668, partnShardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]], err: 2023-10-21T04:31:26.536+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:26.542+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8732144615635911668 2023-10-21T04:31:26.542+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8732144615635911668 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:26.543+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:31:26.543+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:26.546+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8732144615635911668 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:26.549+05:30 [Info] switched currmeta from 45 -> 45 force true 2023-10-21T04:31:26.547+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:26.550+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:31:26.550+05:30 [Info] StorageMgr::openSnapshot IndexInst:8732144615635911668 Partition:3 No Snapshot Found. 2023-10-21T04:31:26.550+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Adding Nil Snapshot. 2023-10-21T04:31:26.558+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.910258ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:26.558+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 104.529µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:26.599+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.599+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.612+05:30 [Info] switched currmeta from 45 -> 47 force true 2023-10-21T04:31:26.674+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.674+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.678+05:30 [Info] switched currmeta from 47 -> 48 force true 2023-10-21T04:31:26.682+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45992:172.31.5.112:12000] 2023-10-21T04:31:26.696+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.696+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.699+05:30 [Info] switched currmeta from 48 -> 48 force true 2023-10-21T04:31:26.723+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8732144615635911668:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:31:26.723+05:30 [Info] Indexer::handleMergePartition Source 14527226347372019068 Target 8732144615635911668 2023-10-21T04:31:26.723+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007ef8420 2023-10-21T04:31:26.724+05:30 [Info] MergePartition: sent async request to update index instance 8732144615635911668 rstate moved to ACTIVE 2023-10-21T04:31:26.725+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:45990:172.31.5.112:12000] 2023-10-21T04:31:26.734+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.675753ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:26.734+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 14527226347372019068, realInst: 8732144615635911668 2023-10-21T04:31:26.736+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12824340686309573428, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:26.736+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12824340686309573428, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:26.763+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12266944006962282785,realInstId: 0, indexDefn DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:31:26.763+05:30 [Info] Indexer::handleRecoverIndex InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:31:26.763+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 289.142µs 2023-10-21T04:31:26.763+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 336.019µs 2023-10-21T04:31:26.764+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12266944006962282785 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:26.764+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12266944006962282785 PartitionId: 0 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:26.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:31:26.764+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1056128 memSz 2042 2023-10-21T04:31:26.764+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 17 count: 1000 metaSz: 2521 rp [1] sn: 18 count: 1000 metaSz: 2530 2023-10-21T04:31:26.765+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 117920 recovery UsedSpace 47055 2023-10-21T04:31:26.766+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.766+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.766+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12266944006962282785, indexDefn DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:26.767+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 31.279409ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:26.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:26.772+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1055942 memSz 868 2023-10-21T04:31:26.772+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 17 count: 1000 metaSz: 2521 rp [1] sn: 18 count: 1000 metaSz: 2530 2023-10-21T04:31:26.774+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 122181 recovery UsedSpace 43753 2023-10-21T04:31:26.774+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:26.779+05:30 [Info] switched currmeta from 48 -> 50 force true 2023-10-21T04:31:26.786+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.786+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.788+05:30 [Info] switched currmeta from 50 -> 50 force true 2023-10-21T04:31:26.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:31:26.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:31:26.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:31:26.789+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:31:26.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex started 2023-10-21T04:31:26.789+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.789+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:31:26.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:26.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:26.797+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:31:26.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex started 2023-10-21T04:31:26.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:26.797+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12266944006962282785 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:31:26.797+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:26.822+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12266944006962282785 PartitionId 0 Warmup took 58.272843ms 2023-10-21T04:31:26.823+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12266944006962282785 partitionId 0 WriterThreads 4 2023-10-21T04:31:26.823+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12266944006962282785 PartitionId 0 Stopped Billing WUs 2023-10-21T04:31:26.824+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12266944006962282785 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index Index: 12266944006962282785 Partition: 0 2023-10-21T04:31:26.824+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12266944006962282785, shardIdMap: map[0:[17406544375239214575 10170029854712240877]] err: 2023-10-21T04:31:26.824+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12266944006962282785, partnShardIdMap: map[0:[17406544375239214575 10170029854712240877]], err: 2023-10-21T04:31:26.825+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:26.827+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12266944006962282785 2023-10-21T04:31:26.827+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:26.827+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:31:26.828+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:26.828+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:26.828+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:26.829+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12266944006962282785 Partitions [0] 2023-10-21T04:31:26.831+05:30 [Info] StorageMgr::openSnapshot IndexInst:12266944006962282785 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:31:26.834+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.688577ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:26.834+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 72.798µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:26.861+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.861+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.866+05:30 [Info] switched currmeta from 50 -> 52 force true 2023-10-21T04:31:26.881+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:26.881+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:26.884+05:30 [Info] switched currmeta from 52 -> 52 force true 2023-10-21T04:31:26.996+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46000:172.31.5.112:12000] 2023-10-21T04:31:27.003+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[15864358084358521096:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:27.004+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 17797267771846799851, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:27.004+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 17797267771846799851, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:27.031+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3023333489681759512,realInstId: 0, indexDefn DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:31:27.031+05:30 [Info] Indexer::handleRecoverIndex InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:31:27.032+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 294.241µs 2023-10-21T04:31:27.032+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 338.565µs 2023-10-21T04:31:27.032+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3023333489681759512 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:27.032+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3023333489681759512 PartitionId: 0 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:27.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:31:27.032+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1056202 memSz 2460 2023-10-21T04:31:27.032+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 20 count: 1000 metaSz: 2517 rp [1] sn: 360005 count: 1000 metaSz: 2516 2023-10-21T04:31:27.033+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 132522 recovery UsedSpace 64602 2023-10-21T04:31:27.033+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3023333489681759512, indexDefn DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:27.033+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 28.97941ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:27.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:27.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1055988 memSz 882 2023-10-21T04:31:27.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 20 count: 1000 metaSz: 2517 rp [1] sn: 360005 count: 1000 metaSz: 2516 2023-10-21T04:31:27.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 142300 recovery UsedSpace 60365 2023-10-21T04:31:27.039+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:27.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9848196425680529, throttleDelayMs (new, old, change): (7, 4, 3) 2023-10-21T04:31:27.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:31:27.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:31:27.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:31:27.063+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:31:27.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex started 2023-10-21T04:31:27.063+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:27.063+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:27.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:31:27.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:27.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:27.065+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:31:27.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex started 2023-10-21T04:31:27.065+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3023333489681759512 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:31:27.066+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:27.066+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:27.090+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.090+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.093+05:30 [Info] switched currmeta from 52 -> 53 force true 2023-10-21T04:31:27.105+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3023333489681759512 PartitionId 0 Warmup took 72.848191ms 2023-10-21T04:31:27.105+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3023333489681759512 partitionId 0 WriterThreads 4 2023-10-21T04:31:27.105+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3023333489681759512 PartitionId 0 Stopped Billing WUs 2023-10-21T04:31:27.105+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3023333489681759512 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index Index: 3023333489681759512 Partition: 0 2023-10-21T04:31:27.105+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3023333489681759512, shardIdMap: map[0:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:31:27.105+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3023333489681759512, partnShardIdMap: map[0:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:31:27.105+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:27.106+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3023333489681759512 2023-10-21T04:31:27.106+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:27.106+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:31:27.106+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:27.106+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:27.106+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:27.107+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3023333489681759512 Partitions [0] 2023-10-21T04:31:27.108+05:30 [Info] StorageMgr::openSnapshot IndexInst:3023333489681759512 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:31:27.115+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.42318ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:27.115+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 90.66µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:27.136+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.136+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.143+05:30 [Info] switched currmeta from 53 -> 54 force true 2023-10-21T04:31:27.166+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46004:172.31.5.112:12000] 2023-10-21T04:31:27.195+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.195+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.200+05:30 [Info] switched currmeta from 54 -> 55 force true 2023-10-21T04:31:27.234+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46006:172.31.5.112:12000] 2023-10-21T04:31:27.255+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46010:172.31.5.112:12000] 2023-10-21T04:31:27.290+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46014:172.31.5.112:12000] 2023-10-21T04:31:27.294+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46012:172.31.5.112:12000] 2023-10-21T04:31:27.329+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46016:172.31.5.112:12000] 2023-10-21T04:31:27.342+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:46018:172.31.5.112:12000] 2023-10-21T04:31:27.354+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842790443813331, Old rollbacktime: 0 2023-10-21T04:31:27.428+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.428+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.432+05:30 [Info] switched currmeta from 55 -> 56 force true 2023-10-21T04:31:27.705+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.705+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.709+05:30 [Info] switched currmeta from 56 -> 57 force true 2023-10-21T04:31:27.778+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8965923595543532942:INDEX_STATE_RECOVERED 12266944006962282785:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:27.779+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [8910459001587546151 12824340686309573428] belonging to collectionId: c, ttid: ShardToken75:7c:cc:96:58:f4:7c:a8. Initiating build 2023-10-21T04:31:27.811+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.811+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.811+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [8965923595543532942 12266944006962282785] &{2} 2023-10-21T04:31:27.812+05:30 [Info] Indexer::handleBuildRecoveredIndexes [8965923595543532942 12266944006962282785] 2023-10-21T04:31:27.812+05:30 [Info] bulkUpdateRState: Index instance 8965923595543532942 rstate moved to PENDING 2023-10-21T04:31:27.812+05:30 [Info] bulkUpdateRState: Index instance 12266944006962282785 rstate moved to PENDING 2023-10-21T04:31:27.812+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [8965923595543532942 12266944006962282785] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:31:27.813+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:27.814+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:27.814+05:30 [Info] switched currmeta from 57 -> 59 force true 2023-10-21T04:31:27.814+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:27.823+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.823+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.825+05:30 [Info] switched currmeta from 59 -> 59 force true 2023-10-21T04:31:27.838+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:31:27.838+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:27.839+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:31:27.839+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:31:27.839+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:27.839+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:27.839+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:31:27.840+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:31:27.840+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:31:27.840+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId 21066 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:31:27.841+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:31:27.841+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:31:27.841+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:31:27.841+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 12266944006962282785 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:31:27.841+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c1:2]] 2023-10-21T04:31:27.841+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 8965923595543532942 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:31:27.841+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:31:27.841+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=17,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 21066 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:31:27.842+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:31:27.842+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:31:27.848+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:31:27.844+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:27.851+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [8965923595543532942 12266944006962282785] 2023-10-21T04:31:27.851+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 70.668557ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:27.852+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:31:27.853+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 41.079695ms 2023-10-21T04:31:27.853+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[8965923595543532942:true 12266944006962282785:true] 2023-10-21T04:31:27.851+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 38.880318ms 2023-10-21T04:31:27.861+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.711378ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:27.879+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.879487ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:27.900+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:57440" doAuth() ... 2023-10-21T04:31:27.901+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:57440" auth successful 2023-10-21T04:31:27.901+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:57440" +1 2023-10-21T04:31:27.907+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:27.932+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.932+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.946+05:30 [Info] switched currmeta from 59 -> 64 force true 2023-10-21T04:31:27.956+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:27.956+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:27.959+05:30 [Info] switched currmeta from 64 -> 64 force true 2023-10-21T04:31:28.029+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.039+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.043+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3023333489681759512:INDEX_STATE_RECOVERED 15864358084358521096:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:28.043+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [16961575949607441659 17797267771846799851] belonging to collectionId: c, ttid: ShardToken2e:2c:44:28:da:2:14:94. Initiating build 2023-10-21T04:31:28.045+05:30 [Info] switched currmeta from 64 -> 65 force true 2023-10-21T04:31:28.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9594282270072328, throttleDelayMs (new, old, change): (8, 7, 1) 2023-10-21T04:31:28.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.074+05:30 [Info] switched currmeta from 65 -> 67 force true 2023-10-21T04:31:28.090+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.090+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.094+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [15864358084358521096 3023333489681759512] &{2} 2023-10-21T04:31:28.097+05:30 [Info] Indexer::handleBuildRecoveredIndexes [15864358084358521096 3023333489681759512] 2023-10-21T04:31:28.097+05:30 [Info] bulkUpdateRState: Index instance 15864358084358521096 rstate moved to PENDING 2023-10-21T04:31:28.097+05:30 [Info] bulkUpdateRState: Index instance 3023333489681759512 rstate moved to PENDING 2023-10-21T04:31:28.097+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [15864358084358521096 3023333489681759512] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:31:28.102+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:28.102+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:28.103+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:28.119+05:30 [Info] switched currmeta from 67 -> 68 force true 2023-10-21T04:31:28.132+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.132+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.135+05:30 [Info] switched currmeta from 68 -> 68 force true 2023-10-21T04:31:28.183+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:31:28.184+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:28.188+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:28.189+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:28.190+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:31:28.192+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:31:28.193+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:28.194+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:28.195+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc007fe5a40 }] 2023-10-21T04:31:28.195+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:28.195+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:28.195+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:31:28.195+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:31:28.195+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId 55765 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:31:28.196+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:28.196+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:28.196+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:31:28.196+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c1:2 bucket_1:_default:c1:1]] 2023-10-21T04:31:28.196+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15864358084358521096 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:31:28.196+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c1:2 bucket_1:_default:c1:2]] 2023-10-21T04:31:28.196+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3023333489681759512 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:31:28.196+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:31:28.196+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=24,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=27,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=20,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=22,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 55765 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:31:28.197+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc007fe5a40 } bucket_1:_default:c1:{0xc008293800 }] 2023-10-21T04:31:28.198+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:28.203+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 106.644626ms 2023-10-21T04:31:28.201+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:31:28.203+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [15864358084358521096 3023333489681759512] 2023-10-21T04:31:28.205+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 162.433368ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:28.204+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 107.151707ms 2023-10-21T04:31:28.206+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[3023333489681759512:true 15864358084358521096:true] 2023-10-21T04:31:28.208+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:31:28.223+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.607984ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:28.240+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.240+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.254+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:28.256+05:30 [Info] switched currmeta from 68 -> 69 force true 2023-10-21T04:31:28.257+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 32.944317ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:28.277+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.277+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.283+05:30 [Info] switched currmeta from 69 -> 70 force true 2023-10-21T04:31:28.457+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.457+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.489+05:30 [Info] switched currmeta from 70 -> 71 force true 2023-10-21T04:31:28.491+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:28.502+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.503+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.538+05:30 [Info] switched currmeta from 71 -> 71 force true 2023-10-21T04:31:28.751+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.759+05:30 [Info] switched currmeta from 71 -> 72 force true 2023-10-21T04:31:28.770+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.770+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.775+05:30 [Info] switched currmeta from 72 -> 73 force true 2023-10-21T04:31:28.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 274143649547516 29 21066 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:31:28.820+05:30 [Info] NewTsVbuuidCached: bucket_%2, 64 TsVbuuidPoolMap is updated from: map[] to: map[64:0xc00a9238c0] 2023-10-21T04:31:28.822+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 162049908746825 25 21066 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:31:28.824+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 206082810619393 24 21066 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:28.831+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 120349894144442 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.837+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:28.837+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:28.838+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 212967951483349 28 21066 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:28.838+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 78446412189949 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.838+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 143687065777286 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.841+05:30 [Info] switched currmeta from 73 -> 74 force true 2023-10-21T04:31:28.841+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 263088038522959 18 21066 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:28.848+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 127746572208200 19 21066 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:28.848+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 136568163328749 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.853+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 58733680850130 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.853+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 188740876924377 22 21066 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:28.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 261951617336488 35 21066 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:31:28.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 116493903186721 22 21066 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:28.858+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 173457131919400 12 21066 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:31:28.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 274158831830257 20 21066 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:28.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 245247859472006 17 21066 172.31.5.112:9000. HWT [17-17,17]. 2023-10-21T04:31:28.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 206088069380492 13 21066 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:31:28.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 250426459207023 28 21066 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:28.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 125696667341112 27 21066 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:28.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 277338544220217 21 21066 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:28.865+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 34510046292135 16 21066 172.31.5.112:9000. HWT [16-16,16]. 2023-10-21T04:31:28.869+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 35716737784090 24 21066 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:28.869+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 191500187207978 21 21066 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:28.875+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 118920785117980 21 21066 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:28.875+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 106231614376481 21 21066 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:28.877+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 137256376714252 22 21066 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:28.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 146375746634523 17 21066 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:28.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5906787184229 20 21066 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:28.885+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 241391494503904 28 21066 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:28.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 112482982607415 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.888+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 121450709505223 25 21066 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:28.888+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 41101832875819 19 21066 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:28.891+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 216238759475962 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.893+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 127702263378100 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 100008922352935 17 21066 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:28.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 168446351781198 25 21066 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:28.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 169268048765249 19 21066 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:28.900+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 245900669851618 19 21066 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:28.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 106297365959267 27 21066 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:28.909+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 88718752743571 25 21066 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:28.909+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 149033089785397 28 21066 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:28.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 160399698726237 27 21066 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:28.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 39045953044628 18 21066 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:28.916+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:28.917+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 217237947818243 20 21066 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:28.919+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 110286285845649 31 21066 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:31:28.921+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 162870112426804 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.923+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 264001907139144 22 21066 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:28.923+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 132470209691538 26 21066 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:28.925+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 63737850123332 19 21066 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:28.928+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 266604622117081 26 21066 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:28.929+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 32274016651656 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.931+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 84581802233088 22 21066 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:28.933+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 225662991941187 26 21066 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:28.935+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 241607113601772 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.937+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 272523968790649 20 21066 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:28.939+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 77598347829810 19 21066 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:28.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 97677619676504 31 21066 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:31:28.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 136038310030103 28 21066 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:28.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 197957220074784 24 21066 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:28.950+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 105415986286631 18 21066 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:28.950+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 72072509155168 23 21066 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:28.952+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 27479155597282 29 21066 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:31:28.956+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 161870047388121 25 21066 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:28.960+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 InstanceIds [12266944006962282785 8965923595543532942] 2023-10-21T04:31:28.960+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 21066 2023-10-21T04:31:28.960+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 21066 2023-10-21T04:31:28.960+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:31:28.960+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:31:28.960+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 12266944006962282785 INDEX_STATE_CATCHUP 2023-10-21T04:31:28.960+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 8965923595543532942 INDEX_STATE_CATCHUP 2023-10-21T04:31:28.961+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 12266944006962282785 true 2023-10-21T04:31:28.961+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 8965923595543532942 true 2023-10-21T04:31:28.961+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 12266944006962282785 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 21066 BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=17,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] 2023-10-21T04:31:28.961+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} , fetchKVSeq false 2023-10-21T04:31:28.961+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:31:28.961+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:31:28.961+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 21066 STREAM_ACTIVE 2023-10-21T04:31:28.961+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 21066 2023-10-21T04:31:28.961+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2:_default:c1 21066 2023-10-21T04:31:28.961+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:31:28.961+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:28.962+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:28.962+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:28.962+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:28.963+05:30 [Info] send progress stats to clients 2023-10-21T04:31:28.963+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 21066 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:28.963+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 12266944006962282785 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:31:28.963+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 8965923595543532942 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:31:28.963+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:31:28.963+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:31:28.963+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 0 New 0 2023-10-21T04:31:28.963+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 21066 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:28.964+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_default:c1:{0xc008293800 }] 2023-10-21T04:31:28.964+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:28.964+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:28.964+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:28.964+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:28.964+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2:_default:c1 2023-10-21T04:31:28.965+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:31:28.971+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.524076ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:28.987+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.479679ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:29.004+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.005+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.047+05:30 [Info] switched currmeta from 74 -> 76 force true 2023-10-21T04:31:29.065+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.065+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.066+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9380193261536605, throttleDelayMs (new, old, change): (7, 8, -1) 2023-10-21T04:31:29.068+05:30 [Info] switched currmeta from 76 -> 76 force true 2023-10-21T04:31:29.259+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:29.305+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 1949031200982 23 55765 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:29.307+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 245782287380001 24 55765 172.31.5.112:9000. HWT [24-24,24]. 2023-10-21T04:31:29.313+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 52522903947059 28 55765 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:29.318+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 93079113162334 18 55765 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:29.318+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 205201940883916 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.320+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 244791485011286 23 55765 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:29.324+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 167580527121081 23 55765 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:29.327+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 177815184127237 17 55765 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:29.327+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 54236768606397 20 55765 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:29.329+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 134409829206048 26 55765 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:29.331+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 160381752747972 18 55765 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:29.331+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 58522600746999 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.335+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 142387448675642 30 55765 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:31:29.335+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 90115353725987 21 55765 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:29.337+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 161230548567164 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.341+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 85302001459467 21 55765 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:29.343+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 213108110408291 23 55765 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:29.344+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 123957315902602 27 55765 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:31:29.344+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 163460438181870 34 55765 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:31:29.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 150690956328758 26 55765 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:29.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 147946970349859 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 151273673850546 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.354+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 258671969827141 19 55765 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:29.358+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 19989312661772 15 55765 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:31:29.358+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 114823206239184 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 53829314402679 32 55765 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:31:29.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 68847799765163 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 232028101514735 19 55765 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:29.372+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 8305763409275 20 55765 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:29.373+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 25510109770064 28 55765 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:29.375+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 124811039198764 18 55765 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:29.379+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 207489375637709 12 55765 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:31:29.379+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 107407811917992 20 55765 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:29.379+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 100746255343609 19 55765 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:29.386+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 228857796432725 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.387+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 33310272161722 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.387+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 117298435979365 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.387+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 203503670520010 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.390+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 47156288300713 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.399+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 158357249947806 21 55765 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:29.399+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 104136033125373 29 55765 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:31:29.399+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 236904976023311 22 55765 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:29.399+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 124310544527292 21 55765 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:29.406+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 196269910725961 26 55765 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:29.412+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 65885584526743 20 55765 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:29.412+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 246959094981553 34 55765 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:31:29.412+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 276463747117997 22 55765 172.31.5.112:9000. HWT [22-22,22]. 2023-10-21T04:31:29.415+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 172365063201867 21 55765 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:29.422+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 79077905592013 17 55765 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:29.422+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 212884822297038 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.422+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 142324128597367 22 55765 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:29.422+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 246582249071123 20 55765 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:29.422+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 213334533059126 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.425+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 219528837936411 25 55765 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:29.425+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 254944616021293 24 55765 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:29.430+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 80145536651832 17 55765 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:29.430+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 25511316571199 16 55765 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:31:29.430+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 55145461136105 16 55765 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:31:29.432+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 194568213794435 29 55765 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:31:29.434+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 279733370822238 23 55765 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:29.436+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 223881084081370 17 55765 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:29.439+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 36300629322478 32 55765 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:31:29.451+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 106620692996001 23 55765 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:29.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 25657368013413 17 55765 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:29.461+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 InstanceIds [12266944006962282785 8965923595543532942 3023333489681759512 15864358084358521096] 2023-10-21T04:31:29.461+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 55765 2023-10-21T04:31:29.461+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 55765 2023-10-21T04:31:29.461+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:31:29.461+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:31:29.461+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 15864358084358521096 INDEX_STATE_CATCHUP 2023-10-21T04:31:29.461+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 3023333489681759512 INDEX_STATE_CATCHUP 2023-10-21T04:31:29.461+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 15864358084358521096 true 2023-10-21T04:31:29.461+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 3023333489681759512 true 2023-10-21T04:31:29.462+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 15864358084358521096 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 55765 BuildTS: [0=23,1=24,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=27,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=20,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=22,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] 2023-10-21T04:31:29.462+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} , fetchKVSeq false 2023-10-21T04:31:29.462+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:31:29.462+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 55765 STREAM_ACTIVE 2023-10-21T04:31:29.462+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 55765 2023-10-21T04:31:29.462+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1:_default:c1 55765 2023-10-21T04:31:29.463+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:29.463+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:29.463+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:29.463+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:29.464+05:30 [Info] send progress stats to clients 2023-10-21T04:31:29.464+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 55765 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:29.464+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15864358084358521096 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:31:29.464+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3023333489681759512 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:31:29.464+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:31:29.464+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:31:29.464+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:29.464+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 55765 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:29.468+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:31:29.469+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:29.469+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:29.469+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:29.470+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:29.470+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:31:29.470+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:57440: use of closed network connection 2023-10-21T04:31:29.470+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:57440" exit: read tcp4 127.0.0.1:9121->127.0.0.1:57440: use of closed network connection 2023-10-21T04:31:29.470+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:57440" closed ! 2023-10-21T04:31:29.470+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 3 120349894144442 23 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 0 274143649547516 29 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 1 162049908746825 25 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 2 206082810619393 24 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1:_default:c1 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 7 78446412189949 23 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 8 263088038522959 18 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 9 136568163328749 23 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 4 143687065777286 23 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 5 212967951483349 28 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 6 116493903186721 22 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 12 58733680850130 23 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 10 261951617336488 35 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_%2:_default:c1 11 127746572208200 19 2023-10-21T04:31:29.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:31:29.471+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:31:29.476+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.096725ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:29.496+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.69854ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:29.585+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.593+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.619+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1]. Err projector.topicMissing 2023-10-21T04:31:29.633+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:31:29.633+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating projector.topicMissing As Success 2023-10-21T04:31:29.633+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 55765 2023-10-21T04:31:29.633+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1:_default:c1 2023-10-21T04:31:29.633+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:31:29.633+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:29.634+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:31:29.634+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:31:29.634+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:29.634+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:29.634+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:31:29.634+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:31:29.634+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:31:29.634+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 21886 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:31:29.634+05:30 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2023-10-21T04:31:29.635+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:31:29.635+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:31:29.635+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:31:29.635+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:31:29.635+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 21886 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:31:29.637+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:31:29.637+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:31:29.622+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:31:29.638+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 21066 2023-10-21T04:31:29.623+05:30 [Info] switched currmeta from 76 -> 80 force true 2023-10-21T04:31:29.651+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:31:29.669+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:31:29.669+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:29.670+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:29.670+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:29.672+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:31:29.672+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:31:29.672+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:29.672+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:29.672+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc00b21c600 }] 2023-10-21T04:31:29.673+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:29.673+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:29.673+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:31:29.673+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:31:29.673+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 36450 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:31:29.673+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:29.673+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:29.673+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:31:29.673+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1 bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:31:29.673+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:31:29.674+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:31:29.674+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 36450 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:31:29.675+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc00b21c840 } bucket_1:{0xc00b21c600 }] 2023-10-21T04:31:29.684+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:31:29.686+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.688+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.690+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:31:29.690+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.695+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.696+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.694+05:30 [Info] switched currmeta from 80 -> 80 force true 2023-10-21T04:31:29.696+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.697+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.698+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.698+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.698+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.698+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.698+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:31:29.744+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:53336" doAuth() ... 2023-10-21T04:31:29.744+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:53336" auth successful 2023-10-21T04:31:29.748+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:53336" +1 2023-10-21T04:31:29.805+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.807+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.809+05:30 [Info] switched currmeta from 80 -> 82 force true 2023-10-21T04:31:29.824+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.825+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.828+05:30 [Info] switched currmeta from 82 -> 82 force true 2023-10-21T04:31:29.897+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.897+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.902+05:30 [Info] switched currmeta from 82 -> 83 force true 2023-10-21T04:31:29.915+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:29.915+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:29.926+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:29.932+05:30 [Info] switched currmeta from 83 -> 84 force true 2023-10-21T04:31:29.962+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:31:30.074+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.074+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.078+05:30 [Info] switched currmeta from 84 -> 85 force true 2023-10-21T04:31:30.112+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.112+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.127+05:30 [Info] switched currmeta from 85 -> 86 force true 2023-10-21T04:31:30.202+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.202+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.214+05:30 [Info] switched currmeta from 86 -> 88 force true 2023-10-21T04:31:30.232+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.233+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.239+05:30 [Info] switched currmeta from 88 -> 88 force true 2023-10-21T04:31:30.267+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:30.267+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3023333489681759512 Defn: DefnId: 17797267771846799851 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00508c2a0 2023-10-21T04:31:30.268+05:30 [Info] handleUpdateIndexRState: Index instance 3023333489681759512 rstate moved to ACTIVE 2023-10-21T04:31:30.268+05:30 [Info] Indexer::handleMergePartition Source 5145547347365683414 Target 15864358084358521096 2023-10-21T04:31:30.268+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005944840 2023-10-21T04:31:30.268+05:30 [Info] MergePartition: sent async request to update index instance 15864358084358521096 rstate moved to ACTIVE 2023-10-21T04:31:30.281+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.379465ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:30.283+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 3464125101225762400. Retrying... 2023-10-21T04:31:30.283+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[3023333489681759512:true 15864358084358521096:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:31:30.293+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.571231ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:30.293+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 5145547347365683414, realInst: 15864358084358521096 2023-10-21T04:31:30.296+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.296+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.296+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:30.296+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:31:30.301+05:30 [Info] switched currmeta from 88 -> 90 force true 2023-10-21T04:31:30.322+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3464125101225762400,realInstId: 0, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:31:30.323+05:30 [Info] Indexer::handleRecoverIndex InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:31:30.324+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 668.59µs 2023-10-21T04:31:30.324+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 960.256µs 2023-10-21T04:31:30.325+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3464125101225762400 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[16787445029868804812 2925063509317026816]] 2023-10-21T04:31:30.325+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3464125101225762400 PartitionId: 0 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:31:30.325+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3464125101225762400, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:30.326+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.905369ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:30.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:31:30.335+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:31:30.329+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.336+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.335+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 8 count: 5 metaSz: 2481 rp [1] sn: 360006 count: 6 metaSz: 2481 2023-10-21T04:31:30.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 66694 recovery UsedSpace 74828 2023-10-21T04:31:30.342+05:30 [Info] switched currmeta from 90 -> 91 force true 2023-10-21T04:31:30.342+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:30.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:31:30.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:31:30.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:31:30.349+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:31:30.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex started 2023-10-21T04:31:30.349+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:31:30.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:30.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:30.353+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.353+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.356+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Warmup took 30.650369ms 2023-10-21T04:31:30.356+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3464125101225762400 partitionId 0 WriterThreads 4 2023-10-21T04:31:30.357+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3464125101225762400 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index Index: 3464125101225762400 Partition: 0 2023-10-21T04:31:30.357+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3464125101225762400, shardIdMap: map[0:[16787445029868804812]] err: 2023-10-21T04:31:30.357+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3464125101225762400, partnShardIdMap: map[0:[16787445029868804812]], err: 2023-10-21T04:31:30.357+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:30.357+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3464125101225762400 2023-10-21T04:31:30.357+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:30.357+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:31:30.357+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:30.357+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:30.358+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:30.359+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3464125101225762400 Partitions [0] 2023-10-21T04:31:30.360+05:30 [Info] StorageMgr::openSnapshot IndexInst:3464125101225762400 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:31:30.359+05:30 [Info] switched currmeta from 91 -> 91 force true 2023-10-21T04:31:30.360+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842789859955371, Old rollbacktime: 0 2023-10-21T04:31:30.360+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6043349427976659583:[3 4 5 6 7 8 1 2] 14217467447424830582:[0]]. New rollbacktime: 1697842789859955371, Old rollbacktime: 0 2023-10-21T04:31:30.360+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6043349427976659583:[3 4 5 6 7 8 1 2] 14217467447424830582:[0]]. New rollbacktime: 1697842789859955371, Old rollbacktime: 0 2023-10-21T04:31:30.365+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.481856ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:30.365+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 69.166µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:30.374+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:30.374+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:30.383+05:30 [Info] switched currmeta from 91 -> 92 force true 2023-10-21T04:31:30.443+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:31:30.443+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0] 15864358084358521096:[5 6 7 8 1 2 3 4]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:31:30.465+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:31:30.824+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 1949031200982 23 21886 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:30.824+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 245782287380001 24 21886 172.31.5.112:9000. HWT [24-24,24]. 2023-10-21T04:31:30.826+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 52522903947059 28 21886 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:30.828+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 93079113162334 18 21886 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:30.828+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 205201940883916 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.832+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 244791485011286 23 21886 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:30.835+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 167580527121081 23 21886 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:30.837+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 177815184127237 17 21886 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:30.839+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 54236768606397 20 21886 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:30.840+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 134409829206048 26 21886 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:30.842+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 160381752747972 18 21886 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:30.844+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 58522600746999 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.848+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 142387448675642 30 21886 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:31:30.850+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 90115353725987 21 21886 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:30.869+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 161230548567164 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 163460438181870 34 21886 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 85302001459467 21 21886 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 213108110408291 23 21886 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 147946970349859 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 53829314402679 32 21886 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 19989312661772 15 21886 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 123957315902602 27 21886 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 151273673850546 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 150690956328758 26 21886 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 258671969827141 19 21886 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:30.880+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 114823206239184 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.882+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 68847799765163 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.889+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 232028101514735 19 21886 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:30.897+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 8305763409275 20 21886 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:30.897+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 107407811917992 20 21886 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:30.897+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 124811039198764 18 21886 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:30.897+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 207489375637709 12 21886 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:31:30.898+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 33310272161722 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.898+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 25510109770064 28 21886 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:30.898+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 100746255343609 19 21886 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:30.901+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 117298435979365 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.901+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 228857796432725 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.916+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 158357249947806 21 21886 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:30.917+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 47156288300713 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.917+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 203503670520010 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.917+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 236904976023311 22 21886 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:30.919+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 124310544527292 21 21886 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:30.919+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 104136033125373 29 21886 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:31:30.925+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 276463747117997 22 21886 172.31.5.112:9000. HWT [22-22,22]. 2023-10-21T04:31:30.925+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 246959094981553 34 21886 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:31:30.925+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 196269910725961 26 21886 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:30.929+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 65885584526743 20 21886 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:30.932+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:30.932+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc004e17500 2023-10-21T04:31:30.933+05:30 [Info] handleUpdateIndexRState: Index instance 12266944006962282785 rstate moved to ACTIVE 2023-10-21T04:31:30.933+05:30 [Info] Indexer::handleMergePartition Source 7309083551510465457 Target 8965923595543532942 2023-10-21T04:31:30.933+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006fb4960 2023-10-21T04:31:30.933+05:30 [Info] MergePartition: sent async request to update index instance 8965923595543532942 rstate moved to ACTIVE 2023-10-21T04:31:30.935+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 213334533059126 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.935+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 172365063201867 21 21886 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:30.936+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 79077905592013 17 21886 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:30.936+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 212884822297038 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.940+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 142324128597367 22 21886 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:30.940+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 254944616021293 24 21886 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:30.940+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 55145461136105 16 21886 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:31:30.940+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 246582249071123 20 21886 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:30.941+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.910991ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:30.944+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 219528837936411 25 21886 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:30.944+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 194568213794435 29 21886 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:31:30.944+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 80145536651832 17 21886 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:30.945+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 17434304260397056806. Retrying... 2023-10-21T04:31:30.945+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[8965923595543532942:true 12266944006962282785:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:31:30.948+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.726329ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:30.950+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 7309083551510465457, realInst: 8965923595543532942 2023-10-21T04:31:30.969+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:31:31.018+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 3381524789655566900, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:31:31.018+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 3381524789655566900, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:31:31.018+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 InstanceIds [15864358084358521096 3023333489681759512] 2023-10-21T04:31:31.018+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 21886 2023-10-21T04:31:31.018+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 21886 2023-10-21T04:31:31.018+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:31:31.018+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:31:31.018+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:31:31.018+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:31:31.018+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 21886 2023-10-21T04:31:31.018+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 21886 STREAM_ACTIVE 2023-10-21T04:31:31.018+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:31:31.019+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 25657368013413 17 21886 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:31.019+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 25511316571199 16 21886 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:31:31.019+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 223881084081370 17 21886 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:31.019+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 106620692996001 23 21886 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.019+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 279733370822238 23 21886 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.019+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 36300629322478 32 21886 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:31:31.026+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.026+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.027+05:30 [Info] switched currmeta from 92 -> 94 force true 2023-10-21T04:31:31.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842885353317281, Old rollbacktime: 0 2023-10-21T04:31:31.027+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842885353317281, Old rollbacktime: 0 2023-10-21T04:31:31.032+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.032+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.034+05:30 [Info] switched currmeta from 94 -> 94 force true 2023-10-21T04:31:31.046+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17434304260397056806,realInstId: 0, indexDefn DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:31:31.048+05:30 [Info] Indexer::handleRecoverIndex InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:31:31.053+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 5.41209ms 2023-10-21T04:31:31.053+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17434304260397056806 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[17406544375239214575 10170029854712240877]] 2023-10-21T04:31:31.056+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17434304260397056806 PartitionId: 0 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:31:31.053+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17434304260397056806, indexDefn DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:31:31.057+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 38.937958ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:31.053+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 5.647505ms 2023-10-21T04:31:31.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:31:31.057+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:31:31.057+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: RP version: 7 numRPs: 2 rp [0] sn: 7 count: 6 metaSz: 2490 rp [1] sn: 9 count: 6 metaSz: 2493 2023-10-21T04:31:31.062+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:31.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9666817578913242, throttleDelayMs (new, old, change): (9, 7, 2) 2023-10-21T04:31:31.068+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.068+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.069+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: doInit: data UsedSpace 57359 recovery UsedSpace 58200 2023-10-21T04:31:31.078+05:30 [Info] switched currmeta from 94 -> 95 force true 2023-10-21T04:31:31.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:31:31.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:31:31.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:31:31.080+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:31:31.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex started 2023-10-21T04:31:31.081+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:31:31.082+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:31:31.083+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:31:31.094+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Warmup took 38.41988ms 2023-10-21T04:31:31.095+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17434304260397056806 partitionId 0 WriterThreads 4 2023-10-21T04:31:31.096+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17434304260397056806 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index Index: 17434304260397056806 Partition: 0 2023-10-21T04:31:31.107+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17434304260397056806, shardIdMap: map[0:[17406544375239214575]] err: 2023-10-21T04:31:31.111+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17434304260397056806, partnShardIdMap: map[0:[17406544375239214575]], err: 2023-10-21T04:31:31.111+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:31.112+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17434304260397056806 2023-10-21T04:31:31.113+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:31.113+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:31:31.113+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:31.113+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:31.113+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:31.113+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17434304260397056806 Partitions [0] 2023-10-21T04:31:31.114+05:30 [Info] StorageMgr::openSnapshot IndexInst:17434304260397056806 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:31:31.120+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.498285ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:31.120+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 72.17µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:31:31.148+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.148+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.152+05:30 [Info] switched currmeta from 95 -> 96 force true 2023-10-21T04:31:31.213+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.213+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.215+05:30 [Info] switched currmeta from 96 -> 97 force true 2023-10-21T04:31:31.244+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.248+05:30 [Info] switched currmeta from 97 -> 98 force true 2023-10-21T04:31:31.346+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3464125101225762400:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:31.346+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [11276338709617092948] belonging to collectionId: 9, ttid: ShardToken2e:2c:44:28:da:2:14:94. Initiating build 2023-10-21T04:31:31.357+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [3464125101225762400] &{2} 2023-10-21T04:31:31.358+05:30 [Info] Indexer::handleBuildRecoveredIndexes [3464125101225762400] 2023-10-21T04:31:31.358+05:30 [Info] bulkUpdateRState: Index instance 3464125101225762400 rstate moved to PENDING 2023-10-21T04:31:31.358+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [3464125101225762400] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:31:31.358+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:31.358+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:31.358+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:31.359+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:31:31.359+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:31.359+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:31.359+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:31.359+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:31:31.359+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:31:31.359+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:31.359+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:31.359+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:31:31.359+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:31.359+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:31.359+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:31:31.359+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:31:31.360+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 31407 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:31:31.360+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:31.360+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:31.360+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:31:31.360+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:2] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:31:31.360+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3464125101225762400 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:31:31.360+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:31:31.360+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=23,1=24,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=27,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=20,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=22,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 31407 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 24 24 24} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 27 27 27} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 20 0 20} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 22 22 22} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:31:31.367+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:31:31.367+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:31:31.367+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:31.367+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 9.758686ms 2023-10-21T04:31:31.367+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 9.802186ms 2023-10-21T04:31:31.368+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [3464125101225762400] 2023-10-21T04:31:31.368+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 20.869907ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:31.369+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[3464125101225762400:true] 2023-10-21T04:31:31.369+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.369+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.371+05:30 [Info] switched currmeta from 98 -> 99 force true 2023-10-21T04:31:31.372+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:31:31.386+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.701691ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:31.406+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:31.411+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:31.411+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:31.421+05:30 [Info] switched currmeta from 99 -> 100 force true 2023-10-21T04:31:31.441+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842885353317281, Old rollbacktime: 0 2023-10-21T04:31:31.441+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842885353317281, Old rollbacktime: 0 2023-10-21T04:31:31.441+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842885353317281, Old rollbacktime: 0 2023-10-21T04:31:31.502+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 274143649547516 29 36450 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:31:31.507+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 120349894144442 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.507+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 162049908746825 25 36450 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:31:31.507+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 206082810619393 24 36450 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:31.509+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 143687065777286 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.511+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 212967951483349 28 36450 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:31.513+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 116493903186721 22 36450 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:31.516+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 78446412189949 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.520+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 263088038522959 18 36450 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:31.520+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 136568163328749 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.522+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 127746572208200 19 36450 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:31.522+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 261951617336488 35 36450 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:31:31.524+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 58733680850130 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.525+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 188740876924377 22 36450 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:31.529+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 173457131919400 12 36450 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:31:31.529+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 277338544220217 21 36450 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:31.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 206088069380492 13 36450 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:31:31.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 245247859472006 17 36450 172.31.5.112:9000. HWT [17-17,17]. 2023-10-21T04:31:31.534+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 34510046292135 16 36450 172.31.5.112:9000. HWT [16-16,16]. 2023-10-21T04:31:31.536+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 250426459207023 28 36450 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:31.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 125696667341112 27 36450 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:31.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 35716737784090 24 36450 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:31.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 274158831830257 20 36450 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:31.547+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 191500187207978 21 36450 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:31.549+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 106231614376481 21 36450 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:31.552+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 118920785117980 21 36450 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:31.553+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 137256376714252 22 36450 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:31.555+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 5906787184229 20 36450 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:31.555+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 146375746634523 17 36450 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:31.558+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 112482982607415 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.562+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 41101832875819 19 36450 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:31.562+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 241391494503904 28 36450 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:31.563+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 121450709505223 25 36450 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:31.565+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 216238759475962 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.567+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 127702263378100 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.567+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 168446351781198 25 36450 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:31.571+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 100008922352935 17 36450 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:31.571+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 169268048765249 19 36450 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:31.575+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 245900669851618 19 36450 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:31.575+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 149033089785397 28 36450 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:31.577+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 88718752743571 25 36450 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:31.577+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 106297365959267 27 36450 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:31.581+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 160399698726237 27 36450 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:31.581+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 39045953044628 18 36450 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:31.583+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 217237947818243 20 36450 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:31.587+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 110286285845649 31 36450 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:31:31.587+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 162870112426804 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.589+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 132470209691538 26 36450 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:31.592+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 264001907139144 22 36450 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:31.592+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 63737850123332 19 36450 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:31.594+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 266604622117081 26 36450 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:31.597+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 32274016651656 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.599+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 84581802233088 22 36450 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:31.601+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 225662991941187 26 36450 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:31.603+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 241607113601772 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.605+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 272523968790649 20 36450 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:31.606+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 77598347829810 19 36450 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:31.609+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 197957220074784 24 36450 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:31.611+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 136038310030103 28 36450 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:31.614+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 97677619676504 31 36450 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:31:31.614+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 72072509155168 23 36450 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:31.614+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 105415986286631 18 36450 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:31.619+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 27479155597282 29 36450 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:31:31.619+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 161870047388121 25 36450 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:31.622+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 InstanceIds [15864358084358521096 3023333489681759512 12266944006962282785 8965923595543532942] 2023-10-21T04:31:31.622+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 36450 2023-10-21T04:31:31.622+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 36450 2023-10-21T04:31:31.622+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:31:31.622+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:31:31.622+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:31:31.622+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 36450 2023-10-21T04:31:31.623+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 36450 STREAM_ACTIVE 2023-10-21T04:31:32.019+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:31:32.019+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:31:32.063+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8602930852623937, throttleDelayMs (new, old, change): (4, 9, -5) 2023-10-21T04:31:32.072+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[17434304260397056806:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:31:32.072+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [3381524789655566900] belonging to collectionId: 9, ttid: ShardToken75:7c:cc:96:58:f4:7c:a8. Initiating build 2023-10-21T04:31:32.080+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [17434304260397056806] &{2} 2023-10-21T04:31:32.080+05:30 [Info] Indexer::handleBuildRecoveredIndexes [17434304260397056806] 2023-10-21T04:31:32.080+05:30 [Info] bulkUpdateRState: Index instance 17434304260397056806 rstate moved to PENDING 2023-10-21T04:31:32.080+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [17434304260397056806] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:31:32.080+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:32.080+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:32.080+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:32.082+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:31:32.082+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:32.082+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:32.082+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:32.082+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:32.082+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:32.083+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:31:32.083+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:31:32.083+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:32.083+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:32.083+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_system:_query:{0xc00b21cb40 }] 2023-10-21T04:31:32.083+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:32.083+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:32.083+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:32.083+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:32.088+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_system:_query reset bucket state to NIL 2023-10-21T04:31:32.089+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:31:32.090+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId 43684 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:31:32.090+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:32.090+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:32.090+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:32.090+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:32.091+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:31:32.091+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:2] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:31:32.091+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17434304260397056806 stream INIT_STREAM keyspaceId bucket_%2:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:31:32.091+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:31:32.092+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=17,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 43684 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:31:32.092+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_system:_query:{0xc0082920c0 } bucket_1:_system:_query:{0xc00b21cb40 }] 2023-10-21T04:31:32.095+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:32.098+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 17.975665ms 2023-10-21T04:31:32.098+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 18.029858ms 2023-10-21T04:31:32.098+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [17434304260397056806] 2023-10-21T04:31:32.098+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 26.159122ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:32.100+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[17434304260397056806:true] 2023-10-21T04:31:32.102+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_system:_query Check pending stream merge. 2023-10-21T04:31:32.113+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.254567ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:32.117+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:31:32.125+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:32.128+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:32.128+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:32.133+05:30 [Info] switched currmeta from 100 -> 102 force true 2023-10-21T04:31:32.151+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:32.151+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:32.153+05:30 [Info] switched currmeta from 102 -> 102 force true 2023-10-21T04:31:32.166+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:57740" doAuth() ... 2023-10-21T04:31:32.170+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:57740" auth successful 2023-10-21T04:31:32.171+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:57740" +1 2023-10-21T04:31:32.293+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:32.293+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:32.296+05:30 [Info] switched currmeta from 102 -> 104 force true 2023-10-21T04:31:32.303+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:32.303+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:32.306+05:30 [Info] switched currmeta from 104 -> 104 force true 2023-10-21T04:31:32.370+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:31:32.411+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:32.624+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:31:32.624+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:31:33.015+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 1 162049908746825 25 43684 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:31:33.019+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 2 206082810619393 24 43684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:33.019+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:31:33.024+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 3 120349894144442 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.028+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 7 78446412189949 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.028+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 5 212967951483349 28 43684 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:33.029+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 6 116493903186721 22 43684 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:33.031+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 0 274143649547516 29 43684 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:31:33.032+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 9 136568163328749 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.038+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 10 261951617336488 35 43684 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:31:33.038+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 11 127746572208200 19 43684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:33.042+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 14 173457131919400 12 43684 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:31:33.042+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 13 188740876924377 22 43684 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:33.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 17 206088069380492 13 43684 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:31:33.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 8 263088038522959 18 43684 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:33.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 18 34510046292135 16 43684 172.31.5.112:9000. HWT [16-16,16]. 2023-10-21T04:31:33.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 4 143687065777286 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 15 277338544220217 21 43684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:33.055+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 12 58733680850130 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.057+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 21 125696667341112 27 43684 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:33.057+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 16 245247859472006 17 43684 172.31.5.112:9000. HWT [17-17,17]. 2023-10-21T04:31:33.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 22 35716737784090 24 43684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:33.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8394952630602696, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:31:33.065+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 25 118920785117980 21 43684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:33.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 26 137256376714252 22 43684 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:33.070+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 29 112482982607415 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.072+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 30 41101832875819 19 43684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:33.078+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 34 168446351781198 25 43684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:33.084+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 38 245900669851618 19 43684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:33.105+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 19 250426459207023 28 43684 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 35 127702263378100 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 31 241391494503904 28 43684 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 51 32274016651656 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 33 216238759475962 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 37 169268048765249 19 43684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 27 5906787184229 20 43684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 23 191500187207978 21 43684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 39 149033089785397 28 43684 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 43 160399698726237 27 43684 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:33.115+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 47 132470209691538 26 43684 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 40 106297365959267 27 43684 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 49 63737850123332 19 43684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 24 106231614376481 21 43684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 53 225662991941187 26 43684 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 45 110286285845649 31 43684 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 28 146375746634523 17 43684 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 32 121450709505223 25 43684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 36 100008922352935 17 43684 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:31:33.119+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 41 88718752743571 25 43684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:33.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 57 197957220074784 24 43684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:31:33.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 58 136038310030103 28 43684 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:31:33.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 46 162870112426804 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 42 39045953044628 18 43684 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:33.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 50 266604622117081 26 43684 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:31:33.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 48 264001907139144 22 43684 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:33.120+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 44 217237947818243 20 43684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:33.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 60 72072509155168 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 52 84581802233088 22 43684 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:31:33.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 54 241607113601772 23 43684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:31:33.121+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 55 272523968790649 20 43684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:33.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 20 274158831830257 20 43684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:31:33.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 59 97677619676504 31 43684 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:31:33.123+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 62 27479155597282 29 43684 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:31:33.125+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 56 77598347829810 19 43684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:31:33.125+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 61 105415986286631 18 43684 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:31:33.125+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 63 161870047388121 25 43684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:31:33.142+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:33.171+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_system:_query InstanceIds [17434304260397056806] 2023-10-21T04:31:33.172+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query Cid 9 SessionId 43684 2023-10-21T04:31:33.172+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43684 2023-10-21T04:31:33.172+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:31:33.172+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:31:33.173+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 17434304260397056806 INDEX_STATE_CATCHUP 2023-10-21T04:31:33.173+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 17434304260397056806 true 2023-10-21T04:31:33.173+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 17434304260397056806 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query Session: 43684 BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=17,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] 2023-10-21T04:31:33.173+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_system:_query, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} , fetchKVSeq false 2023-10-21T04:31:33.173+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:31:33.174+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:31:33.174+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43684 STREAM_ACTIVE 2023-10-21T04:31:33.174+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 43684 2023-10-21T04:31:33.175+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_system:_query 43684 2023-10-21T04:31:33.175+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:33.175+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:33.175+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:33.176+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:33.176+05:30 [Info] send progress stats to clients 2023-10-21T04:31:33.176+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:31:33.177+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:31:33.177+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:2] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:31:33.177+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 43684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:31:33.177+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:31:33.180+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=29,1=25,2=4,3=4,4=4,5=4,6=4,7=4,8=4,9=4,10=4,11=4,12=4,13=4,14=4,15=4,16=17,17=4,18=16,19=4,20=4,21=4,22=4,23=4,24=4,25=4,26=4,27=4,28=4,29=4,30=4,31=4,32=4,33=4,34=4,35=4,36=4,37=4,38=4,39=4,40=4,41=4,42=4,43=27,44=4,45=31,46=4,47=4,48=4,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=4,59=4,60=4,61=4,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:33.193+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:31:33.209+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 2023-10-21T04:31:33.209+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43684 2023-10-21T04:31:33.209+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43684 2023-10-21T04:31:33.209+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:31:33.209+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:31:33.209+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 17434304260397056806 false 2023-10-21T04:31:33.209+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_system:_query 17434304260397056806 2023-10-21T04:31:33.209+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:31:33.209+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 43684 2023-10-21T04:31:33.216+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 38.405857ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:33.258+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:33.258+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:33.266+05:30 [Info] switched currmeta from 104 -> 105 force true 2023-10-21T04:31:33.431+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:33.460+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:33.460+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:33.465+05:30 [Info] switched currmeta from 105 -> 107 force true 2023-10-21T04:31:33.557+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:33.557+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:33.587+05:30 [Info] switched currmeta from 107 -> 107 force true 2023-10-21T04:31:33.671+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:33.671+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:33.694+05:30 [Info] switched currmeta from 107 -> 108 force true 2023-10-21T04:31:33.717+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:33.787+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:33.787+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:33.791+05:30 [Info] switched currmeta from 108 -> 108 force true 2023-10-21T04:31:33.993+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:33.993+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:34.016+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:34.034+05:30 [Info] switched currmeta from 108 -> 110 force true 2023-10-21T04:31:34.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9759062512036745, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:31:34.120+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:34.120+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:34.127+05:30 [Info] switched currmeta from 110 -> 110 force true 2023-10-21T04:31:34.175+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:31:34.178+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:31:34.178+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_system:_query 2023-10-21T04:31:34.178+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 17434304260397056806 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId: 43684, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 17 17 17} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:31:34.178+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 17434304260397056806 INDEX_STATE_ACTIVE 2023-10-21T04:31:34.178+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query State Changed to INACTIVE 2023-10-21T04:31:34.178+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:31:34.178+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:31:34.178+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:34.178+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 43684 2023-10-21T04:31:34.178+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:34.178+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:34.178+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:34.178+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 43684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:34.179+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17434304260397056806 from stream INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:31:34.179+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:31:34.179+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:31:34.179+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:34.179+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 43684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:34.179+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_system:_query:{0xc00b21cb40 }] 2023-10-21T04:31:34.179+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:34.179+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:34.179+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:34.179+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:34.179+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:34.179+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:34.179+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:34.179+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:34.179+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:31:34.179+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:31:34.179+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId 43684 2023-10-21T04:31:34.180+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:31:34.200+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.421025ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:34.211+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:56102. Error = read tcp4 127.0.0.1:9118->127.0.0.1:56102: read: connection reset by peer. Kill Pipe. 2023-10-21T04:31:34.211+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = EOF. Cleaning Pipe. 2023-10-21T04:31:34.211+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly. 2023-10-21T04:31:34.212+05:30 [Info] messageListener.start(): message channel closed. Remove peer 12:aa:89:61:6d:d0:9f:81 as follower. 2023-10-21T04:31:34.284+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:34.284+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:34.295+05:30 [Info] switched currmeta from 110 -> 111 force true 2023-10-21T04:31:34.313+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:31:34.313+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/c596d1e7da6bfc94cbe77df771748cc5 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:c596d1e7da6bfc94cbe77df771748cc5]}]} 2023-10-21T04:31:34.318+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:31:34.318+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:31:34.321+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/c596d1e7da6bfc94cbe77df771748cc5, rev: [] 2023-10-21T04:31:34.321+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/c596d1e7da6bfc94cbe77df771748cc5, rev: [] 2023-10-21T04:31:34.321+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:31:34.321+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:31:34.321+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:31:34.321+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:31:34.439+05:30 [Info] ShardRebalancer::waitForIndexState Cancel Received 2023-10-21T04:31:34.611+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:31:35.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9999726149222914, throttleDelayMs (new, old, change): (8, 3, 5) 2023-10-21T04:31:35.141+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:31:35.175+05:30 [Info] ShardRebalancer::waitForIndexState Cancel Received 2023-10-21T04:31:35.175+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:31:35.200+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:31:35.200+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:35.200+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken2e:2c:44:28:da:2:14:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenae:b8:e2:e7:52:ef:e5:ce Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9447329072527540494 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5145547347365683414 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3023333489681759512 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:35.201+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: c596d1e7da6bfc94cbe77df771748cc5 TransferTokenId: ShardToken2e:2c:44:28:da:2:14:94 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:31:35.201+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000127396 2023-10-21T04:31:35.201+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken2e:2c:44:28:da:2:14:94, dropIndexes: true 2023-10-21T04:31:35.201+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94, shardId :0 2023-10-21T04:31:35.201+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken2e_2c_44_28_da_2_14_94, shardId :0 2023-10-21T04:31:35.201+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16337157693634799600 instance id 9447329072527540494 real instance id 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:35.201+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16337157693634799600 instance 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:35.201+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16337157693634799600 instance id 10032435514095102923 2023-10-21T04:31:35.202+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16337157693634799600 2023-10-21T04:31:35.211+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10032435514095102923 &{1} 2023-10-21T04:31:35.211+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10032435514095102923 2023-10-21T04:31:35.211+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16337157693634799600, instId: 10032435514095102923 2023-10-21T04:31:35.212+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:31:35.212+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:31:35.212+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:35.212+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10032435514095102923] 2023-10-21T04:31:35.212+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.212+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.212+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.212+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:35.213+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:35.213+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:35.213+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:35.213+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:31:35.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:31:35.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.213+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.215+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:35.215+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:35.215+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:35.215+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:35.215+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:31:35.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.215+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.215+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:35.215+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:35.216+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:35.216+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:35.216+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:31:35.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.216+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:35.216+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:35.216+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:35.216+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:35.216+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:31:35.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.216+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.216+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:35.216+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:35.216+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:35.216+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:35.217+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:31:35.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.217+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:35.217+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 5.923376ms 2023-10-21T04:31:35.217+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 5.964053ms 2023-10-21T04:31:35.217+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:35.217+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.217+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:35.217+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:35.218+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10032435514095102923 2023-10-21T04:31:35.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:31:35.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:35.219+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:35.219+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:35.219+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:35.219+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:31:35.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.219+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.219+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:35.219+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:35.220+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:35.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:31:35.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:35.220+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:35.220+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:35.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:31:35.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:31:35.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:35.220+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:35.220+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:35.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:31:35.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.220+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.221+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.249+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:31:35.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:31:35.249+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.249+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:31:35.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:31:35.306+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:35.306+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:35.309+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.314+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:31:35.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:31:35.342+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:31:35.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:31:35.346+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.346+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.348+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:31:35.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:31:35.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.350+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.350+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.350+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:35.352+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.352+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:35.359+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:31:35.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:31:35.361+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.367+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.371+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.373+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:35.372+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:31:35.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:31:35.374+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.374+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.374+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.374+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:35.331+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 129.813201ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:35.377+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16961575949607441659 instance id 5145547347365683414 real instance id 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:35.377+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16961575949607441659 instance 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:35.377+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16961575949607441659 instance id 15864358084358521096 2023-10-21T04:31:35.377+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16961575949607441659 2023-10-21T04:31:35.382+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:31:35.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:31:35.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:31:35.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:31:35.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:31:35.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:31:35.395+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.397+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.397+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.399+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.396+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.397+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:35.402+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.402+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.408+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.409+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:35.411+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15864358084358521096 &{1} 2023-10-21T04:31:35.414+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15864358084358521096 2023-10-21T04:31:35.413+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:35.417+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16961575949607441659, instId: 15864358084358521096 2023-10-21T04:31:35.420+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:31:35.423+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:35.425+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:31:35.422+05:30 [Info] switched currmeta from 111 -> 113 force true 2023-10-21T04:31:35.426+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15864358084358521096] 2023-10-21T04:31:35.428+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15864358084358521096] 2023-10-21T04:31:35.429+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:35.430+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15864358084358521096] 2023-10-21T04:31:35.431+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:31:35.431+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:31:35.431+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:31:35.431+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:31:35.431+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:31:35.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex stopped 2023-10-21T04:31:35.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.432+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.432+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.432+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.432+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.432+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:31:35.432+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:31:35.432+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:31:35.432+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:31:35.432+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:31:35.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex stopped 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.433+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:31:35.433+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:31:35.433+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:31:35.433+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:31:35.433+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:31:35.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex stopped 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.433+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:31:35.433+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:31:35.433+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:31:35.433+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:31:35.433+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:31:35.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex stopped 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.433+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.431+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.434+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.434+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.434+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:31:35.445+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:31:35.435+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.434+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.434+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.435+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.433+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 21886 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:35.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:31:35.446+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.446+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Destroy Done 2023-10-21T04:31:35.446+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:31:35.446+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:31:35.446+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:31:35.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex stopped 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:31:35.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex closed 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.446+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:31:35.447+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:31:35.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex stopped 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:31:35.447+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:31:35.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex stopped 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:31:35.447+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:31:35.447+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:31:35.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex stopped 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.447+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.447+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 21886 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:35.448+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 34.058632ms 2023-10-21T04:31:35.448+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:35.448+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:35.448+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:35.448+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:31:35.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:31:35.448+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.448+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.448+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.448+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.448+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15864358084358521096 2023-10-21T04:31:35.449+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:35.449+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:35.449+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:35.449+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:31:35.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:31:35.449+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.449+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.449+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.449+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:31:35.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:31:35.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:31:35.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:35.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:31:35.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.451+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:35.451+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:35.451+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:35.451+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:31:35.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.451+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:31:35.451+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:31:35.451+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:31:35.451+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:31:35.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex stopped 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.451+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.465+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:31:35.466+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex closed 2023-10-21T04:31:35.467+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.467+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:31:35.468+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:31:35.468+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:31:35.468+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:31:35.469+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:31:35.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex stopped 2023-10-21T04:31:35.469+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.469+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.470+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:31:35.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex closed 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:31:35.472+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:31:35.472+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:31:35.472+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:31:35.472+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:31:35.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex stopped 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.472+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.475+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:31:35.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex closed 2023-10-21T04:31:35.484+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.484+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.484+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.483+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.483+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 106.407467ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:35.488+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:31:35.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex closed 2023-10-21T04:31:35.488+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.488+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.488+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.488+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:31:35.488+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:31:35.490+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:31:35.490+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:31:35.490+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:31:35.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex stopped 2023-10-21T04:31:35.490+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.491+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.491+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.491+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.493+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:31:35.493+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:31:35.493+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:31:35.494+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:31:35.494+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:31:35.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex stopped 2023-10-21T04:31:35.494+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.494+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.494+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.494+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:31:35.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex closed 2023-10-21T04:31:35.495+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.495+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.495+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.495+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:31:35.495+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:31:35.496+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:31:35.496+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:31:35.496+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:31:35.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex stopped 2023-10-21T04:31:35.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.496+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.509+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:31:35.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex closed 2023-10-21T04:31:35.525+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.525+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.525+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.525+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:31:35.525+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:31:35.526+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:31:35.526+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:31:35.526+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:31:35.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex stopped 2023-10-21T04:31:35.526+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.526+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.526+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.527+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.559+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:31:35.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex closed 2023-10-21T04:31:35.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.561+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:35.562+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:35.562+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:35.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:31:35.563+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:31:35.564+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:31:35.564+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:31:35.565+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:31:35.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex stopped 2023-10-21T04:31:35.567+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:35.570+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:35.572+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:35.582+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:35.600+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3023333489681759512 &{1} 2023-10-21T04:31:35.623+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:31:35.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:31:35.628+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:31:35.628+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.628+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.629+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:35.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:31:35.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:31:35.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:31:35.631+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:31:35.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex closed 2023-10-21T04:31:35.635+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:31:35.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:31:35.636+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:31:35.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:31:35.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:31:35.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:31:35.638+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.638+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:31:35.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:31:35.636+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.640+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.640+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.640+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.640+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.640+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.640+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.641+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:31:35.643+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.643+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Destroy Done 2023-10-21T04:31:35.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:31:35.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex closed 2023-10-21T04:31:35.643+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.643+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:31:35.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:31:35.644+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.645+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:31:35.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:31:35.651+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex ... 2023-10-21T04:31:35.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex ... 2023-10-21T04:31:35.652+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:31:35.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:31:35.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:31:35.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:31:35.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex closed 2023-10-21T04:31:35.657+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.657+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex ... 2023-10-21T04:31:35.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex ... 2023-10-21T04:31:35.658+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.658+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.658+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Destroy Done 2023-10-21T04:31:35.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index destroyed 2023-10-21T04:31:35.703+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.703+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 4 Destroy Done 2023-10-21T04:31:35.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:31:35.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex closed 2023-10-21T04:31:35.704+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.704+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.705+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:31:35.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex closed 2023-10-21T04:31:35.705+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.705+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.709+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:31:35.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:31:35.710+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.712+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:31:35.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:31:35.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex ... 2023-10-21T04:31:35.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex ... 2023-10-21T04:31:35.730+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.734+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:31:35.734+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.735+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Destroy Done 2023-10-21T04:31:35.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:31:35.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex closed 2023-10-21T04:31:35.735+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.736+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:31:35.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex closed 2023-10-21T04:31:35.736+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.737+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 5 Close Done 2023-10-21T04:31:35.735+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:31:35.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:31:35.738+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.738+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:31:35.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex ... 2023-10-21T04:31:35.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex ... 2023-10-21T04:31:35.742+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:35.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index destroyed 2023-10-21T04:31:35.748+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.749+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 8 Destroy Done 2023-10-21T04:31:35.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:31:35.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex closed 2023-10-21T04:31:35.750+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:35.750+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:35.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:31:35.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:31:35.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:31:35.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:31:35.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex ... 2023-10-21T04:31:35.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex ... 2023-10-21T04:31:35.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:31:35.763+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.763+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Destroy Done 2023-10-21T04:31:35.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:31:35.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:31:35.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:31:35.784+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.784+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.785+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Destroy Done 2023-10-21T04:31:35.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex ... 2023-10-21T04:31:35.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex ... 2023-10-21T04:31:35.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex ... 2023-10-21T04:31:35.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex ... 2023-10-21T04:31:35.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index destroyed 2023-10-21T04:31:35.790+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.790+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 6 Destroy Done 2023-10-21T04:31:35.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex ... 2023-10-21T04:31:35.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex ... 2023-10-21T04:31:35.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex ... 2023-10-21T04:31:35.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex ... 2023-10-21T04:31:35.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index destroyed 2023-10-21T04:31:35.795+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.796+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 3 Destroy Done 2023-10-21T04:31:35.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex ... 2023-10-21T04:31:35.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex ... 2023-10-21T04:31:35.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex ... 2023-10-21T04:31:35.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex ... 2023-10-21T04:31:35.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index destroyed 2023-10-21T04:31:35.845+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.845+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 7 Destroy Done 2023-10-21T04:31:35.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex ... 2023-10-21T04:31:35.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex ... 2023-10-21T04:31:35.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:31:35.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:31:35.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex ... 2023-10-21T04:31:35.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex ... 2023-10-21T04:31:35.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index destroyed 2023-10-21T04:31:35.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:31:35.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:31:35.858+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.859+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 1 Destroy Done 2023-10-21T04:31:35.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:31:35.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:31:35.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:31:35.864+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.864+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Destroy Done 2023-10-21T04:31:35.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex ... 2023-10-21T04:31:35.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex ... 2023-10-21T04:31:35.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex ... 2023-10-21T04:31:35.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex ... 2023-10-21T04:31:35.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index destroyed 2023-10-21T04:31:35.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:31:35.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:31:35.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex sucessfully destroyed 2023-10-21T04:31:35.870+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.871+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 5 Destroy Done 2023-10-21T04:31:35.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:31:35.872+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:31:35.872+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Destroy Done 2023-10-21T04:31:35.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex ... 2023-10-21T04:31:35.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex ... 2023-10-21T04:31:35.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:35.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index destroyed 2023-10-21T04:31:35.874+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:31:35.875+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 2 Destroy Done 2023-10-21T04:31:35.932+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:36.030+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:36.047+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:36.047+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:36.047+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 185] updated cluster rebalancing status to false 2023-10-21T04:31:36.061+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9999156060957655, throttleDelayMs (new, old, change): (10, 8, 2) 2023-10-21T04:31:36.080+05:30 [Info] switched currmeta from 113 -> 116 force true 2023-10-21T04:31:36.080+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:31:36.080+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:31:36.080+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:31:36.172+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [15864358084358521096] 2023-10-21T04:31:36.175+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:31:36.175+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 21886 2023-10-21T04:31:36.175+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 727.041236ms 2023-10-21T04:31:36.175+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 761.167614ms 2023-10-21T04:31:36.175+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3023333489681759512 2023-10-21T04:31:36.175+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17797267771846799851, instId: 3023333489681759512 2023-10-21T04:31:36.175+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:31:36.175+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:36.176+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3023333489681759512 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:31:36.176+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3023333489681759512] 2023-10-21T04:31:36.176+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3023333489681759512] 2023-10-21T04:31:36.176+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:36.176+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3023333489681759512] 2023-10-21T04:31:36.177+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:36.177+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:36.177+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:36.177+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:36.177+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 21886 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:36.177+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc00b21c840 }] 2023-10-21T04:31:36.177+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 21886 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:36.177+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:31:36.177+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:31:36.177+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:36.177+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:36.177+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:36.177+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:36.178+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:36.178+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.625106ms 2023-10-21T04:31:36.178+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3023333489681759512, PartitionId 0, IndexDefnId 17797267771846799851. 2023-10-21T04:31:36.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:36.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:36.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:36.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex] 2023-10-21T04:31:36.178+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:31:36.178+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex] 2023-10-21T04:31:36.178+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex] 2023-10-21T04:31:36.178+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex 2023-10-21T04:31:36.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex stopped 2023-10-21T04:31:36.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:36.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:36.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:36.178+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:36.178+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1] 2023-10-21T04:31:36.178+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3023333489681759512 2023-10-21T04:31:36.186+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:36.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex] 2023-10-21T04:31:36.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex closed 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:36.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex] 2023-10-21T04:31:36.187+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:31:36.187+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex] 2023-10-21T04:31:36.187+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex] 2023-10-21T04:31:36.187+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex 2023-10-21T04:31:36.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex stopped 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:36.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:36.191+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex] 2023-10-21T04:31:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex closed 2023-10-21T04:31:36.191+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:36.191+05:30 [Info] Indexer::cleanupIndexData IndexInst 3023333489681759512 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex ... 2023-10-21T04:31:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex ... 2023-10-21T04:31:36.200+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 8 54236768606397 20 2023-10-21T04:31:36.200+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.200+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 5 244791485011286 23 2023-10-21T04:31:36.200+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.200+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 6 167580527121081 23 2023-10-21T04:31:36.200+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.200+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 7 177815184127237 17 2023-10-21T04:31:36.200+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.201+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 3 205201940883916 24 2023-10-21T04:31:36.201+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.201+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 4 93079113162334 18 2023-10-21T04:31:36.201+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.201+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 0 1949031200982 23 2023-10-21T04:31:36.201+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.201+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 1 245782287380001 24 2023-10-21T04:31:36.201+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.201+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 2 52522903947059 28 2023-10-21T04:31:36.201+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:36.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/mainIndex sucessfully destroyed 2023-10-21T04:31:36.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex ... 2023-10-21T04:31:36.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex ... 2023-10-21T04:31:36.202+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 15 85302001459467 21 2023-10-21T04:31:36.202+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.202+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 10 160381752747972 18 2023-10-21T04:31:36.202+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.202+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 16 123957315902602 27 2023-10-21T04:31:36.202+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.202+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 17 213108110408291 23 2023-10-21T04:31:36.202+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.202+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 9 134409829206048 26 2023-10-21T04:31:36.202+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.202+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 11 58522600746999 24 2023-10-21T04:31:36.202+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 12 142387448675642 30 2023-10-21T04:31:36.203+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 13 90115353725987 21 2023-10-21T04:31:36.203+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 14 161230548567164 25 2023-10-21T04:31:36.203+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 24 19989312661772 15 2023-10-21T04:31:36.203+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 18 163460438181870 34 2023-10-21T04:31:36.203+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 20 151273673850546 25 2023-10-21T04:31:36.203+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 22 258671969827141 19 2023-10-21T04:31:36.203+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.203+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 25 114823206239184 25 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 26 68847799765163 25 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 19 150690956328758 26 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 21 147946970349859 24 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 23 53829314402679 32 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 29 8305763409275 20 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 30 124811039198764 18 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 34 100746255343609 19 2023-10-21T04:31:36.204+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.204+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 36 228857796432725 25 2023-10-21T04:31:36.208+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.208+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 27 232028101514735 19 2023-10-21T04:31:36.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:31:36.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_3023333489681759512_0.index destroyed 2023-10-21T04:31:36.210+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3023333489681759512, PartitionId 0, IndexDefnId 17797267771846799851. 2023-10-21T04:31:36.210+05:30 [Info] Indexer::cleanupIndexData IndexInst 3023333489681759512 Partition 0 Destroy Done 2023-10-21T04:31:36.208+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.210+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 33 107407811917992 20 2023-10-21T04:31:36.210+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.210+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 35 117298435979365 24 2023-10-21T04:31:36.210+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.210+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 31 207489375637709 12 2023-10-21T04:31:36.210+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 28 25510109770064 28 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 32 33310272161722 24 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 57 80145536651832 17 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 42 124310544527292 21 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 39 158357249947806 21 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 40 104136033125373 29 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 41 236904976023311 22 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 37 203503670520010 24 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 38 47156288300713 24 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 43 196269910725961 26 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 44 246959094981553 34 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 46 65885584526743 20 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 47 172365063201867 21 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 50 246582249071123 20 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 51 213334533059126 25 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.211+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 60 279733370822238 23 2023-10-21T04:31:36.211+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 45 276463747117997 22 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 48 79077905592013 17 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 49 212884822297038 25 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 53 254944616021293 24 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 55 219528837936411 25 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 61 223881084081370 17 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 58 25511316571199 16 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 59 106620692996001 23 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 54 55145461136105 16 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 52 142324128597367 22 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.212+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 56 194568213794435 29 2023-10-21T04:31:36.212+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.213+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 63 25657368013413 17 2023-10-21T04:31:36.213+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.213+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 62 36300629322478 32 2023-10-21T04:31:36.213+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:31:36.213+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1] 2023-10-21T04:31:36.213+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 21886 2023-10-21T04:31:36.213+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 32.658549ms 2023-10-21T04:31:36.214+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 38.612729ms 2023-10-21T04:31:36.224+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 719.381244ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:36.236+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3464125101225762400 &{1} 2023-10-21T04:31:36.236+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3464125101225762400 2023-10-21T04:31:36.236+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 11276338709617092948, instId: 3464125101225762400 2023-10-21T04:31:36.237+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:36.238+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:36.238+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:31:36.239+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3464125101225762400] 2023-10-21T04:31:36.239+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3464125101225762400] 2023-10-21T04:31:36.239+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:36.239+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3464125101225762400] 2023-10-21T04:31:36.240+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:36.240+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:36.240+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:36.240+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:31:36.240+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 31407 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:36.240+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:31:36.241+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:36.241+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:36.241+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:31:36.241+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:57740: use of closed network connection 2023-10-21T04:31:36.241+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:57740" exit: read tcp4 127.0.0.1:9121->127.0.0.1:57740: use of closed network connection 2023-10-21T04:31:36.241+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:57740" closed ! 2023-10-21T04:31:36.241+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:31:36.241+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:57740" already gone 2023-10-21T04:31:36.241+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:31:36.241+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:36.241+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:36.241+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:36.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:31:36.241+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:31:36.241+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:31:36.242+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 31407 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:36.242+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3464125101225762400 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:31:36.242+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:31:36.242+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:31:36.242+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:36.242+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:36.243+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:36.243+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 6.336574ms 2023-10-21T04:31:36.243+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3464125101225762400 2023-10-21T04:31:36.271+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 44.495664ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:36.272+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:31:36.292+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:31:36.292+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:31:36.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex stopped 2023-10-21T04:31:36.292+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:36.292+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:36.292+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:36.292+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:36.295+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:36.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:31:36.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex closed 2023-10-21T04:31:36.295+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:36.295+05:30 [Info] Indexer::cleanupIndexData IndexInst 3464125101225762400 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:36.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:31:36.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:31:36.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:31:36.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index destroyed 2023-10-21T04:31:36.297+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:31:36.297+05:30 [Info] Indexer::cleanupIndexData IndexInst 3464125101225762400 Partition 0 Destroy Done 2023-10-21T04:31:36.319+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:31:36.526+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:36.526+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:36.539+05:30 [Info] switched currmeta from 116 -> 119 force true 2023-10-21T04:31:36.547+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:36.547+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:36.553+05:30 [Info] switched currmeta from 119 -> 119 force true 2023-10-21T04:31:36.553+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:31:36.553+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 1697842885760483465 2023-10-21T04:31:36.553+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 {Index inst : [partitions]}: map[3023333489681759512:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 1697842885760483465 2023-10-21T04:31:36.564+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:36.564+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:36.569+05:30 [Info] switched currmeta from 119 -> 119 force true 2023-10-21T04:31:37.061+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9990871062856789 2023-10-21T04:31:37.072+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_system:_query]. Err projector.topicMissing 2023-10-21T04:31:37.072+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:31:37.072+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Treating projector.topicMissing As Success 2023-10-21T04:31:37.072+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43684 2023-10-21T04:31:37.072+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43684 2023-10-21T04:31:37.072+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:31:37.074+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query InstanceIds [17434304260397056806 3464125101225762400] 2023-10-21T04:31:37.074+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 31407 2023-10-21T04:31:37.074+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31407 2023-10-21T04:31:37.074+05:30 [Info] Indexer::deleteStreamKeyspaceIdCurrRequest CurrRequest for keyspaceId Not Present SessionId 31407 StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query cmd &{82 3 bucket_1:_system:_query 0xc00027cf70 [] false 0xc00bf5d6c0 0xc006076e40 31407} 2023-10-21T04:31:37.074+05:30 [Info] Indexer::handleRecoveryDone Skip Recovery for Stream INIT_STREAM. Inactive KeyspaceId bucket_1:_system:_query 2023-10-21T04:31:37.075+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:31:37.075+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query]. Err projector.topicMissing 2023-10-21T04:31:37.075+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:31:37.075+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Treating projector.topicMissing As Success 2023-10-21T04:31:37.075+05:30 [Info] Indexer::removeIndexesFromStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31407 2023-10-21T04:31:37.075+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 832.686162ms 2023-10-21T04:31:37.075+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 839.159809ms 2023-10-21T04:31:37.077+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1:_system:_query 2023-10-21T04:31:37.101+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:37.113+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2925063509317026816 2023-10-21T04:31:37.113+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 16787445029868804812 2023-10-21T04:31:37.113+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 16787445029868804812 2023-10-21T04:31:37.113+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2925063509317026816 2023-10-21T04:31:37.113+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [16787445029868804812 2925063509317026816], elapsed: 840.534615ms 2023-10-21T04:31:37.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:37.113+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:31:37.113+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper stopped 2023-10-21T04:31:37.113+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': stopped 2023-10-21T04:31:37.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:37.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:37.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:37.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:37.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:31:37.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:37.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:31:37.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:31:37.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:37.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:37.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:37.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:37.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:31:37.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:31:37.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:37.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:37.117+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:37.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:31:37.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shutdown completed 2023-10-21T04:31:37.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:37.117+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:31:37.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:31:37.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:31:37.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance destroyed 2023-10-21T04:31:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:31:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:37.119+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:31:37.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:37.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:37.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:37.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:37.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:31:37.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:37.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:31:37.129+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:31:37.129+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:37.129+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:37.129+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:37.129+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:37.129+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:31:37.131+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:31:37.131+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:37.131+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:37.131+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:37.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:31:37.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shutdown completed 2023-10-21T04:31:37.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:37.131+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:31:37.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:31:37.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:31:37.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance destroyed 2023-10-21T04:31:37.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:31:37.138+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Cleaned up Tenant with name:bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:31:37.139+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [16787445029868804812 2925063509317026816], elapsed(sec): 0.866153092 2023-10-21T04:31:37.142+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken75:7c:cc:96:58:f4:7c:a8 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7309083551510465457 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14527226347372019068 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12266944006962282785 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:37.142+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: c596d1e7da6bfc94cbe77df771748cc5 TransferTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:31:37.143+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000217646 2023-10-21T04:31:37.143+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken75:7c:cc:96:58:f4:7c:a8, dropIndexes: true 2023-10-21T04:31:37.143+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8, shardId :0 2023-10-21T04:31:37.143+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardToken75_7c_cc_96_58_f4_7c_a8, shardId :0 2023-10-21T04:31:37.143+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 8910459001587546151 instance id 7309083551510465457 real instance id 8965923595543532942 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:37.143+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 8910459001587546151 instance 8965923595543532942 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:37.143+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 8910459001587546151 instance id 8965923595543532942 2023-10-21T04:31:37.143+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 8910459001587546151 2023-10-21T04:31:37.160+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 8965923595543532942 &{1} 2023-10-21T04:31:37.161+05:30 [Info] Indexer::handleDropIndex - IndexInstId 8965923595543532942 2023-10-21T04:31:37.161+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 8910459001587546151, instId: 8965923595543532942 2023-10-21T04:31:37.161+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:31:37.161+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:37.161+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:31:37.161+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [8965923595543532942] 2023-10-21T04:31:37.161+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [8965923595543532942] 2023-10-21T04:31:37.161+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:37.162+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [8965923595543532942] 2023-10-21T04:31:37.162+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.162+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:31:37.162+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:31:37.162+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:31:37.162+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex stopped 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.162+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.170+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:31:37.170+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:31:37.170+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:31:37.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex stopped 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.170+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:31:37.170+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:31:37.170+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:31:37.170+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:31:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex stopped 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.171+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:31:37.171+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:31:37.171+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:31:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex stopped 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.171+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:31:37.171+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:31:37.171+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:31:37.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex stopped 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.171+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.171+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.172+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:31:37.172+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:31:37.172+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:31:37.172+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:31:37.172+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:31:37.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex stopped 2023-10-21T04:31:37.172+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.172+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.172+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.172+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.172+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:37.178+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.178+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:31:37.179+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:31:37.179+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:31:37.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex stopped 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.179+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:31:37.179+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:31:37.179+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:31:37.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex stopped 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.179+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.179+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:37.188+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:31:37.188+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 27.545201ms 2023-10-21T04:31:37.188+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 8965923595543532942 2023-10-21T04:31:37.193+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [8965923595543532942] 2023-10-21T04:31:37.202+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:31:37.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex closed 2023-10-21T04:31:37.202+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:31:37.203+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:31:37.203+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:31:37.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex stopped 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.203+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.237+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:31:37.237+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 36450 2023-10-21T04:31:37.237+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 49.392696ms 2023-10-21T04:31:37.238+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 77.053725ms 2023-10-21T04:31:37.249+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.249+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.260+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:31:37.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex closed 2023-10-21T04:31:37.263+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.264+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex ... 2023-10-21T04:31:37.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex ... 2023-10-21T04:31:37.271+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:31:37.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex closed 2023-10-21T04:31:37.271+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.271+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.271+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.271+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:31:37.275+05:30 [Info] switched currmeta from 119 -> 120 force true 2023-10-21T04:31:37.276+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:31:37.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex closed 2023-10-21T04:31:37.277+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.277+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.277+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.282+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:31:37.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex ... 2023-10-21T04:31:37.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex ... 2023-10-21T04:31:37.269+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.278+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.297+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:31:37.297+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:31:37.298+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:31:37.298+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:31:37.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex stopped 2023-10-21T04:31:37.300+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.300+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.300+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.300+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.299+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.302+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:31:37.302+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:31:37.302+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:31:37.302+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:31:37.302+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex stopped 2023-10-21T04:31:37.303+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.303+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.303+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.303+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index destroyed 2023-10-21T04:31:37.318+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.318+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 5 Destroy Done 2023-10-21T04:31:37.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:31:37.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex closed 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:31:37.319+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:31:37.319+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:31:37.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex stopped 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:31:37.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex closed 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.319+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.335+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 192.024732ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:37.336+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.342+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:31:37.336+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:31:37.356+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:31:37.356+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:31:37.356+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:31:37.356+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:31:37.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex stopped 2023-10-21T04:31:37.356+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.357+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.357+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.357+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex closed 2023-10-21T04:31:37.357+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.358+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.358+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.367+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:31:37.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:31:37.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex closed 2023-10-21T04:31:37.360+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:31:37.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex closed 2023-10-21T04:31:37.362+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:31:37.368+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex closed 2023-10-21T04:31:37.362+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 10606939961365971246 instance id 14527226347372019068 real instance id 8732144615635911668 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:37.368+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 10606939961365971246 instance 8732144615635911668 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:31:37.368+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 10606939961365971246 instance id 8732144615635911668 2023-10-21T04:31:37.368+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 10606939961365971246 2023-10-21T04:31:37.368+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.368+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:31:37.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:31:37.365+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.370+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.370+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.371+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.371+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.372+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.370+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:31:37.372+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.374+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.374+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.374+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.378+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:31:37.378+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:31:37.378+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:31:37.378+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:31:37.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:31:37.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex closed 2023-10-21T04:31:37.378+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:31:37.378+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:31:37.379+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:31:37.379+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:31:37.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:31:37.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex closed 2023-10-21T04:31:37.379+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.379+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.380+05:30 [Info] switched currmeta from 120 -> 121 force true 2023-10-21T04:31:37.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex ... 2023-10-21T04:31:37.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex ... 2023-10-21T04:31:37.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:31:37.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex closed 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:31:37.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex stopped 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:31:37.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:31:37.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex stopped 2023-10-21T04:31:37.380+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.383+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.384+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.384+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.384+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.384+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.387+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex ... 2023-10-21T04:31:37.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex ... 2023-10-21T04:31:37.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index destroyed 2023-10-21T04:31:37.390+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.390+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 7 Destroy Done 2023-10-21T04:31:37.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex ... 2023-10-21T04:31:37.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex ... 2023-10-21T04:31:37.391+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 8732144615635911668 &{1} 2023-10-21T04:31:37.391+05:30 [Info] Indexer::handleDropIndex - IndexInstId 8732144615635911668 2023-10-21T04:31:37.392+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 10606939961365971246, instId: 8732144615635911668 2023-10-21T04:31:37.397+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [8732144615635911668] 2023-10-21T04:31:37.397+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [8732144615635911668] 2023-10-21T04:31:37.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.393+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.396+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.398+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:37.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index destroyed 2023-10-21T04:31:37.399+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.401+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 2 Destroy Done 2023-10-21T04:31:37.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex ... 2023-10-21T04:31:37.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex ... 2023-10-21T04:31:37.402+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:31:37.403+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:31:37.403+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:31:37.403+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:31:37.400+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [8732144615635911668] 2023-10-21T04:31:37.400+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex stopped 2023-10-21T04:31:37.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex ... 2023-10-21T04:31:37.404+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 2, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.404+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 3, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.405+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 4, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.405+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 5, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.406+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex ... 2023-10-21T04:31:37.406+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 6, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.407+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 8, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.407+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 1, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.408+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.406+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 7, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.408+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.408+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.408+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.407+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:31:37.409+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.409+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex sucessfully destroyed 2023-10-21T04:31:37.410+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.412+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.413+05:30 [Info] switched currmeta from 121 -> 122 force true 2023-10-21T04:31:37.413+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.414+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.414+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.414+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 22.26516ms 2023-10-21T04:31:37.414+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 22.297359ms 2023-10-21T04:31:37.414+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 8732144615635911668 2023-10-21T04:31:37.414+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:37.416+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index destroyed 2023-10-21T04:31:37.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:37.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:37.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:37.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.417+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.418+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.419+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:37.422+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:31:37.423+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.432+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 69.409906ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:37.433+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:37.434+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.434+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:37.434+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:37.434+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:31:37.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:31:37.434+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.435+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:37.435+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:31:37.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:37.436+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:37.436+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:37.436+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:37.436+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:31:37.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:31:37.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex ... 2023-10-21T04:31:37.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex ... 2023-10-21T04:31:37.434+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex closed 2023-10-21T04:31:37.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:31:37.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex closed 2023-10-21T04:31:37.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:31:37.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex closed 2023-10-21T04:31:37.438+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.438+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex ... 2023-10-21T04:31:37.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex ... 2023-10-21T04:31:37.434+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 1 Destroy Done 2023-10-21T04:31:37.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:37.439+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.439+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.439+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.439+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex sucessfully destroyed 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:37.443+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:31:37.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index destroyed 2023-10-21T04:31:37.444+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.444+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 6 Destroy Done 2023-10-21T04:31:37.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.444+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex ... 2023-10-21T04:31:37.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex ... 2023-10-21T04:31:37.474+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.474+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.474+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex ... 2023-10-21T04:31:37.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex ... 2023-10-21T04:31:37.478+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12266944006962282785 &{1} 2023-10-21T04:31:37.479+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12266944006962282785 2023-10-21T04:31:37.479+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 12824340686309573428, instId: 12266944006962282785 2023-10-21T04:31:37.479+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:31:37.479+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:37.479+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12266944006962282785 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:31:37.479+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12266944006962282785] 2023-10-21T04:31:37.480+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12266944006962282785] 2023-10-21T04:31:37.480+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:37.480+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12266944006962282785] 2023-10-21T04:31:37.480+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:37.481+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 12266944006962282785 Defn: DefnId: 12824340686309573428 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:37.481+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.590103ms 2023-10-21T04:31:37.482+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12266944006962282785, PartitionId 0, IndexDefnId 12824340686309573428. 2023-10-21T04:31:37.483+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.483+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.483+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex] 2023-10-21T04:31:37.483+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12266944006962282785 2023-10-21T04:31:37.484+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [12266944006962282785] 2023-10-21T04:31:37.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex ... 2023-10-21T04:31:37.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex ... 2023-10-21T04:31:37.484+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index destroyed 2023-10-21T04:31:37.487+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.487+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 3 Destroy Done 2023-10-21T04:31:37.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:31:37.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:31:37.487+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.487+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.488+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.488+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:37.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex ... 2023-10-21T04:31:37.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex ... 2023-10-21T04:31:37.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex sucessfully destroyed 2023-10-21T04:31:37.493+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:31:37.493+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex] 2023-10-21T04:31:37.494+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex] 2023-10-21T04:31:37.494+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex 2023-10-21T04:31:37.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex stopped All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index destroyed 2023-10-21T04:31:37.494+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.494+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 8 Destroy Done 2023-10-21T04:31:37.494+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.495+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.495+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:31:37.495+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 36450 2023-10-21T04:31:37.495+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 13.685037ms 2023-10-21T04:31:37.495+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 16.394545ms 2023-10-21T04:31:37.495+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.499+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex ... 2023-10-21T04:31:37.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex ... 2023-10-21T04:31:37.510+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:37.512+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:37.512+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:37.512+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:31:37.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:31:37.512+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.512+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.512+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.512+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.518+05:30 [Info] switched currmeta from 122 -> 124 force true 2023-10-21T04:31:37.530+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.544+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index destroyed 2023-10-21T04:31:37.546+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4, IndexDefnId 8910459001587546151. 2023-10-21T04:31:37.546+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 4 Destroy Done 2023-10-21T04:31:37.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:31:37.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:31:37.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.548+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.548+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:37.548+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:37.548+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:37.548+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:37.548+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:31:37.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:31:37.548+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.549+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.549+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.549+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:31:37.549+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:31:37.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.548+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.553+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:31:37.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:31:37.554+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.554+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:37.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:31:37.554+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.557+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.557+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:37.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex] 2023-10-21T04:31:37.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex closed 2023-10-21T04:31:37.557+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.557+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.557+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.557+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex] 2023-10-21T04:31:37.559+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:31:37.566+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:31:37.567+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.567+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.567+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.567+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:37.563+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:31:37.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:31:37.568+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.568+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.568+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.568+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:37.570+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:31:37.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:31:37.570+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.570+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.570+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.570+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:37.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.572+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 131.496675ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:37.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:37.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:31:37.577+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:37.577+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:37.577+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:37.577+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:31:37.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:31:37.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.577+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:31:37.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:31:37.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.580+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:37.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:31:37.582+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.588+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 17434304260397056806 &{1} 2023-10-21T04:31:37.589+05:30 [Info] Indexer::handleDropIndex - IndexInstId 17434304260397056806 2023-10-21T04:31:37.589+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 3381524789655566900, instId: 17434304260397056806 2023-10-21T04:31:37.590+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:37.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.601+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:31:37.601+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex] 2023-10-21T04:31:37.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:31:37.611+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 2, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.611+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 2 Destroy Done 2023-10-21T04:31:37.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:31:37.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:31:37.611+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.611+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.611+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:37.602+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:37.612+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:31:37.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex stopped 2023-10-21T04:31:37.614+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.614+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.614+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.614+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.614+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:37.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:31:37.611+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.609+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:31:37.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:31:37.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:31:37.613+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:31:37.599+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:37.612+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.616+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.616+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.616+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.616+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.616+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.616+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.617+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.617+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.617+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.617+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.619+05:30 [Info] switched currmeta from 124 -> 126 force true 2023-10-21T04:31:37.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:31:37.620+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 3, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.620+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 3 Destroy Done 2023-10-21T04:31:37.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:31:37.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:31:37.620+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.620+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.620+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.620+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:37.624+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.624+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.624+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:31:37.626+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.627+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:37.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:31:37.629+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:31:37.630+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [17434304260397056806] 2023-10-21T04:31:37.630+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [17434304260397056806] 2023-10-21T04:31:37.630+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:31:37.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.633+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:37.633+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:37.633+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:37.633+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:31:37.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:31:37.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.631+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [17434304260397056806] 2023-10-21T04:31:37.634+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:31:37.634+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:31:37.634+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:37.635+05:30 [Info] MutationStreamReader:Shutdown StreamReader MAINT_STREAM 2023-10-21T04:31:37.635+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:37.636+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:31:37.636+05:30 [Error] DATP[->dataport ":9123"] Accept() Error: accept tcp4 0.0.0.0:9123: use of closed network connection 2023-10-21T04:31:37.636+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9123->127.0.0.1:53336: use of closed network connection 2023-10-21T04:31:37.636+05:30 [Error] DATP[->dataport ":9123"] worker "127.0.0.1:53336" exit: read tcp4 127.0.0.1:9123->127.0.0.1:53336: use of closed network connection 2023-10-21T04:31:37.636+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:53336" closed ! 2023-10-21T04:31:37.636+05:30 [Info] DATP[->dataport ":9123"] ... stopped 2023-10-21T04:31:37.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:37.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:31:37.638+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0, IndexDefnId 3381524789655566900. 2023-10-21T04:31:37.638+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:31:37.638+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:31:37.638+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:31:37.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:31:37.642+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:31:37.672+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:31:37.672+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:31:37.655+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.670+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.673+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:31:37.676+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 87.316332ms 2023-10-21T04:31:37.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex] 2023-10-21T04:31:37.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex closed 2023-10-21T04:31:37.677+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.677+05:30 [Info] Indexer::cleanupIndexData IndexInst 12266944006962282785 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex ... 2023-10-21T04:31:37.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex ... 2023-10-21T04:31:37.679+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.679+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.680+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.683+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 17434304260397056806 2023-10-21T04:31:37.684+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2] 2023-10-21T04:31:37.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:31:37.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:31:37.691+05:30 [Info] switched currmeta from 126 -> 126 force true 2023-10-21T04:31:37.691+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:37.685+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:31:37.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:31:37.691+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.691+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:37.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:31:37.692+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:37.692+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:37.692+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:37.693+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8] 12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:37.693+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[1 2 3 4 5 6 7 8] 12266944006962282785:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:31:37.694+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:31:37.701+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 1, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.701+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 1 Destroy Done 2023-10-21T04:31:37.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:31:37.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:31:37.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:31:37.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:31:37.701+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:31:37.702+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:31:37.702+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.702+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.702+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.702+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.703+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.702+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:37.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:31:37.710+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2]. Err genServer.closed 2023-10-21T04:31:37.710+05:30 [Error] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:31:37.711+05:30 [Info] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Treating genServer.closed As Success 2023-10-21T04:31:37.711+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 36450 2023-10-21T04:31:37.710+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:31:37.711+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:31:37.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex stopped 2023-10-21T04:31:37.711+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:31:37.712+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:31:37.712+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:31:37.712+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:31:37.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex ... 2023-10-21T04:31:37.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex ... 2023-10-21T04:31:37.711+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 34.528131ms 2023-10-21T04:31:37.715+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 126.18407ms 2023-10-21T04:31:37.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:37.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:31:37.721+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.721+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.728+05:30 [Info] switched currmeta from 126 -> 126 force true 2023-10-21T04:31:37.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index/mainIndex sucessfully destroyed 2023-10-21T04:31:37.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:31:37.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12266944006962282785_0.index destroyed 2023-10-21T04:31:37.731+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12266944006962282785, PartitionId 0, IndexDefnId 12824340686309573428. 2023-10-21T04:31:37.731+05:30 [Info] Indexer::cleanupIndexData IndexInst 12266944006962282785 Partition 0 Destroy Done 2023-10-21T04:31:37.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:37.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:31:37.734+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:31:37.736+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 6, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.736+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 6 Destroy Done 2023-10-21T04:31:37.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:37.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:31:37.738+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:31:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:31:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:31:37.740+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.740+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:37.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:31:37.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:31:37.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.748+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:31:37.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:37.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:31:37.750+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 4, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.751+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 4 Destroy Done 2023-10-21T04:31:37.751+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 176.622984ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:37.752+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17406544375239214575 10170029854712240877] 2023-10-21T04:31:37.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed 2023-10-21T04:31:37.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:31:37.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex closed 2023-10-21T04:31:37.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:31:37.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:31:37.760+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:31:37.762+05:30 [Info] Indexer::cleanupIndexData IndexInst 17434304260397056806 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:31:37.762+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 5, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.762+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 5 Destroy Done 2023-10-21T04:31:37.762+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:37.762+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:37.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:31:37.769+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 7, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.769+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 7 Destroy Done 2023-10-21T04:31:37.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:37.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:31:37.769+05:30 [Info] switched currmeta from 126 -> 127 force true 2023-10-21T04:31:37.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:31:37.774+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 8, IndexDefnId 10606939961365971246. 2023-10-21T04:31:37.774+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 8 Destroy Done 2023-10-21T04:31:37.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex ... 2023-10-21T04:31:37.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex ... 2023-10-21T04:31:37.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:31:37.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index destroyed 2023-10-21T04:31:37.793+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0, IndexDefnId 3381524789655566900. 2023-10-21T04:31:37.793+05:30 [Info] Indexer::cleanupIndexData IndexInst 17434304260397056806 Partition 0 Destroy Done 2023-10-21T04:31:38.064+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9980648842899658 2023-10-21T04:31:38.113+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 10170029854712240877 2023-10-21T04:31:38.113+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17406544375239214575 2023-10-21T04:31:38.113+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17406544375239214575 2023-10-21T04:31:38.113+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 10170029854712240877 2023-10-21T04:31:38.113+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17406544375239214575 10170029854712240877], elapsed: 360.755755ms 2023-10-21T04:31:38.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:38.113+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) removed 2023-10-21T04:31:38.113+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper stopped 2023-10-21T04:31:38.113+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': stopped 2023-10-21T04:31:38.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:38.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:38.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:38.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:38.113+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All daemons stopped 2023-10-21T04:31:38.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:38.113+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.Close: Begin close 2023-10-21T04:31:38.114+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:38.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.Close: Done close 2023-10-21T04:31:38.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:38.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:38.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:38.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:38.116+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.Close: Begin close 2023-10-21T04:31:38.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.Close: Done close 2023-10-21T04:31:38.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:38.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:38.119+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:38.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All instance closed 2023-10-21T04:31:38.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shutdown completed 2023-10-21T04:31:38.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:38.119+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) removed 2023-10-21T04:31:38.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All daemons stopped 2023-10-21T04:31:38.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All instance closed 2023-10-21T04:31:38.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : All instance destroyed 2023-10-21T04:31:38.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:31:38.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:38.121+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) removed 2023-10-21T04:31:38.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:38.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:38.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:31:38.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:31:38.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All daemons stopped 2023-10-21T04:31:38.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:38.121+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.Close: Begin close 2023-10-21T04:31:38.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.Close: Done close 2023-10-21T04:31:38.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:38.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:38.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:38.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:31:38.125+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.Close: Begin close 2023-10-21T04:31:38.126+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.Close: Done close 2023-10-21T04:31:38.126+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:31:38.126+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:31:38.126+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:31:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All instance closed 2023-10-21T04:31:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shutdown completed 2023-10-21T04:31:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:31:38.126+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) removed 2023-10-21T04:31:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All daemons stopped 2023-10-21T04:31:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All instance closed 2023-10-21T04:31:38.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : All instance destroyed 2023-10-21T04:31:38.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:31:38.128+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': Cleaned up Tenant with name:bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:31:38.129+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17406544375239214575 10170029854712240877], elapsed(sec): 0.376614998 2023-10-21T04:31:38.137+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:38.155+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokena9:85:c:da:3c:ac:ab:94 2023-10-21T04:31:38.155+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: c596d1e7da6bfc94cbe77df771748cc5 TransferTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:31:38.155+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000126251 2023-10-21T04:31:38.155+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:31:38.155+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:31:38.155+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.454µs 2023-10-21T04:31:38.155+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardTokena9_85_c_da_3c_ac_ab_94, shardId :0 2023-10-21T04:31:38.155+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardTokena9_85_c_da_3c_ac_ab_94, shardId :0 2023-10-21T04:31:38.155+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:31:38.155+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:31:38.155+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:31:38.160+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:31:38.160+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:31:38.160+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:31:38.160+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:31:38.160+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 74.583µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:38.161+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:31:38.162+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:31:38.162+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:31:38.162+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:31:38.162+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:31:38.162+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:31:38.163+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:31:38.163+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 68.126µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:38.169+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:31:38.169+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:31:38.170+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 3.848813654s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:31:38.170+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/c596d1e7da6bfc94cbe77df771748cc5, rev: [] call. err: 2023-10-21T04:31:38.170+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:31:38.170+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:31:38.170+05:30 [Warn] util::traceLockLog: Long wait for svcMgrMu.Lock() 3.029249062s in RebalanceServiceManager::rebalanceJanitor 2023-10-21T04:31:38.187+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:31:38.187+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:31:38.198+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:31:38.198+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:31:38.206+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Found 2 tokens. Cleaning up. 2023-10-21T04:31:38.206+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:31:38.206+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokena9:85:c:da:3c:ac:ab:94 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:7c:cc:96:58:f4:7c:a8 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14217467447424830582 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 15660027118472860698 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7293622283379232651 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:38.206+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenae:b8:e2:e7:52:ef:e5:ce MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: c596d1e7da6bfc94cbe77df771748cc5 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken2e:2c:44:28:da:2:14:94 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2214262137454092125 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17803515799466969650 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17205882817792019450 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:31:38.206+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:31:38.208+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:31:38.244+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokena9:85:c:da:3c:ac:ab:94 2023-10-21T04:31:38.244+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: c596d1e7da6bfc94cbe77df771748cc5 TransferTokenId: ShardTokena9:85:c:da:3c:ac:ab:94 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:31:38.244+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000180178 2023-10-21T04:31:38.244+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardTokena9_85_c_da_3c_ac_ab_94, shardId :0 2023-10-21T04:31:38.244+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/c596d1e7da6bfc94cbe77df771748cc5_ShardTokena9_85_c_da_3c_ac_ab_94, shardId :0 2023-10-21T04:31:38.585+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:38.594+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:38.594+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:38.601+05:30 [Info] switched currmeta from 127 -> 127 force true 2023-10-21T04:31:39.062+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9975589840334836 2023-10-21T04:31:39.124+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:39.767+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:31:39.917+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:39.917+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:39.919+05:30 [Info] switched currmeta from 127 -> 127 force true 2023-10-21T04:31:40.065+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9916316407427346 2023-10-21T04:31:40.132+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:40.250+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:31:41.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.772155197367533, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:31:41.141+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:41.615+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57464 2023-10-21T04:31:41.644+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 22.030701ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:41.649+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 11.392µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:41.766+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40826" doAuth() ... 2023-10-21T04:31:41.767+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:40826" doReceive() ... 2023-10-21T04:31:41.808+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:41.809+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:41.813+05:30 [Info] switched currmeta from 127 -> 132 force true 2023-10-21T04:31:41.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:41.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:41.863+05:30 [Info] switched currmeta from 132 -> 133 force true 2023-10-21T04:31:41.881+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:41.881+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:41.890+05:30 [Info] switched currmeta from 133 -> 134 force true 2023-10-21T04:31:41.890+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14217467447424830582:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:31:41.897+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:41.898+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:41.903+05:30 [Info] switched currmeta from 134 -> 134 force true 2023-10-21T04:31:42.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7523818203487027, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:31:42.145+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:31:42.147+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:42.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6043349427976659583:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:31:42.827+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:42.827+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:42.834+05:30 [Info] switched currmeta from 134 -> 135 force true 2023-10-21T04:31:42.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:42.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:42.855+05:30 [Info] switched currmeta from 135 -> 135 force true 2023-10-21T04:31:42.876+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:42.876+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:42.885+05:30 [Info] switched currmeta from 135 -> 136 force true 2023-10-21T04:31:42.901+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:42.901+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:42.904+05:30 [Info] switched currmeta from 136 -> 136 force true 2023-10-21T04:31:42.973+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:42.973+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:42.979+05:30 [Info] switched currmeta from 136 -> 138 force true 2023-10-21T04:31:43.007+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:43.007+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:43.014+05:30 [Info] switched currmeta from 138 -> 139 force true 2023-10-21T04:31:43.031+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:43.031+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:43.040+05:30 [Info] switched currmeta from 139 -> 139 force true 2023-10-21T04:31:43.050+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:43.050+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:43.052+05:30 [Info] switched currmeta from 139 -> 139 force true 2023-10-21T04:31:43.070+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:43.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:43.075+05:30 [Info] switched currmeta from 139 -> 140 force true 2023-10-21T04:31:43.153+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:43.600+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:43.615+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:43.616+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:43.621+05:30 [Info] switched currmeta from 140 -> 140 force true 2023-10-21T04:31:44.159+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:44.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:31:44.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[4 5 6 7 8 1 2 3] 17205882817792019450:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:31:44.873+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[4 5 6 7 8 1 2 3] 17205882817792019450:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 1697842790443813331 2023-10-21T04:31:45.164+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:46.169+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:47.174+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:48.175+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:31:48.179+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:48.205+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.205+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.208+05:30 [Info] switched currmeta from 140 -> 140 force true 2023-10-21T04:31:48.255+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.255+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.260+05:30 [Info] switched currmeta from 140 -> 141 force true 2023-10-21T04:31:48.325+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.325+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.329+05:30 [Info] switched currmeta from 141 -> 142 force true 2023-10-21T04:31:48.356+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.356+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.364+05:30 [Info] switched currmeta from 142 -> 142 force true 2023-10-21T04:31:48.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:48.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:48.373+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:48.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.391+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.401+05:30 [Info] switched currmeta from 142 -> 143 force true 2023-10-21T04:31:48.465+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.465+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.469+05:30 [Info] switched currmeta from 143 -> 143 force true 2023-10-21T04:31:48.509+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.509+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.513+05:30 [Info] switched currmeta from 143 -> 144 force true 2023-10-21T04:31:48.560+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.560+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.565+05:30 [Info] switched currmeta from 144 -> 145 force true 2023-10-21T04:31:48.599+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.599+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.603+05:30 [Info] switched currmeta from 145 -> 145 force true 2023-10-21T04:31:48.645+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.645+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.653+05:30 [Info] switched currmeta from 145 -> 146 force true 2023-10-21T04:31:48.701+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.702+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.709+05:30 [Info] switched currmeta from 146 -> 148 force true 2023-10-21T04:31:48.720+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.720+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.730+05:30 [Info] switched currmeta from 148 -> 148 force true 2023-10-21T04:31:48.798+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:48.808+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:48.808+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:48.814+05:30 [Info] switched currmeta from 148 -> 148 force true 2023-10-21T04:31:48.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:48.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:48.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:48.871+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0] 17205882817792019450:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:31:49.099+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57822 2023-10-21T04:31:49.112+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 10.487289ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:31:49.185+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:49.613+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:31:50.198+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40946 2023-10-21T04:31:51.061+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9835377761503723, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:31:51.209+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57822. Reason = EOF. Cleaning Pipe. 2023-10-21T04:31:51.209+05:30 [Info] messageListener.start(): message channel closed. Remove peer ff:41:aa:0:89:f5:f8:89 as follower. 2023-10-21T04:31:52.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8526048429149883, throttleDelayMs (new, old, change): (0, 3, -3) 2023-10-21T04:31:53.663+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:31:53.672+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:31:53.672+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:31:53.675+05:30 [Info] switched currmeta from 148 -> 148 force true 2023-10-21T04:32:03.098+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 252] aggregate storage info: map[bucket_%2:map[kv:2304192 index:4350220] bucket_1:map[kv:2369728 index:10023159]] 2023-10-21T04:32:03.804+05:30 [Info] janitor: running cleanup. 2023-10-21T04:32:03.804+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[18046371353728341033 18323460706027642575 4535197604682045916 573521267164877540 6985049432225870291 12140458060295645238 16162012140118277065 3674078063353481907 14889401051926000176 249887442666643345 4138047918192025493 11619165464560280276 12027054384338714804 12833670934079048728 8378417662949994532 15968368593314043432 17656144409930763141 7392796230534034390 5465835793930010953 8669649359784409226 8857269605176042330 18026660730857541767 2048006213723315482 4860549750398470299 12195715103909237684 12338036531800130039 15431181238515168303 4532813151040661705 7525085113233084253 8504686614261377811 13856704970496601958 14292583894719865678 14741949230702299196 11365633301282701856 1252359852432998131 15945117906390072636]} 2023-10-21T04:32:03.804+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:32:03.815+05:30 [Info] memstats {"Alloc":118973368, "TotalAlloc":121745928, "Sys":138690840, "Lookups":0, "Mallocs":233282,"Frees":45925, "HeapAlloc":118973368, "HeapSys":122978304, "HeapIdle":1744896, "HeapInuse":121233408,"HeapReleased":1564672, "HeapObjects":187357,"MSpanInuse":464960, "MSpanSys": 473280, "StackInuse": 2850816,"GCSys":10035336, "LastGC":1697842862908656612,"PauseTotalNs":2435903, "PauseNs":[451244, 393769, 40493, 286392, 1264005], "NumGC":5} 2023-10-21T04:32:04.000+05:30 [Info] connected with 4 indexers 2023-10-21T04:32:04.000+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:32:04.051+05:30 [Info] num concurrent scans {0} 2023-10-21T04:32:04.051+05:30 [Info] average scan response {0 ms} 2023-10-21T04:32:04.055+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:32:04.068+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:32:04.093+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:32:04.104+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:32:04.814+05:30 [Info] Indexer::ReadMemstats Time Taken 688.32µs 2023-10-21T04:32:07.080+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:07.080+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:07.102+05:30 [Info] switched currmeta from 148 -> 152 force true 2023-10-21T04:32:07.166+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:07.166+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:07.174+05:30 [Info] switched currmeta from 152 -> 152 force true 2023-10-21T04:32:07.222+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:07.223+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:07.237+05:30 [Info] switched currmeta from 152 -> 153 force true 2023-10-21T04:32:07.289+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:07.289+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:07.306+05:30 [Info] switched currmeta from 153 -> 156 force true 2023-10-21T04:32:07.323+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:07.323+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:07.326+05:30 [Info] switched currmeta from 156 -> 156 force true 2023-10-21T04:32:08.207+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:32:08.207+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:32:08.208+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:32:08.208+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:32:08.209+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:32:08.209+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:32:08.210+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:32:08.245+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:32:08.702+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:32:08.707+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:08.707+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:08.709+05:30 [Info] switched currmeta from 156 -> 156 force true 2023-10-21T04:32:27.833+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 4790214984019693894 2023-10-21T04:32:27.833+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 250.428µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:27.891+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43356 2023-10-21T04:32:27.896+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43356 2023-10-21T04:32:27.920+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43356 2023-10-21T04:32:27.971+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 4790214984019693894 2023-10-21T04:32:27.971+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 330.677µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:28.017+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.018+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.020+05:30 [Info] switched currmeta from 156 -> 159 force true 2023-10-21T04:32:28.027+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.027+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.029+05:30 [Info] switched currmeta from 159 -> 159 force true 2023-10-21T04:32:28.066+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.066+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.068+05:30 [Info] switched currmeta from 159 -> 161 force true 2023-10-21T04:32:28.086+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.086+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.088+05:30 [Info] switched currmeta from 161 -> 162 force true 2023-10-21T04:32:28.102+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.102+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.119+05:30 [Info] switched currmeta from 162 -> 163 force true 2023-10-21T04:32:28.150+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.150+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.159+05:30 [Info] switched currmeta from 163 -> 164 force true 2023-10-21T04:32:28.229+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.229+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.242+05:30 [Info] switched currmeta from 164 -> 165 force true 2023-10-21T04:32:28.250+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.250+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.255+05:30 [Info] switched currmeta from 165 -> 165 force true 2023-10-21T04:32:28.801+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:32:28.807+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:28.808+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:28.812+05:30 [Info] switched currmeta from 165 -> 165 force true 2023-10-21T04:32:28.955+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58554 2023-10-21T04:32:28.979+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.392933ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:29.038+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 4790214984019693894. No update is performed. 2023-10-21T04:32:29.038+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 124.127µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:29.063+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58554. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:29.063+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6a:1d:b1:c8:a4:2f:13:a2 as follower. 2023-10-21T04:32:29.103+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:32:29.103+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:32:29.162+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:32:29.228+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:32:29.228+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:32:29.228+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:32:29.228+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58574 2023-10-21T04:32:29.239+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58578 2023-10-21T04:32:29.244+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:32:29.258+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.945265ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:29.275+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:32:29.275+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:32:29.275+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:32:29.276+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 31.57336ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:29.289+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58582 2023-10-21T04:32:29.289+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:32:29.317+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.381018ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:29.318+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:32:29.318+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:32:29.318+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:32:29.330+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58578. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:29.330+05:30 [Info] messageListener.start(): message channel closed. Remove peer e2:d8:27:fb:16:b9:88:50 as follower. 2023-10-21T04:32:29.331+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58574. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:29.331+05:30 [Info] messageListener.start(): message channel closed. Remove peer 72:39:24:ad:7:10:af:f9 as follower. 2023-10-21T04:32:29.343+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:32:29.368+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:32:29.368+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:32:29.368+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:32:29.368+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:32:29.368+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:32:29.368+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:32:29.368+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:32:29.368+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:32:29.368+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:32:29.368+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:32:29.368+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:49818->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:29.368+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:50286->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:29.369+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:58582->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:29.369+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:54962->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:29.369+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58582. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:29.370+05:30 [Info] messageListener.start(): message channel closed. Remove peer ce:28:32:c0:5d:cf:0:64 as follower. 2023-10-21T04:32:29.480+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:29.502+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:29.502+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:29.508+05:30 [Info] switched currmeta from 165 -> 166 force true 2023-10-21T04:32:29.528+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:29.528+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:29.533+05:30 [Info] switched currmeta from 166 -> 167 force true 2023-10-21T04:32:29.759+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:29.777+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:29.801+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:29.802+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:29.811+05:30 [Info] switched currmeta from 167 -> 168 force true 2023-10-21T04:32:29.818+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:29.864+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:29.864+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:29.868+05:30 [Info] switched currmeta from 168 -> 169 force true 2023-10-21T04:32:30.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10402379389576076017:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:32:30.371+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10402379389576076017:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:32:31.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4083560586502645495:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:32:33.883+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:32:33.943+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:33.943+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:33.948+05:30 [Info] switched currmeta from 169 -> 169 force true 2023-10-21T04:32:34.682+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 13268712595133993155 2023-10-21T04:32:34.682+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 182.312µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:34.786+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43356 2023-10-21T04:32:34.795+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43356 2023-10-21T04:32:34.829+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43356 2023-10-21T04:32:34.895+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 13268712595133993155 2023-10-21T04:32:34.895+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 247.716µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:34.932+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:34.932+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:34.935+05:30 [Info] switched currmeta from 169 -> 171 force true 2023-10-21T04:32:34.946+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:34.947+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:34.985+05:30 [Info] switched currmeta from 171 -> 172 force true 2023-10-21T04:32:35.017+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:35.017+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:35.025+05:30 [Info] switched currmeta from 172 -> 175 force true 2023-10-21T04:32:35.063+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9885272382506025, throttleDelayMs (new, old, change): (4, 0, 4) 2023-10-21T04:32:35.100+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:35.100+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:35.117+05:30 [Info] switched currmeta from 175 -> 178 force true 2023-10-21T04:32:35.160+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:35.160+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:35.164+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:32:35.847+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58770 2023-10-21T04:32:35.890+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 38.764862ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:35.942+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 13268712595133993155. No update is performed. 2023-10-21T04:32:35.942+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 114.881µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:35.953+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58770. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:35.953+05:30 [Info] messageListener.start(): message channel closed. Remove peer 1d:aa:c:fd:f2:9c:c4:7c as follower. 2023-10-21T04:32:36.064+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58774 2023-10-21T04:32:36.074+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:32:36.074+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:32:36.087+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 18.45266ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:36.091+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:32:36.130+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:32:36.130+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:32:36.130+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:32:36.157+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:32:36.204+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58792 2023-10-21T04:32:36.227+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.923784ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:36.229+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58774. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:36.230+05:30 [Info] messageListener.start(): message channel closed. Remove peer 71:b:94:88:7b:c:9c:9b as follower. 2023-10-21T04:32:36.231+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:32:36.231+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:32:36.231+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:32:36.238+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58800 2023-10-21T04:32:36.239+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:32:36.256+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.036443ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:36.257+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:32:36.257+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:32:36.257+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:32:36.269+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:32:36.307+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:32:36.307+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:32:36.307+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:32:36.307+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:32:36.307+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:32:36.307+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:32:36.307+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:32:36.308+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:32:36.308+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:32:36.308+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:32:36.308+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:58800->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:36.308+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:55176->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:36.308+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:50030->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:36.309+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:50496->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:32:36.309+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58792. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:36.309+05:30 [Info] messageListener.start(): message channel closed. Remove peer d3:65:ae:81:7:78:a7:ba as follower. 2023-10-21T04:32:36.309+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58800. Reason = EOF. Cleaning Pipe. 2023-10-21T04:32:36.309+05:30 [Info] messageListener.start(): message channel closed. Remove peer 93:5b:a9:33:e6:43:9b:fc as follower. 2023-10-21T04:32:36.392+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:36.454+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:36.454+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:36.457+05:30 [Info] switched currmeta from 178 -> 180 force true 2023-10-21T04:32:36.482+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:36.482+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:36.484+05:30 [Info] switched currmeta from 180 -> 180 force true 2023-10-21T04:32:36.588+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:36.728+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:36.758+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:32:36.790+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:36.790+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:36.798+05:30 [Info] switched currmeta from 180 -> 181 force true 2023-10-21T04:32:36.876+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:36.876+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:36.883+05:30 [Info] switched currmeta from 181 -> 182 force true 2023-10-21T04:32:37.063+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8203464984092089, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:32:37.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4238649781004957942:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:32:37.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4238649781004957942:[0]]. New rollbacktime: 1697842641346326805, Old rollbacktime: 0 2023-10-21T04:32:38.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2670343739772083055:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:32:38.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2670343739772083055:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:32:38.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2670343739772083055:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:32:38.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2670343739772083055:[0]]. New rollbacktime: 1697842771555766255, Old rollbacktime: 0 2023-10-21T04:32:38.209+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:32:38.209+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:32:38.211+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:32:38.211+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:32:38.211+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:32:38.211+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:32:38.212+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:32:38.248+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:32:38.860+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:32:38.866+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:38.866+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:38.868+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:32:44.010+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:32:44.047+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:44.047+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:44.052+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:32:49.250+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:32:49.350+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:49.350+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:49.359+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:32:49.472+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:32:49.475+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:32:49.506+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:49.506+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:49.524+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 185] updated cluster rebalancing status to true 2023-10-21T04:32:49.524+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 185] updating rebalancing status to true 2023-10-21T04:32:49.525+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:32:49.761+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:49.762+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:49.768+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:32:51.063+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9788080802024547, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:32:52.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.911650133764767, throttleDelayMs (new, old, change): (0, 3, -3) 2023-10-21T04:32:52.104+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {1f784fc0142ee312f0d92d98bd188c9a [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:32:52.108+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.793µs 2023-10-21T04:32:52.108+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:32:52.108+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:32:52.108+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:32:52.111+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:32:52.111+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:52.111+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.84731ms 2023-10-21T04:32:52.111+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:32:52.111+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:32:52.111+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:32:52.111+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 8.403µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:52.111+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1f784fc0142ee312f0d92d98bd188c9a Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]}]} 2023-10-21T04:32:52.113+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:32:52.114+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:32:52.125+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {1f784fc0142ee312f0d92d98bd188c9a [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:32:52.128+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:32:52.163+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"1f784fc0142ee312f0d92d98bd188c9a","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:32:52.165+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:32:52.167+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:32:53.177+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:32:53.186+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:32:53.186+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:32:53.193+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:32:53.193+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:32:53.193+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:32:53.193+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.068156669s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:32:53.193+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:32:53.194+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1f784fc0142ee312f0d92d98bd188c9a Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/1f784fc0142ee312f0d92d98bd188c9a Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]}]} 2023-10-21T04:32:53.194+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:32:53.194+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:32:53.197+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:32:53.197+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:32:53.197+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1f784fc0142ee312f0d92d98bd188c9a Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/1f784fc0142ee312f0d92d98bd188c9a Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]}]} 2023-10-21T04:32:53.197+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:32:53.197+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:32:53.197+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:32:53.199+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:32:53.199+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:32:53.253+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{7f731cd927c72e28ff24e69497b93571 7f731cd927c72e28ff24e69497b93571 plasma map[] [{29 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{3345094951175719474 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{6377153279641237139 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 4790214984019693894 [{10402379389576076017 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908 15531034489713578274] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{10601152754683910907 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [4718895791748289908] []}] 1 0 0 1 false plasma 0}]}]} {29 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{6043349427976659583 4 1 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{6225851069678382014 1 0 [{1 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 13268712595133993155 [{4238649781004957942 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593 1249143295319901191] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{2535881715450691825 4 1 [{0 0 {[{0 0 7f731cd927c72e28ff24e69497b93571}]} {[] []} [7434639758576776593] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {964f96068adef144590fc7ab85b2311c 964f96068adef144590fc7ab85b2311c plasma map[] [{6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{17434304260397056806 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575] []}] 1 0 0 1 false plasma 0}]}]} {29 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{15864358084358521096 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{10032435514095102923 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 4790214984019693894 [{4083560586502645495 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812 2925063509317026816] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{3464125101225762400 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [16787445029868804812] []}] 1 0 0 0 false plasma 0}]}]} {29 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{8965923595543532942 4 1 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{8732144615635911668 1 0 [{1 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 13268712595133993155 [{2670343739772083055 4 1 [{0 0 {[{0 0 964f96068adef144590fc7ab85b2311c}]} {[] []} [17406544375239214575 10170029854712240877] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {18194302e3f10df59bd076e1c4142d0b 18194302e3f10df59bd076e1c4142d0b plasma map[] [{8 bucket_%2 _system _query []} {33 bucket_%2 _default c1 []} {26 bucket_1 _default c1 []} {11 bucket_1 _system _query []}] [] 0 0 0 false} {8806f5706747af4e635b8da5fcd8aeee 8806f5706747af4e635b8da5fcd8aeee plasma map[] [{12 bucket_%2 _system _query []} {43 bucket_1 _default c1 []} {8 bucket_1 _system _query []} {33 bucket_%2 _default c1 []}] [] 0 0 0 false}] map[]} 2023-10-21T04:32:53.336+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40798 2023-10-21T04:32:53.339+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:32:53.339+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:32:53.339+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:32:53.339+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:32:53.340+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40798 2023-10-21T04:32:53.364+05:30 [Info] Planner::processDeleteToken: processing 38 tokens for node 127.0.0.1:9004 2023-10-21T04:32:53.364+05:30 [Info] Planner::processDeleteToken: processing 38 tokens for node 127.0.0.1:9002 2023-10-21T04:32:53.364+05:30 [Info] Planner::processDeleteToken: processing 38 tokens for node 127.0.0.1:9001 2023-10-21T04:32:53.364+05:30 [Info] Planner::processDeleteToken: processing 38 tokens for node 127.0.0.1:9003 2023-10-21T04:32:53.365+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:40798 2023-10-21T04:32:53.375+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:32:53.375+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:32:53.375+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:32:53.375+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:32:53.425+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:32:53.425+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:32:53.425+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:32:53.425+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:32:53.425+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 10606939961365971246:6225851069678382014:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 4790214984019693894:10402379389576076017:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 11276338709617092948:10601152754683910907:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.425+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:3345094951175719474:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16337157693634799600:6377153279641237139:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 13268712595133993155:4238649781004957942:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 3381524789655566900:2535881715450691825:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:6043349427976659583:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 16961575949607441659:15864358084358521096:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 13268712595133993155:2670343739772083055:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 3381524789655566900:17434304260397056806:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 8910459001587546151:8965923595543532942:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.427+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 10606939961365971246:8732144615635911668:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 11276338709617092948:3464125101225762400:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 4790214984019693894:4083560586502645495:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Moving index 16337157693634799600:10032435514095102923:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:32:53.428+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17574152 Units 0 2023-10-21T04:32:53.428+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 17717246 Units 0 2023-10-21T04:32:53.428+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 632144 Units 23 Tenants 0 2023-10-21T04:32:53.428+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 488989 Units 6 Tenants 0 2023-10-21T04:32:53.428+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:32:53.428+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:32:53.428+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:32:53.428+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:32:53.428+05:30 [Info] ---------------------------------------- 2023-10-21T04:32:53.428+05:30 [Info] 2023-10-21T04:32:53.428+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:8806f5706747af4e635b8da5fcd8aeee, useLiveData:false 2023-10-21T04:32:53.428+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:32:53.428+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:32:53.428+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.428+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.428+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.428+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.428+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.428+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.428+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.428+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.428+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.428+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.428+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.428+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.428+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.428+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.428+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.428+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.428+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.428+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.428+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.428+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.428+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.428+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.428+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.428+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.428+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.428+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.429+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.429+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.429+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.429+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:4790214984019693894, instId:10402379389576076017, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.429+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.429+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:10601152754683910907, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:157293295 (150.007M), mem:5970 (5.83008K), overhead:157287325 (150.001M), min:157293295 (150.007M), data:5970 (5.83008K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.429+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.429+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.429+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.429+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0120 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.429+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.429+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:157327258 (150.039M), mem:35375 (34.5459K), overhead:157291883 (150.005M), min:157327258 (150.039M), data:35375 (34.5459K) cpu:0.0110 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.429+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.429+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.429+05:30 [Info] Index total memory:157327980 (150.04M), mem:36000 (35.1562K), overhead:157291980 (150.005M), min:157327980 (150.04M), data:36000 (35.1562K) cpu:0.0110 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.429+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.432+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.432+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.432+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.432+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.432+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.432+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.432+05:30 [Info] Index total memory:157327827 (150.04M), mem:35868 (35.0273K), overhead:157291959 (150.005M), min:157327827 (150.04M), data:35868 (35.0273K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.432+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.432+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.432+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.432+05:30 [Info] Index total memory:157328340 (150.04M), mem:36312 (35.4609K), overhead:157292028 (150.005M), min:157328340 (150.04M), data:36312 (35.4609K) cpu:0.0120 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.432+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.432+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.432+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.432+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0092 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.432+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.432+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.432+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.432+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.432+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.432+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.432+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.432+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:13268712595133993155, instId:4238649781004957942, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:2535881715450691825, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157293697 (150.007M), mem:6318 (6.16992K), overhead:157287379 (150.001M), min:157293697 (150.007M), data:6318 (6.16992K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157328090 (150.04M), mem:36096 (35.25K), overhead:157291994 (150.005M), min:157328090 (150.04M), data:36096 (35.25K) cpu:0.0098 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157328309 (150.04M), mem:36285 (35.4346K), overhead:157292024 (150.005M), min:157328309 (150.04M), data:36285 (35.4346K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157327695 (150.039M), mem:35754 (34.916K), overhead:157291941 (150.005M), min:157327695 (150.039M), data:35754 (34.916K) cpu:0.0092 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157328797 (150.04M), mem:36708 (35.8477K), overhead:157292089 (150.005M), min:157328797 (150.04M), data:36708 (35.8477K) cpu:0.0108 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.433+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.433+05:30 [Info] Index total memory:157327911 (150.04M), mem:35941 (35.0986K), overhead:157291970 (150.005M), min:157327911 (150.04M), data:35941 (35.0986K) cpu:0.0098 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.433+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.433+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157328160 (150.04M), mem:36156 (35.3086K), overhead:157292004 (150.005M), min:157328160 (150.04M), data:36156 (35.3086K) cpu:0.0102 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327148 (150.039M), mem:35280 (34.4531K), overhead:157291868 (150.005M), min:157327148 (150.039M), data:35280 (34.4531K) cpu:0.0086 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] 2023-10-21T04:32:53.434+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:18194302e3f10df59bd076e1c4142d0b, useLiveData:false 2023-10-21T04:32:53.434+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:32:53.434+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0116 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327869 (150.04M), mem:35904 (35.0625K), overhead:157291965 (150.005M), min:157327869 (150.04M), data:35904 (35.0625K) cpu:0.0124 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157326969 (150.039M), mem:35125 (34.3018K), overhead:157291844 (150.005M), min:157326969 (150.039M), data:35125 (34.3018K) cpu:0.0114 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327402 (150.039M), mem:35500 (34.668K), overhead:157291902 (150.005M), min:157327402 (150.039M), data:35500 (34.668K) cpu:0.0114 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327740 (150.039M), mem:35793 (34.9541K), overhead:157291947 (150.005M), min:157327740 (150.039M), data:35793 (34.9541K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327968 (150.04M), mem:35990 (35.1465K), overhead:157291978 (150.005M), min:157327968 (150.04M), data:35990 (35.1465K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0124 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15864358084358521096, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157326067 (150.038M), mem:34344 (33.5391K), overhead:157291723 (150.005M), min:157326067 (150.038M), data:34344 (33.5391K) cpu:0.0096 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:13268712595133993155, instId:2670343739772083055, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:17434304260397056806, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157292969 (150.006M), mem:5688 (5.55469K), overhead:157287281 (150.001M), min:157292969 (150.006M), data:5688 (5.55469K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327056 (150.039M), mem:35200 (34.375K), overhead:157291856 (150.005M), min:157327056 (150.039M), data:35200 (34.375K) cpu:0.0128 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0122 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.434+05:30 [Info] Index total memory:157328167 (150.04M), mem:36162 (35.3145K), overhead:157292005 (150.005M), min:157328167 (150.04M), data:36162 (35.3145K) cpu:0.0122 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.434+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.434+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.434+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:157327287 (150.039M), mem:35400 (34.5703K), overhead:157291887 (150.005M), min:157327287 (150.039M), data:35400 (34.5703K) cpu:0.0118 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:157328319 (150.04M), mem:36294 (35.4434K), overhead:157292025 (150.005M), min:157328319 (150.04M), data:36294 (35.4434K) cpu:0.0138 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:157327325 (150.039M), mem:35433 (34.6025K), overhead:157291892 (150.005M), min:157327325 (150.039M), data:35433 (34.6025K) cpu:0.0126 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:157327252 (150.039M), mem:35370 (34.541K), overhead:157291882 (150.005M), min:157327252 (150.039M), data:35370 (34.541K) cpu:0.0130 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:157326760 (150.038M), mem:34944 (34.125K), overhead:157291816 (150.005M), min:157326760 (150.038M), data:34944 (34.125K) cpu:0.0112 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:3464125101225762400, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:157293828 (150.007M), mem:6432 (6.28125K), overhead:157287396 (150.001M), min:157293828 (150.007M), data:6432 (6.28125K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:4790214984019693894, instId:4083560586502645495, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.435+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.435+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.435+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.435+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.436+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.436+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.436+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.436+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.436+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.436+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.436+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.436+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.436+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.436+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.436+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.436+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.436+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.436+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.436+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.436+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.436+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.436+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.436+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.436+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.436+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.436+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.436+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:32:53.436+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10032435514095102923, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [16787445029868804812 2925063509317026816], shardIdsInDefn: map[] 2023-10-21T04:32:53.436+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:32:53.436+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:32:53.436+05:30 [Info] **************************************** 2023-10-21T04:32:53.437+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 632144 Units 23 Tenants 0 2023-10-21T04:32:53.437+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 488989 Units 6 Tenants 0 2023-10-21T04:32:53.437+05:30 [Info] Generating Shard Transfer Token (ShardTokena2:b2:c5:e2:b2:69:50:c0) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: Region: InstId: 2670343739772083055 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 16711923683289406198 RealInstId: 8965923595543532942 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3283688418228810121 RealInstId: 8732144615635911668 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17434304260397056806 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:32:53.438+05:30 [Info] Generating Shard Transfer Token (ShardToken7f:de:a2:ba:54:17:8e:fe) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: Region: InstId: 8248238408530887194 RealInstId: 6225851069678382014 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6225851069678382014 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4238649781004957942 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4238649781004957942 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2332250308932951681 RealInstId: 6043349427976659583 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6043349427976659583 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2535881715450691825 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2535881715450691825 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:32:53.439+05:30 [Info] Generating Shard Transfer Token (ShardTokenff:34:4:b4:54:ed:73:4a) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: Region: InstId: 10402379389576076017 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10402379389576076017 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10465155089296381829 RealInstId: 3345094951175719474 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3345094951175719474 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 2062526591622067910 RealInstId: 6377153279641237139 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 6377153279641237139 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10601152754683910907 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10601152754683910907 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:32:53.439+05:30 [Info] Generating Shard Transfer Token (ShardTokenb2:f8:fb:ca:53:2e:1c:c4) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: Region: InstId: 11304674148443950662 RealInstId: 15864358084358521096 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 4083560586502645495 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4708891605960345271 RealInstId: 10032435514095102923 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3464125101225762400 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:32:53.440+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:32:53.440+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:32:53.445+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 2023-10-21T04:32:53.448+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokena2:b2:c5:e2:b2:69:50:c0 2023-10-21T04:32:53.490+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken7f:de:a2:ba:54:17:8e:fe 2023-10-21T04:32:53.502+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenff:34:4:b4:54:ed:73:4a 2023-10-21T04:32:53.502+05:30 [Info] ShardRebalancer::observeRebalance &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:32:53.564+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:34:4:b4:54:ed:73:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10402379389576076017 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10465155089296381829 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2062526591622067910 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.579+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken7f:de:a2:ba:54:17:8e:fe MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8248238408530887194 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4238649781004957942 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2332250308932951681 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.581+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena2:b2:c5:e2:b2:69:50:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2670343739772083055 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16711923683289406198 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3283688418228810121 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.582+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.584+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"1f784fc0142ee312f0d92d98bd188c9a","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:32:53.584+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:32:53.588+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.588+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:32:53.588+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:32:53.588+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:32:53.588+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.42µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:53.618+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.623+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:53.693+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena2:b2:c5:e2:b2:69:50:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2670343739772083055 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16711923683289406198 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3283688418228810121 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.693+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:32:53.693+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:32:53.693+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:32:53.694+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.458µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:53.726+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken7f:de:a2:ba:54:17:8e:fe MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8248238408530887194 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4238649781004957942 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2332250308932951681 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.727+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:34:4:b4:54:ed:73:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10402379389576076017 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10465155089296381829 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2062526591622067910 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.727+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena2:b2:c5:e2:b2:69:50:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2670343739772083055 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16711923683289406198 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3283688418228810121 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.738+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:53.753+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken7f:de:a2:ba:54:17:8e:fe MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8248238408530887194 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4238649781004957942 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2332250308932951681 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.765+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:53.766+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:34:4:b4:54:ed:73:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10402379389576076017 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10465155089296381829 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2062526591622067910 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.773+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:53.773+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:32:53.779+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:32:53.779+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 2 2023-10-21T04:32:53.814+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[7f731cd927c72e28ff24e69497b93571:ShardToken7f:de:a2:ba:54:17:8e:fe 964f96068adef144590fc7ab85b2311c:ShardTokena2:b2:c5:e2:b2:69:50:c0], currActiveTransfers: map[7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:2] 2023-10-21T04:32:53.814+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"1f784fc0142ee312f0d92d98bd188c9a","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:32:53.814+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:32:53.814+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:32:53.814+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:32:53.814+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:32:53.814+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:32:53.815+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.816+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:34:4:b4:54:ed:73:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10402379389576076017 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10465155089296381829 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2062526591622067910 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.816+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena2:b2:c5:e2:b2:69:50:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2670343739772083055 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16711923683289406198 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3283688418228810121 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.817+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken7f:de:a2:ba:54:17:8e:fe MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8248238408530887194 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4238649781004957942 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2332250308932951681 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:53.817+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 15.533µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:54.010+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.010+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:32:54.010+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:32:54.010+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:32:54.010+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:32:54.010+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 28.654µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:54.010+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2925063509317026816:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/2925063509317026816/shards/shard2925063509317026816 16787445029868804812:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/16787445029868804812/shards/shard16787445029868804812] 1f784fc0142ee312f0d92d98bd188c9a ShardTokenb2:f8:fb:ca:53:2e:1c:c4 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc007154540 0xc0071545a0 map[] 0xc0097934a0 0xc00c8a7620 false} 2023-10-21T04:32:54.011+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/2925063509317026816/shards/shard2925063509317026816 2023-10-21T04:32:54.012+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/16787445029868804812/shards/shard16787445029868804812 2023-10-21T04:32:54.013+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/2925063509317026816/shards/shard2925063509317026816/data: super block snapshot head offset :0 tail offset:999424 sbgen :166 2023-10-21T04:32:54.014+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/2925063509317026816/shards/shard2925063509317026816/data/recovery: super block snapshot head offset :0 tail offset:913408 sbgen :166 2023-10-21T04:32:54.017+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/16787445029868804812/shards/shard16787445029868804812/data: super block snapshot head offset :0 tail offset:1105920 sbgen :180 2023-10-21T04:32:54.017+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/16787445029868804812/shards/shard16787445029868804812/data/recovery: super block snapshot head offset :0 tail offset:1040384 sbgen :180 2023-10-21T04:32:54.020+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/2925063509317026816/shards/shard2925063509317026816: metadata saved successfully 2023-10-21T04:32:54.021+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/16787445029868804812/shards/shard16787445029868804812: metadata saved successfully 2023-10-21T04:32:54.021+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:32:54.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:32:54.021+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 193193200 totalBytes: 1931932 numShards: 2 duration: 10.894501ms 2023-10-21T04:32:54.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/2925063509317026816/shards/shard2925063509317026816, newshardId 2925063509317026816, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, shardsize 1931932, duration 10.972127ms 2023-10-21T04:32:54.022+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2925063509317026816, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, err: , elapsed(sec): 0.011342521 2023-10-21T04:32:54.022+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2925063509317026816 bytesWritten: 1931932, totalBytes: 1931932, transferRate: 1.931932e+08 2023-10-21T04:32:54.023+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:32:54.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:32:54.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokenb2_f8_fb_ca_53_2e_1c_c4/16787445029868804812/shards/shard16787445029868804812, newshardId 16787445029868804812, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, shardsize 2165563, duration 11.813469ms 2023-10-21T04:32:54.024+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 16787445029868804812, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, err: , elapsed(sec): 0.013182032 2023-10-21T04:32:54.024+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 1f784fc0142ee312f0d92d98bd188c9a, transferId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 2023-10-21T04:32:54.024+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 1f784fc0142ee312f0d92d98bd188c9a, transferId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4, elapsed(sec): 0.000210843 2023-10-21T04:32:54.024+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2925063509317026816: 16787445029868804812:], shardPaths: map[2925063509317026816:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 16787445029868804812:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.013855374 2023-10-21T04:32:54.024+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 16787445029868804812 bytesWritten: 2165563, totalBytes: 2165563, transferRate: 1.9686936363636363e+08 2023-10-21T04:32:54.025+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [16787445029868804812 2925063509317026816], ttid: ShardTokenb2:f8:fb:ca:53:2e:1c:c4, elapsed(sec): 0.014521628 2023-10-21T04:32:54.036+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:54.055+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.055+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:1], batchSize: 2 2023-10-21T04:32:54.055+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 2023-10-21T04:32:54.056+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2925063509317026816:true 16787445029868804812:true] 2023-10-21T04:32:54.056+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.471µs 2023-10-21T04:32:54.056+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [16787445029868804812 2925063509317026816], isLockedForRecovery: true 2023-10-21T04:32:54.056+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [16787445029868804812 2925063509317026816], errMap: map[2925063509317026816: 16787445029868804812:], elapsed: 12.578µs 2023-10-21T04:32:54.056+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:32:54.056+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:32:54.089+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena2:b2:c5:e2:b2:69:50:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2670343739772083055 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16711923683289406198 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3283688418228810121 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.090+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:32:54.090+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:32:54.090+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:32:54.090+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:32:54.090+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[10170029854712240877:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/10170029854712240877/shards/shard10170029854712240877 17406544375239214575:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/17406544375239214575/shards/shard17406544375239214575] 1f784fc0142ee312f0d92d98bd188c9a ShardTokena2:b2:c5:e2:b2:69:50:c0 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc007154540 0xc0071545a0 map[] 0xc0099306c0 0xc00d0cdce0 false} 2023-10-21T04:32:54.090+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/17406544375239214575/shards/shard17406544375239214575 2023-10-21T04:32:54.091+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/10170029854712240877/shards/shard10170029854712240877 2023-10-21T04:32:54.092+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/17406544375239214575/shards/shard17406544375239214575/data: super block snapshot head offset :0 tail offset:962560 sbgen :160 2023-10-21T04:32:54.093+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/17406544375239214575/shards/shard17406544375239214575/data/recovery: super block snapshot head offset :0 tail offset:888832 sbgen :160 2023-10-21T04:32:54.094+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/10170029854712240877/shards/shard10170029854712240877/data: super block snapshot head offset :0 tail offset:802816 sbgen :148 2023-10-21T04:32:54.094+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/10170029854712240877/shards/shard10170029854712240877/data/recovery: super block snapshot head offset :0 tail offset:720896 sbgen :148 2023-10-21T04:32:54.097+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/10170029854712240877/shards/shard10170029854712240877: metadata saved successfully 2023-10-21T04:32:54.098+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:32:54.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:32:54.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/10170029854712240877/shards/shard10170029854712240877, newshardId 10170029854712240877, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, shardsize 1542816, duration 6.943031ms 2023-10-21T04:32:54.098+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10170029854712240877, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877, err: , elapsed(sec): 0.00822945 2023-10-21T04:32:54.098+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10170029854712240877 bytesWritten: 1542816, totalBytes: 1542816, transferRate: 2.57136e+08 2023-10-21T04:32:54.100+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/17406544375239214575/shards/shard17406544375239214575: metadata saved successfully 2023-10-21T04:32:54.100+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:32:54.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 from metadata completed, tenant :bucket_%2_b9538b29d7e025c819e734b646db039c 2023-10-21T04:32:54.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/1f784fc0142ee312f0d92d98bd188c9a_ShardTokena2_b2_c5_e2_b2_69_50_c0/17406544375239214575/shards/shard17406544375239214575, newshardId 17406544375239214575, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, shardsize 1870658, duration 10.544136ms 2023-10-21T04:32:54.101+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17406544375239214575, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575, err: , elapsed(sec): 0.010915876 2023-10-21T04:32:54.101+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 1f784fc0142ee312f0d92d98bd188c9a, transferId: ShardTokena2:b2:c5:e2:b2:69:50:c0 2023-10-21T04:32:54.101+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 1f784fc0142ee312f0d92d98bd188c9a, transferId: ShardTokena2:b2:c5:e2:b2:69:50:c0, elapsed(sec): 0.000327016 2023-10-21T04:32:54.101+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[10170029854712240877: 17406544375239214575:], shardPaths: map[10170029854712240877:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 17406544375239214575:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.011500242 2023-10-21T04:32:54.101+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17406544375239214575 bytesWritten: 1870658, totalBytes: 1870658, transferRate: 1.870658e+08 2023-10-21T04:32:54.101+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17406544375239214575 10170029854712240877], ttid: ShardTokena2:b2:c5:e2:b2:69:50:c0, elapsed(sec): 0.011839918 2023-10-21T04:32:54.114+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15864358084358521096,realInstId: 0, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:32:54.120+05:30 [Info] Indexer::handleRecoverIndex InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:32:54.121+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 972.342µs 2023-10-21T04:32:54.121+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.121+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15864358084358521096, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:32:54.121+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 65.287608ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:54.122+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 2 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.122+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 30.466µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:32:54.122+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:32:54.122+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 2.117164ms 2023-10-21T04:32:54.127+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner initialized 2023-10-21T04:32:54.127+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.130+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:54.130+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner initialized 2023-10-21T04:32:54.130+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.130+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:32:54.136+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCleaner initialized for recovery 2023-10-21T04:32:54.136+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery 2023-10-21T04:32:54.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true] 2023-10-21T04:32:54.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[913408] Data log Head [0] Tail [999424] 2023-10-21T04:32:54.139+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [913408] 2023-10-21T04:32:54.141+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCleaner initialized for recovery 2023-10-21T04:32:54.148+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.143+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.149+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:32:54.150+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.150+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:32:54.152+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery 2023-10-21T04:32:54.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.153+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:32:54.187+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true] 2023-10-21T04:32:54.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1040384] Data log Head [0] Tail [1105920] 2023-10-21T04:32:54.188+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1040384] 2023-10-21T04:32:54.188+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:32:54.188+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.188+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:32:54.188+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.188+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:32:54.188+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.188+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:32:54.188+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.188+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.189+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.189+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.189+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.189+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.189+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.189+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.190+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:54.190+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:32:54.190+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.190+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:54.190+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:32:54.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:32:54.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:32:54.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:32:54.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:32:54.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:32:54.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:32:54.191+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.191+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:32:54.192+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.192+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:54.192+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.192+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:54.193+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:54.193+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:54.193+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:54.193+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:54.193+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:54.193+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:54.194+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.194+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:32:54.194+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : replayOffset: 1097728 (RP:1089536) (CP:1097728) 2023-10-21T04:32:54.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1097728] took [6.8986ms] 2023-10-21T04:32:54.194+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1055176, resident 1, memSz 562, memSzIndex 0, numRecords 8 2023-10-21T04:32:54.194+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.194+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057276, resident 1, memSz 12809, memSzIndex 0, numRecords 127 2023-10-21T04:32:54.194+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057276, resident 1, memSz 13698, memSzIndex 0, numRecords 136 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057426, resident 1, memSz 12641, memSzIndex 0, numRecords 125 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057276, resident 1, memSz 13717, memSzIndex 0, numRecords 136 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1056344, resident 1, memSz 75000, memSzIndex 297, numRecords 1000 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057426, resident 1, memSz 10752, memSzIndex 0, numRecords 106 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057426, resident 1, memSz 12643, memSzIndex 0, numRecords 125 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057426, resident 1, memSz 12433, memSzIndex 0, numRecords 123 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1057276, resident 1, memSz 12321, memSzIndex 0, numRecords 122 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.195+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44748749 controller mem 2686 2023-10-21T04:32:54.195+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1105920] replayOffset [1097728] 2023-10-21T04:32:54.212+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena2:b2:c5:e2:b2:69:50:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2670343739772083055 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16711923683289406198 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3283688418228810121 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.212+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:32:54.212+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardTokena2:b2:c5:e2:b2:69:50:c0 2023-10-21T04:32:54.212+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[10170029854712240877:true 17406544375239214575:true] 2023-10-21T04:32:54.212+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.503µs 2023-10-21T04:32:54.212+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [17406544375239214575 10170029854712240877], isLockedForRecovery: true 2023-10-21T04:32:54.212+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [17406544375239214575 10170029854712240877], errMap: map[10170029854712240877: 17406544375239214575:], elapsed: 17.63µs 2023-10-21T04:32:54.213+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.213+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:32:54.213+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : replayOffset: 991232 (RP:983040) (CP:991232) 2023-10-21T04:32:54.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [991232] took [78.316262ms] 2023-10-21T04:32:54.218+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:32:54.216+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 13268712595133993155, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:32:54.216+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:54.219+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 13268712595133993155, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:32:54.220+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.220+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.221+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.224+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1056160, resident 1, memSz 66636, memSzIndex 1023, numRecords 1000 2023-10-21T04:32:54.225+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.225+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053288, resident 1, memSz 10500, memSzIndex 77, numRecords 136 2023-10-21T04:32:54.225+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.226+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053364, resident 1, memSz 9269, memSzIndex 92, numRecords 125 2023-10-21T04:32:54.226+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [37.841534ms] 2023-10-21T04:32:54.232+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.232+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.232+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.232+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.232+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.233+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.233+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.233+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.229+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:32:54.237+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.237+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.238+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.237+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.239+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.241+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.238+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053288, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:32:54.246+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.258+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053288, resident 1, memSz 9501, memSzIndex 76, numRecords 127 2023-10-21T04:32:54.246+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.259+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.259+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.258+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.261+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.260+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.262+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.263+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.263+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true]. Built [18] plasmas, took [76.060365ms] 2023-10-21T04:32:54.264+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:54.264+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 136 metaSz: 2512 rp [1] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:32:54.264+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 71404 recovery UsedSpace 91224 2023-10-21T04:32:54.262+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 45803113 controller mem 5204 2023-10-21T04:32:54.265+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [999424] replayOffset [991232] 2023-10-21T04:32:54.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [126.65007ms] 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true]. Built [17] plasmas, took [127.422607ms] 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 10 numRPs: 3 rp [0] sn: 12 count: 136 metaSz: 2512 rp [1] sn: 360005 count: 136 metaSz: 2512 rp [2] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:32:54.266+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 70377 recovery UsedSpace 87560 2023-10-21T04:32:54.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:32:54.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.271+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:32:54.271+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:32:54.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex started 2023-10-21T04:32:54.271+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.271+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:32:54.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.284+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:32:54.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex started 2023-10-21T04:32:54.284+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:32:54.285+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.314+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Warmup took 192.691454ms 2023-10-21T04:32:54.316+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 2 WriterThreads 1 2023-10-21T04:32:54.316+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Stopped Billing WUs 2023-10-21T04:32:54.316+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index Index: 15864358084358521096 Partition: 2 2023-10-21T04:32:54.316+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.317+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 3 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.318+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:32:54.318+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 125 metaSz: 2512 rp [1] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:32:54.318+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 74175 recovery UsedSpace 91224 2023-10-21T04:32:54.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.322+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 258 2023-10-21T04:32:54.323+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 125 metaSz: 2512 rp [1] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:32:54.324+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 71919 recovery UsedSpace 95584 2023-10-21T04:32:54.332+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2670343739772083055,realInstId: 0, indexDefn DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:32:54.333+05:30 [Info] Indexer::handleRecoverIndex InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:32:54.334+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 474.091µs 2023-10-21T04:32:54.334+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2670343739772083055 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:54.334+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2670343739772083055 PartitionId: 0 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:54.334+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': added 2023-10-21T04:32:54.335+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2670343739772083055, indexDefn DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:32:54.335+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 118.940674ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:54.337+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner initialized 2023-10-21T04:32:54.337+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.337+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner initialized 2023-10-21T04:32:54.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:54.337+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:54.337+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.575239ms 2023-10-21T04:32:54.341+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCleaner initialized for recovery 2023-10-21T04:32:54.341+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery 2023-10-21T04:32:54.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:54.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true] 2023-10-21T04:32:54.341+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[888832] Data log Head [0] Tail [962560] 2023-10-21T04:32:54.341+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [888832] 2023-10-21T04:32:54.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:32:54.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.342+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:32:54.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex started 2023-10-21T04:32:54.342+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.342+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.343+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.343+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:32:54.343+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:32:54.343+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.344+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:32:54.344+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.344+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:32:54.344+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.344+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:32:54.344+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.344+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:32:54.345+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.345+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:32:54.345+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.345+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:32:54.345+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.346+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:32:54.346+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.346+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:32:54.353+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCleaner initialized for recovery 2023-10-21T04:32:54.353+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:32:54.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery 2023-10-21T04:32:54.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:54.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true] 2023-10-21T04:32:54.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[720896] Data log Head [0] Tail [802816] 2023-10-21T04:32:54.353+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [720896] 2023-10-21T04:32:54.355+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.355+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:32:54.355+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.355+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:32:54.356+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.356+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:32:54.356+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.356+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:32:54.356+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.356+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:32:54.356+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.356+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:32:54.356+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.356+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:32:54.356+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.357+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.357+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:54.357+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.357+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:54.357+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.357+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:54.357+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:54.358+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:54.358+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:54.358+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:54.358+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:54.358+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.358+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:32:54.359+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : replayOffset: 954368 (RP:946176) (CP:954368) 2023-10-21T04:32:54.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [954368] took [18.06866ms] 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12901, memSzIndex 0, numRecords 128 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12465, memSzIndex 0, numRecords 123 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.359+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1055176, resident 0.2, memSz 438, memSzIndex 0, numRecords 10 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13215, memSzIndex 0, numRecords 131 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1055940, resident 1, memSz 70469, memSzIndex 352, numRecords 1000 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12819, memSzIndex 0, numRecords 127 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11323, memSzIndex 0, numRecords 112 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13881, memSzIndex 0, numRecords 138 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12446, memSzIndex 0, numRecords 123 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11919, memSzIndex 0, numRecords 118 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.360+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 35274654 controller mem 1472 2023-10-21T04:32:54.360+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [962560] replayOffset [954368] 2023-10-21T04:32:54.385+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:32:54.387+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.387+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:54.387+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.388+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:54.388+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.388+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:54.388+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.388+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:54.388+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.388+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:54.389+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.422+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:54.423+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.423+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:54.423+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.423+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:54.423+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:54.423+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:32:54.391+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken7f:de:a2:ba:54:17:8e:fe MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8248238408530887194 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4238649781004957942 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2332250308932951681 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.424+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:1 8806f5706747af4e635b8da5fcd8aeee:1 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:32:54.425+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:34:4:b4:54:ed:73:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10402379389576076017 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10465155089296381829 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2062526591622067910 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.425+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:0 8806f5706747af4e635b8da5fcd8aeee:2 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:32:54.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [55.362594ms] 2023-10-21T04:32:54.425+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.425+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.425+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.425+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.425+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.425+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.425+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true]. Built [18] plasmas, took [85.099419ms] 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1055940 memSz 896 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 22 count: 1000 metaSz: 2517 rp [1] sn: 23 count: 1000 metaSz: 2525 2023-10-21T04:32:54.426+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 110879 recovery UsedSpace 22566 2023-10-21T04:32:54.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:32:54.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.427+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:32:54.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex started 2023-10-21T04:32:54.427+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:32:54.428+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.428+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.439+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : replayOffset: 794624 (RP:659456) (CP:794624) 2023-10-21T04:32:54.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [794624] took [86.275856ms] 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9834, memSzIndex 0, numRecords 123 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.440+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9511, memSzIndex 76, numRecords 127 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053288, resident 1, memSz 8667, memSzIndex 0, numRecords 112 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9853, memSzIndex 0, numRecords 123 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9287, memSzIndex 0, numRecords 118 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1056196, resident 1, memSz 67249, memSzIndex 994, numRecords 1000 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9997, memSzIndex 77, numRecords 131 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9669, memSzIndex 76, numRecords 128 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10687, memSzIndex 75, numRecords 138 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:32:54.443+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45802290 controller mem 4932 2023-10-21T04:32:54.443+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [802816] replayOffset [794624] 2023-10-21T04:32:54.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [90.192446ms] 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.444+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.445+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.445+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.445+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.445+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.445+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.446+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.446+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:32:54.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true]. Built [17] plasmas, took [92.560612ms] 2023-10-21T04:32:54.446+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1056196 memSz 2244 2023-10-21T04:32:54.446+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 22 count: 1000 metaSz: 2517 rp [1] sn: 23 count: 1000 metaSz: 2525 2023-10-21T04:32:54.447+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 88174 recovery UsedSpace 25089 2023-10-21T04:32:54.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:32:54.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:54.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:54.447+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper started 2023-10-21T04:32:54.447+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:32:54.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex started 2023-10-21T04:32:54.447+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.447+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.451+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Warmup took 133.630997ms 2023-10-21T04:32:54.451+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 3 WriterThreads 1 2023-10-21T04:32:54.451+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Stopped Billing WUs 2023-10-21T04:32:54.451+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index Index: 15864358084358521096 Partition: 3 2023-10-21T04:32:54.451+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.451+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 4 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.452+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 256 2023-10-21T04:32:54.452+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 10 numRPs: 3 rp [0] sn: 12 count: 125 metaSz: 2512 rp [1] sn: 360005 count: 125 metaSz: 2512 rp [2] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:32:54.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.452+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:32:54.452+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 10 numRPs: 3 rp [0] sn: 12 count: 125 metaSz: 2512 rp [1] sn: 360005 count: 125 metaSz: 2512 rp [2] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:32:54.452+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 71542 recovery UsedSpace 87366 2023-10-21T04:32:54.452+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 74095 recovery UsedSpace 83010 2023-10-21T04:32:54.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:32:54.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.458+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:32:54.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex started 2023-10-21T04:32:54.458+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.458+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:32:54.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.461+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:32:54.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex started 2023-10-21T04:32:54.462+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:32:54.462+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.462+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.483+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Warmup took 31.690871ms 2023-10-21T04:32:54.484+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 4 WriterThreads 1 2023-10-21T04:32:54.484+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Stopped Billing WUs 2023-10-21T04:32:54.484+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index Index: 15864358084358521096 Partition: 4 2023-10-21T04:32:54.484+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.485+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 5 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:32:54.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 123 metaSz: 2512 rp [1] sn: 360006 count: 123 metaSz: 2513 2023-10-21T04:32:54.485+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 73935 recovery UsedSpace 91224 2023-10-21T04:32:54.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.486+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:32:54.486+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 123 metaSz: 2512 rp [1] sn: 360006 count: 123 metaSz: 2513 2023-10-21T04:32:54.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 70595 recovery UsedSpace 95320 2023-10-21T04:32:54.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:32:54.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.494+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:32:54.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex started 2023-10-21T04:32:54.494+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.494+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.499+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:54.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:32:54.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:54.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:54.504+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:32:54.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex started 2023-10-21T04:32:54.505+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2670343739772083055 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:32:54.505+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.505+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:32:54.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.512+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:32:54.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex started 2023-10-21T04:32:54.513+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:32:54.514+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.515+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.528+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:54.528+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:54.537+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken7f:de:a2:ba:54:17:8e:fe MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8248238408530887194 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4238649781004957942 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2332250308932951681 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.537+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:0 8806f5706747af4e635b8da5fcd8aeee:1 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:32:54.556+05:30 [Info] switched currmeta from 182 -> 184 force true 2023-10-21T04:32:54.560+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2670343739772083055 PartitionId 0 Warmup took 225.293617ms 2023-10-21T04:32:54.560+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2670343739772083055 partitionId 0 WriterThreads 4 2023-10-21T04:32:54.560+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2670343739772083055 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index Index: 2670343739772083055 Partition: 0 2023-10-21T04:32:54.560+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2670343739772083055, shardIdMap: map[0:[17406544375239214575 10170029854712240877]] err: 2023-10-21T04:32:54.560+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2670343739772083055, partnShardIdMap: map[0:[17406544375239214575 10170029854712240877]], err: 2023-10-21T04:32:54.560+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:54.561+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2670343739772083055 2023-10-21T04:32:54.561+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:54.561+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:32:54.561+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:54.561+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:54.561+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:54.561+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2670343739772083055 Partitions [0] 2023-10-21T04:32:54.563+05:30 [Info] StorageMgr::openSnapshot IndexInst:2670343739772083055 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:32:54.570+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Warmup took 85.504877ms 2023-10-21T04:32:54.571+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 5 WriterThreads 1 2023-10-21T04:32:54.571+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Stopped Billing WUs 2023-10-21T04:32:54.572+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index Index: 15864358084358521096 Partition: 5 2023-10-21T04:32:54.572+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.572+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 6 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.573+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.024379ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:54.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.575+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:32:54.575+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 10 numRPs: 3 rp [0] sn: 12 count: 122 metaSz: 2512 rp [1] sn: 360005 count: 122 metaSz: 2512 rp [2] sn: 360006 count: 122 metaSz: 2513 2023-10-21T04:32:54.575+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 73770 recovery UsedSpace 87106 2023-10-21T04:32:54.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.575+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:54.575+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 122 metaSz: 2512 rp [1] sn: 360006 count: 122 metaSz: 2513 2023-10-21T04:32:54.576+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 77663 recovery UsedSpace 91224 2023-10-21T04:32:54.575+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 489.497µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:54.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:32:54.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.580+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:32:54.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex started 2023-10-21T04:32:54.580+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.580+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:32:54.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.581+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:32:54.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex started 2023-10-21T04:32:54.581+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:32:54.581+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.581+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.602+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenff:34:4:b4:54:ed:73:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10402379389576076017 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10465155089296381829 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2062526591622067910 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:32:54.602+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 7f731cd927c72e28ff24e69497b93571:0 8806f5706747af4e635b8da5fcd8aeee:0 964f96068adef144590fc7ab85b2311c:0], batchSize: 2 2023-10-21T04:32:54.608+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Warmup took 34.32587ms 2023-10-21T04:32:54.608+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 6 WriterThreads 1 2023-10-21T04:32:54.609+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Stopped Billing WUs 2023-10-21T04:32:54.609+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index Index: 15864358084358521096 Partition: 6 2023-10-21T04:32:54.609+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.609+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 7 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.609+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:32:54.609+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 10 numRPs: 3 rp [0] sn: 12 count: 136 metaSz: 2512 rp [1] sn: 360005 count: 136 metaSz: 2512 rp [2] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:32:54.609+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 77215 recovery UsedSpace 87554 2023-10-21T04:32:54.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.610+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:54.610+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 136 metaSz: 2512 rp [1] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:32:54.610+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 78978 recovery UsedSpace 91224 2023-10-21T04:32:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:32:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.613+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:32:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex started 2023-10-21T04:32:54.614+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.614+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:32:54.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.626+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:32:54.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex started 2023-10-21T04:32:54.626+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:32:54.626+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.626+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.666+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Warmup took 56.840177ms 2023-10-21T04:32:54.666+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 7 WriterThreads 1 2023-10-21T04:32:54.667+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Stopped Billing WUs 2023-10-21T04:32:54.667+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index Index: 15864358084358521096 Partition: 7 2023-10-21T04:32:54.667+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.667+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 8 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.667+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:32:54.667+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 10 numRPs: 3 rp [0] sn: 12 count: 106 metaSz: 2511 rp [1] sn: 360005 count: 106 metaSz: 2511 rp [2] sn: 360006 count: 106 metaSz: 2512 2023-10-21T04:32:54.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.667+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:32:54.668+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 106 metaSz: 2511 rp [1] sn: 360006 count: 106 metaSz: 2512 2023-10-21T04:32:54.668+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 72285 recovery UsedSpace 87106 2023-10-21T04:32:54.668+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 76026 recovery UsedSpace 91224 2023-10-21T04:32:54.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:32:54.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.678+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:32:54.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex started 2023-10-21T04:32:54.678+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.678+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:32:54.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.680+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:32:54.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex started 2023-10-21T04:32:54.680+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:32:54.680+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.680+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.701+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Warmup took 33.957597ms 2023-10-21T04:32:54.701+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 8 WriterThreads 1 2023-10-21T04:32:54.701+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Stopped Billing WUs 2023-10-21T04:32:54.701+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index Index: 15864358084358521096 Partition: 8 2023-10-21T04:32:54.702+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:54.702+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 1 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:54.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:54.702+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:32:54.702+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 10 numRPs: 3 rp [0] sn: 13 count: 127 metaSz: 2512 rp [1] sn: 360005 count: 127 metaSz: 2512 rp [2] sn: 360006 count: 127 metaSz: 2513 2023-10-21T04:32:54.702+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 79842 recovery UsedSpace 87460 2023-10-21T04:32:54.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:54.703+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:54.703+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360005 count: 127 metaSz: 2512 rp [1] sn: 360006 count: 127 metaSz: 2513 2023-10-21T04:32:54.703+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 82411 recovery UsedSpace 91224 2023-10-21T04:32:54.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:32:54.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:54.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:54.719+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:32:54.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex started 2023-10-21T04:32:54.719+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.720+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:32:54.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:54.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:54.723+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:32:54.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex started 2023-10-21T04:32:54.723+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:32:54.724+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:54.725+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:54.746+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Warmup took 44.454758ms 2023-10-21T04:32:54.747+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 1 WriterThreads 1 2023-10-21T04:32:54.747+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Stopped Billing WUs 2023-10-21T04:32:54.747+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index Index: 15864358084358521096 Partition: 1 2023-10-21T04:32:54.747+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15864358084358521096, shardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:32:54.747+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15864358084358521096, partnShardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:32:54.747+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:54.747+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15864358084358521096 2023-10-21T04:32:54.747+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:32:54.747+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:32:54.747+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:54.748+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:32:54.748+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:54.748+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15864358084358521096 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:32:54.750+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:2 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:32:54.753+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:3 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:32:54.755+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:4 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:32:54.758+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.632689ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:54.758+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 102.035µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:32:54.762+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:5 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:32:54.763+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:6 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:32:54.764+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:7 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:32:54.779+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:8 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:32:54.793+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:1 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:32:54.823+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:54.823+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:54.846+05:30 [Info] switched currmeta from 184 -> 188 force true 2023-10-21T04:32:54.948+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:54.948+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:54.985+05:30 [Info] switched currmeta from 188 -> 189 force true 2023-10-21T04:32:54.995+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:54.995+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.005+05:30 [Info] switched currmeta from 189 -> 189 force true 2023-10-21T04:32:55.221+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[15864358084358521096:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:32:55.222+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 4790214984019693894, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:32:55.222+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 4790214984019693894, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:32:55.239+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4083560586502645495,realInstId: 0, indexDefn DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:32:55.239+05:30 [Info] Indexer::handleRecoverIndex InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:32:55.239+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 221.372µs 2023-10-21T04:32:55.240+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 261.604µs 2023-10-21T04:32:55.240+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4083560586502645495 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:55.240+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4083560586502645495 PartitionId: 0 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:55.241+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4083560586502645495, indexDefn DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:32:55.241+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 19.575924ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:55.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:55.242+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1056344 memSz 666 2023-10-21T04:32:55.242+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 16 count: 1000 metaSz: 2508 rp [1] sn: 17 count: 1000 metaSz: 2514 2023-10-21T04:32:55.242+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 144883 recovery UsedSpace 22110 2023-10-21T04:32:55.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:55.244+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1056160 memSz 2518 2023-10-21T04:32:55.244+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 16 count: 1000 metaSz: 2508 rp [1] sn: 17 count: 1000 metaSz: 2514 2023-10-21T04:32:55.244+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 145057 recovery UsedSpace 25603 2023-10-21T04:32:55.247+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:55.261+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.261+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:32:55.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:55.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:55.263+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:32:55.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex started 2023-10-21T04:32:55.263+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.263+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:32:55.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:55.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:55.272+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:32:55.267+05:30 [Info] switched currmeta from 189 -> 190 force true 2023-10-21T04:32:55.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex started 2023-10-21T04:32:55.274+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.274+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.274+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4083560586502645495 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:32:55.318+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4083560586502645495 PartitionId 0 Warmup took 78.105039ms 2023-10-21T04:32:55.320+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4083560586502645495 partitionId 0 WriterThreads 4 2023-10-21T04:32:55.320+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4083560586502645495 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index Index: 4083560586502645495 Partition: 0 2023-10-21T04:32:55.322+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4083560586502645495, shardIdMap: map[0:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:32:55.322+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4083560586502645495, partnShardIdMap: map[0:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:32:55.322+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:55.323+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4083560586502645495 2023-10-21T04:32:55.324+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:55.324+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:32:55.324+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:55.324+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:55.324+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:55.324+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4083560586502645495 Partitions [0] 2023-10-21T04:32:55.325+05:30 [Info] StorageMgr::openSnapshot IndexInst:4083560586502645495 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:32:55.332+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.003019ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:55.333+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 248.941µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:32:55.349+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.349+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.352+05:30 [Info] switched currmeta from 190 -> 191 force true 2023-10-21T04:32:55.394+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.394+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.399+05:30 [Info] switched currmeta from 191 -> 192 force true 2023-10-21T04:32:55.502+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2670343739772083055:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:32:55.503+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:32:55.503+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 8910459001587546151, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:32:55.532+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8965923595543532942,realInstId: 0, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:32:55.533+05:30 [Info] Indexer::handleRecoverIndex InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:32:55.533+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 334.57µs 2023-10-21T04:32:55.533+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 383.234µs 2023-10-21T04:32:55.533+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.533+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 5 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.534+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 238 2023-10-21T04:32:55.534+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 14 count: 138 metaSz: 2514 rp [1] sn: 15 count: 138 metaSz: 2518 rp [2] sn: 16 count: 138 metaSz: 2520 rp [3] sn: 360005 count: 138 metaSz: 2520 2023-10-21T04:32:55.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.534+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:55.534+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 65293 recovery UsedSpace 62764 2023-10-21T04:32:55.534+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8965923595543532942, indexDefn DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:32:55.534+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 31.079304ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:55.534+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 16 count: 138 metaSz: 2520 rp [1] sn: 360005 count: 138 metaSz: 2520 2023-10-21T04:32:55.535+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 65326 recovery UsedSpace 78750 2023-10-21T04:32:55.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:32:55.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.545+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:32:55.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex started 2023-10-21T04:32:55.545+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.545+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:32:55.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.547+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:32:55.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex started 2023-10-21T04:32:55.547+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:32:55.549+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.549+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.559+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:55.568+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Warmup took 34.414717ms 2023-10-21T04:32:55.568+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 5 WriterThreads 1 2023-10-21T04:32:55.568+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index Index: 8965923595543532942 Partition: 5 2023-10-21T04:32:55.568+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.569+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 6 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.569+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:32:55.569+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 14 count: 127 metaSz: 2514 rp [1] sn: 15 count: 127 metaSz: 2518 rp [2] sn: 16 count: 127 metaSz: 2520 rp [3] sn: 360005 count: 127 metaSz: 2520 2023-10-21T04:32:55.569+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 64250 recovery UsedSpace 62676 2023-10-21T04:32:55.570+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.570+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:55.570+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 16 count: 127 metaSz: 2520 rp [1] sn: 360005 count: 127 metaSz: 2520 2023-10-21T04:32:55.570+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 64465 recovery UsedSpace 74654 2023-10-21T04:32:55.574+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.574+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:32:55.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.577+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:32:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex started 2023-10-21T04:32:55.577+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.576+05:30 [Info] switched currmeta from 192 -> 193 force true 2023-10-21T04:32:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:32:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.577+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:32:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex started 2023-10-21T04:32:55.577+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:32:55.580+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.580+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.580+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.598+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Warmup took 29.585735ms 2023-10-21T04:32:55.599+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 6 WriterThreads 1 2023-10-21T04:32:55.599+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index Index: 8965923595543532942 Partition: 6 2023-10-21T04:32:55.599+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.599+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 7 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.599+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:32:55.600+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 14 count: 131 metaSz: 2514 rp [1] sn: 15 count: 131 metaSz: 2518 rp [2] sn: 16 count: 131 metaSz: 2520 rp [3] sn: 360005 count: 131 metaSz: 2520 2023-10-21T04:32:55.600+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 67041 recovery UsedSpace 62776 2023-10-21T04:32:55.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.601+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:55.601+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 16 count: 131 metaSz: 2520 rp [1] sn: 360005 count: 131 metaSz: 2520 2023-10-21T04:32:55.601+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 67940 recovery UsedSpace 74654 2023-10-21T04:32:55.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:32:55.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.606+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:32:55.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex started 2023-10-21T04:32:55.606+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.606+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:32:55.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.608+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:32:55.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex started 2023-10-21T04:32:55.608+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.608+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:32:55.608+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.634+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Warmup took 35.153045ms 2023-10-21T04:32:55.635+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 7 WriterThreads 1 2023-10-21T04:32:55.635+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index Index: 8965923595543532942 Partition: 7 2023-10-21T04:32:55.635+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.635+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 8 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.635+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:32:55.635+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:55.635+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 14 count: 112 metaSz: 2514 rp [1] sn: 15 count: 112 metaSz: 2518 rp [2] sn: 16 count: 112 metaSz: 2520 rp [3] sn: 360005 count: 112 metaSz: 2520 2023-10-21T04:32:55.635+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 16 count: 112 metaSz: 2520 rp [1] sn: 360005 count: 112 metaSz: 2520 2023-10-21T04:32:55.636+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 66610 recovery UsedSpace 62322 2023-10-21T04:32:55.636+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 66231 recovery UsedSpace 74654 2023-10-21T04:32:55.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:32:55.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.643+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:32:55.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex started 2023-10-21T04:32:55.644+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:32:55.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.644+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:32:55.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex started 2023-10-21T04:32:55.644+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:32:55.644+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.644+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.644+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.660+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.660+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.663+05:30 [Info] switched currmeta from 193 -> 194 force true 2023-10-21T04:32:55.671+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Warmup took 36.00171ms 2023-10-21T04:32:55.671+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 8 WriterThreads 1 2023-10-21T04:32:55.671+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index Index: 8965923595543532942 Partition: 8 2023-10-21T04:32:55.671+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.672+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 1 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.672+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:55.672+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 15 count: 128 metaSz: 2520 rp [1] sn: 360005 count: 128 metaSz: 2520 2023-10-21T04:32:55.672+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 67490 recovery UsedSpace 74654 2023-10-21T04:32:55.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.673+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:32:55.673+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 13 count: 128 metaSz: 2514 rp [1] sn: 14 count: 128 metaSz: 2518 rp [2] sn: 15 count: 128 metaSz: 2520 rp [3] sn: 360005 count: 128 metaSz: 2520 2023-10-21T04:32:55.673+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 71796 recovery UsedSpace 62770 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.699+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex started 2023-10-21T04:32:55.699+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.699+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.699+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:32:55.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex started 2023-10-21T04:32:55.700+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:32:55.701+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.701+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.712+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.712+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.715+05:30 [Info] switched currmeta from 194 -> 195 force true 2023-10-21T04:32:55.733+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Warmup took 61.575145ms 2023-10-21T04:32:55.733+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 1 WriterThreads 1 2023-10-21T04:32:55.733+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index Index: 8965923595543532942 Partition: 1 2023-10-21T04:32:55.733+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.734+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 2 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.734+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:32:55.734+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 14 count: 123 metaSz: 2514 rp [1] sn: 15 count: 123 metaSz: 2517 rp [2] sn: 16 count: 123 metaSz: 2520 rp [3] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:32:55.734+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 72742 recovery UsedSpace 62322 2023-10-21T04:32:55.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.735+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:32:55.735+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 16 count: 123 metaSz: 2520 rp [1] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:32:55.735+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 70404 recovery UsedSpace 74654 2023-10-21T04:32:55.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:32:55.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.744+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:32:55.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex started 2023-10-21T04:32:55.744+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.744+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:32:55.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.744+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:32:55.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex started 2023-10-21T04:32:55.745+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:32:55.745+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.745+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.774+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.774+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.776+05:30 [Info] switched currmeta from 195 -> 196 force true 2023-10-21T04:32:55.782+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Warmup took 47.944674ms 2023-10-21T04:32:55.782+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 2 WriterThreads 1 2023-10-21T04:32:55.782+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index Index: 8965923595543532942 Partition: 2 2023-10-21T04:32:55.782+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.782+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 3 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.782+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:32:55.782+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 13 count: 123 metaSz: 2514 rp [1] sn: 14 count: 123 metaSz: 2517 rp [2] sn: 15 count: 123 metaSz: 2520 rp [3] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:32:55.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 75504 recovery UsedSpace 62322 2023-10-21T04:32:55.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.783+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:32:55.783+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 15 count: 123 metaSz: 2520 rp [1] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:32:55.783+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 70433 recovery UsedSpace 74654 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.805+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex started 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.805+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:32:55.805+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex started 2023-10-21T04:32:55.805+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.805+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:32:55.805+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.805+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.817+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.819+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.825+05:30 [Info] switched currmeta from 196 -> 197 force true 2023-10-21T04:32:55.856+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Warmup took 74.189879ms 2023-10-21T04:32:55.857+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 3 WriterThreads 1 2023-10-21T04:32:55.857+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index Index: 8965923595543532942 Partition: 3 2023-10-21T04:32:55.857+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:55.857+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8965923595543532942 PartitionId: 4 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:55.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:55.857+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:32:55.857+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: RP version: 7 numRPs: 4 rp [0] sn: 14 count: 118 metaSz: 2514 rp [1] sn: 15 count: 118 metaSz: 2517 rp [2] sn: 16 count: 118 metaSz: 2520 rp [3] sn: 360005 count: 118 metaSz: 2520 2023-10-21T04:32:55.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:55.858+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:32:55.858+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 75088 recovery UsedSpace 62322 2023-10-21T04:32:55.858+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 16 count: 118 metaSz: 2520 rp [1] sn: 360005 count: 118 metaSz: 2520 2023-10-21T04:32:55.858+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 73475 recovery UsedSpace 74654 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:55.872+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:32:55.872+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex started 2023-10-21T04:32:55.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex started 2023-10-21T04:32:55.872+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.872+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:32:55.872+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:55.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:55.889+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Warmup took 31.98735ms 2023-10-21T04:32:55.889+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 4 WriterThreads 1 2023-10-21T04:32:55.889+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index Index: 8965923595543532942 Partition: 4 2023-10-21T04:32:55.889+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8965923595543532942, shardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] err: 2023-10-21T04:32:55.890+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8965923595543532942, partnShardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]], err: 2023-10-21T04:32:55.890+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:55.890+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8965923595543532942 2023-10-21T04:32:55.890+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:55.890+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:32:55.890+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:55.890+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:55.890+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:55.891+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8965923595543532942 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:32:55.892+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:32:55.893+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:7 Attempting to open snapshot (SnapshotInfo: count:131 committed:false) 2023-10-21T04:32:55.894+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:8 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:32:55.898+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.049311ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:55.899+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 124.111µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:32:55.911+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:1 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:32:55.917+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:2 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:32:55.918+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:3 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:32:55.919+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:4 Attempting to open snapshot (SnapshotInfo: count:118 committed:false) 2023-10-21T04:32:55.921+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:5 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:32:55.930+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:55.930+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:55.933+05:30 [Info] switched currmeta from 197 -> 198 force true 2023-10-21T04:32:56.199+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:56.199+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:56.202+05:30 [Info] switched currmeta from 198 -> 199 force true 2023-10-21T04:32:56.250+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4083560586502645495:INDEX_STATE_RECOVERED 15864358084358521096:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:32:56.251+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:32:56.251+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:32:56.272+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10032435514095102923,realInstId: 0, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:32:56.273+05:30 [Info] Indexer::handleRecoverIndex InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:32:56.273+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 341.593µs 2023-10-21T04:32:56.273+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 388.776µs 2023-10-21T04:32:56.273+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.273+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 7 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.274+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.274+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10032435514095102923, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:32:56.274+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 22.848983ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:56.274+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:32:56.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.275+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.275+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:32:56.283+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:56.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:56.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.284+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:56.284+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:32:56.284+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.284+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:56.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.286+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:56.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:32:56.286+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:32:56.286+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:32:56.286+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:56.286+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:32:56.286+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:32:56.286+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:56.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.286+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.287+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.293+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:56.293+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:56.295+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:32:56.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:32:56.297+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.298+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.298+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.298+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:32:56.299+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:56.299+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:32:56.299+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:32:56.299+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:56.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:32:56.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.300+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.301+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.305+05:30 [Info] switched currmeta from 199 -> 200 force true 2023-10-21T04:32:56.307+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:32:56.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:32:56.308+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:32:56.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:32:56.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:32:56.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:32:56.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:32:56.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.327+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.327+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:56.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.338+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:56.338+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:32:56.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:32:56.339+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.339+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.344+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.344+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:56.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.353+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:56.353+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:32:56.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:32:56.353+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Warmup took 79.828477ms 2023-10-21T04:32:56.353+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.353+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.353+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 7 WriterThreads 1 2023-10-21T04:32:56.354+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Stopped Billing WUs 2023-10-21T04:32:56.354+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index Index: 10032435514095102923 Partition: 7 2023-10-21T04:32:56.354+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.354+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 8 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.354+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.354+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.355+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:32:56.355+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:32:56.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:56.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.373+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:56.373+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:32:56.373+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.373+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:56.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.374+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:56.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:32:56.374+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:32:56.374+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.374+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.374+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.374+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.374+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:32:56.374+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:56.375+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:32:56.375+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:32:56.375+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:56.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:32:56.375+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.375+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.375+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.375+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.375+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.381+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:32:56.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:32:56.382+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:56.382+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:32:56.382+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:32:56.382+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:56.382+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.382+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.387+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:32:56.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:32:56.387+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.387+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:32:56.388+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:32:56.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:32:56.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:32:56.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:32:56.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.397+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.397+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.399+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.399+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:56.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.404+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:56.404+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:32:56.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:32:56.404+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.404+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:56.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.406+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:56.406+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:32:56.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:32:56.406+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Warmup took 51.707558ms 2023-10-21T04:32:56.406+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.406+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.406+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 8 WriterThreads 1 2023-10-21T04:32:56.406+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Stopped Billing WUs 2023-10-21T04:32:56.406+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index Index: 10032435514095102923 Partition: 8 2023-10-21T04:32:56.406+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.406+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 1 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.407+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.407+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:32:56.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.407+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.407+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:32:56.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:56.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.412+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:56.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:32:56.412+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.412+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:56.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.413+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:56.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:32:56.413+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:32:56.413+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:32:56.413+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:56.413+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:32:56.413+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:32:56.413+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:56.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.413+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:32:56.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:32:56.420+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:56.420+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:32:56.420+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:32:56.420+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:56.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.420+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.425+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:32:56.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:32:56.425+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:32:56.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:32:56.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:32:56.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:32:56.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:32:56.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.443+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.454+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.456+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:56.456+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:32:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:32:56.456+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.457+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:56.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.463+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:56.463+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:32:56.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:32:56.463+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Warmup took 56.500713ms 2023-10-21T04:32:56.463+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.463+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 1 WriterThreads 1 2023-10-21T04:32:56.463+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.463+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Stopped Billing WUs 2023-10-21T04:32:56.463+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index Index: 10032435514095102923 Partition: 1 2023-10-21T04:32:56.463+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.463+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 2 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.464+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.464+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.464+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:32:56.464+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:32:56.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:56.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.473+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:56.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:32:56.474+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.474+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.473+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:56.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.474+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:56.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:32:56.474+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:32:56.474+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.474+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.474+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.474+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.474+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.474+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:32:56.475+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:56.475+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:32:56.475+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:32:56.475+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:56.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:32:56.475+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.475+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.475+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.475+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:32:56.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:32:56.483+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:56.483+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:32:56.483+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:32:56.483+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:56.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.483+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:32:56.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:32:56.487+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:32:56.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:32:56.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:32:56.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:32:56.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:32:56.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.498+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.498+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.500+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.500+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:56.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.503+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:56.503+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:32:56.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:32:56.503+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.504+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:56.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.506+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:56.506+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:32:56.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:32:56.506+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Warmup took 42.371077ms 2023-10-21T04:32:56.506+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 2 WriterThreads 1 2023-10-21T04:32:56.506+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Stopped Billing WUs 2023-10-21T04:32:56.506+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index Index: 10032435514095102923 Partition: 2 2023-10-21T04:32:56.506+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.506+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 3 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.507+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.507+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.507+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.507+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:32:56.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.507+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.507+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:32:56.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:56.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.513+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:56.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:32:56.513+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.513+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:56.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.514+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:56.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:32:56.514+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:32:56.514+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:32:56.514+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:56.514+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:32:56.514+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:32:56.514+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:56.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.514+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.515+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.518+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:32:56.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:32:56.518+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:32:56.519+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:56.519+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:32:56.519+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:32:56.519+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:56.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:32:56.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.519+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.524+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:32:56.524+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:32:56.524+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:32:56.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:32:56.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:32:56.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:32:56.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:32:56.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.534+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.540+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.540+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:56.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.547+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:56.547+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:32:56.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:32:56.548+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.548+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:56.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.553+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:56.553+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:32:56.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:32:56.553+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Warmup took 46.644238ms 2023-10-21T04:32:56.553+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.553+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.553+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 3 WriterThreads 1 2023-10-21T04:32:56.553+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Stopped Billing WUs 2023-10-21T04:32:56.553+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index Index: 10032435514095102923 Partition: 3 2023-10-21T04:32:56.553+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.553+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 4 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.554+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.554+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:32:56.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.554+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.555+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:32:56.564+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2670343739772083055:INDEX_STATE_RECOVERED 8965923595543532942:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:32:56.565+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 10606939961365971246, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:32:56.565+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 10606939961365971246, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:32:56.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:56.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.569+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:56.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:32:56.569+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.569+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:56.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.572+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:56.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:32:56.572+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.572+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:32:56.573+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:32:56.573+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:56.573+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:32:56.573+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:32:56.573+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:56.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.573+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.585+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8732144615635911668,realInstId: 0, indexDefn DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:32:56.586+05:30 [Info] Indexer::handleRecoverIndex InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:32:56.586+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 367.192µs 2023-10-21T04:32:56.586+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 432.285µs 2023-10-21T04:32:56.586+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:56.586+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 5 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:56.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.587+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.587+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:32:56.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:32:56.594+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:56.594+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:32:56.594+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:32:56.594+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:56.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.594+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.586+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8732144615635911668, indexDefn DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:32:56.595+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 29.716565ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:56.597+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:56.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.601+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.601+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 8236 2023-10-21T04:32:56.609+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:56.609+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:56.611+05:30 [Info] switched currmeta from 200 -> 201 force true 2023-10-21T04:32:56.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:56.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.611+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:56.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:32:56.611+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.611+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.612+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:32:56.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:32:56.612+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:32:56.612+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:32:56.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:56.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.613+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:56.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:32:56.613+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.613+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:32:56.613+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.613+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.613+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.613+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:32:56.613+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.614+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:56.614+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:32:56.614+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:32:56.614+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:56.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:32:56.614+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.614+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.614+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.614+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:32:56.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:32:56.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:32:56.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:32:56.619+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:32:56.619+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:56.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.619+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.620+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.623+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.623+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.626+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:32:56.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:32:56.626+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.626+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.626+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:32:56.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:32:56.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:56.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.629+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:56.629+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:32:56.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:32:56.629+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.629+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:32:56.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:32:56.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:32:56.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:56.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.632+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:56.632+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:32:56.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:32:56.632+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Warmup took 78.896667ms 2023-10-21T04:32:56.633+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 4 WriterThreads 1 2023-10-21T04:32:56.633+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Stopped Billing WUs 2023-10-21T04:32:56.633+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index Index: 10032435514095102923 Partition: 4 2023-10-21T04:32:56.633+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.633+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.633+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 5 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.633+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.633+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.633+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.633+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:32:56.633+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:32:56.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.635+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.635+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.640+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.640+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.644+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.644+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.644+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.644+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:32:56.644+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:56.644+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:32:56.644+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:32:56.644+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:56.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.644+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:56.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.648+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:56.648+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:32:56.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:32:56.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:56.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.654+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:56.654+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:32:56.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:32:56.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Warmup took 67.410115ms 2023-10-21T04:32:56.654+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.654+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.654+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 5 WriterThreads 1 2023-10-21T04:32:56.654+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Stopped Billing WUs 2023-10-21T04:32:56.654+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index Index: 8732144615635911668 Partition: 5 2023-10-21T04:32:56.654+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:56.654+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 6 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:56.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.655+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 34133 recovery UsedSpace 8236 2023-10-21T04:32:56.655+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 8236 2023-10-21T04:32:56.664+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:32:56.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:32:56.665+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:56.665+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:32:56.665+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:32:56.665+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:56.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.665+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.679+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:32:56.679+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.679+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:56.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:32:56.679+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:32:56.679+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.679+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.679+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.679+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.679+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:32:56.680+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:32:56.680+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:56.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.680+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.689+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:32:56.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:32:56.689+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:32:56.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:32:56.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:32:56.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:32:56.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:32:56.700+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:32:56.700+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:56.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.700+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.701+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:32:56.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.715+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.715+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.716+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:32:56.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:32:56.716+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.719+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:32:56.719+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:32:56.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:56.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.723+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:56.723+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:32:56.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:32:56.724+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.724+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:32:56.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:32:56.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:56.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.736+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:56.736+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:32:56.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:32:56.736+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Warmup took 102.852459ms 2023-10-21T04:32:56.736+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 5 WriterThreads 1 2023-10-21T04:32:56.736+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Stopped Billing WUs 2023-10-21T04:32:56.736+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index Index: 10032435514095102923 Partition: 5 2023-10-21T04:32:56.736+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:32:56.736+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.736+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.736+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 6 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:32:56.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:32:56.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.746+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.747+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:32:56.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.747+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.747+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:32:56.754+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:56.754+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:56.759+05:30 [Info] switched currmeta from 201 -> 202 force true 2023-10-21T04:32:56.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.762+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:56.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.767+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:56.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:32:56.767+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.767+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.767+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:56.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.768+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.768+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:56.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:32:56.768+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:32:56.768+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:32:56.768+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:56.768+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:32:56.768+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:32:56.768+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:56.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.768+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:56.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.772+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:56.772+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:32:56.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:32:56.772+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.772+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.774+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:32:56.774+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:56.774+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:32:56.774+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:32:56.774+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.774+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.774+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:32:56.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:32:56.774+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Warmup took 120.027602ms 2023-10-21T04:32:56.775+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 6 WriterThreads 1 2023-10-21T04:32:56.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.775+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Stopped Billing WUs 2023-10-21T04:32:56.775+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index Index: 8732144615635911668 Partition: 6 2023-10-21T04:32:56.775+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:56.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.775+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 7 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:56.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.775+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.775+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:32:56.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.776+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.776+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 8236 2023-10-21T04:32:56.789+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:32:56.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:32:56.789+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:32:56.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:32:56.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:56.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.793+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:56.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:32:56.793+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.793+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:56.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.793+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:56.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:32:56.793+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:32:56.793+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.793+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.793+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.793+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:32:56.794+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:32:56.794+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:56.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.794+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:32:56.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:32:56.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:32:56.809+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:32:56.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:32:56.810+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:32:56.810+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:56.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.810+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:32:56.813+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.814+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:32:56.822+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.822+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.822+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:32:56.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:32:56.823+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:32:56.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:32:56.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:56.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:32:56.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:32:56.829+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:56.829+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:32:56.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:32:56.829+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.829+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:32:56.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:32:56.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:56.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:32:56.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:32:56.835+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:56.835+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:32:56.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:32:56.835+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Warmup took 98.893967ms 2023-10-21T04:32:56.835+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.835+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.836+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 6 WriterThreads 1 2023-10-21T04:32:56.836+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Stopped Billing WUs 2023-10-21T04:32:56.836+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index Index: 10032435514095102923 Partition: 6 2023-10-21T04:32:56.836+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 10032435514095102923, shardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:32:56.836+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 10032435514095102923, partnShardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:32:56.836+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:56.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:32:56.836+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10032435514095102923 2023-10-21T04:32:56.836+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:32:56.836+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:32:56.837+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:56.837+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:32:56.837+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:56.837+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:32:56.837+05:30 [Info] StorageMgr::openSnapshot IndexInst:10032435514095102923 Partition:2 No Snapshot Found. 2023-10-21T04:32:56.837+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Adding Nil Snapshot. 2023-10-21T04:32:56.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.847+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.847+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.854+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.854+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.854+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.574036ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:56.855+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 137.337µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:56.877+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:56.877+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:56.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:56.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.884+05:30 [Info] switched currmeta from 202 -> 203 force true 2023-10-21T04:32:56.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.884+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:56.884+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:32:56.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:32:56.884+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.884+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:56.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.886+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:56.886+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:32:56.886+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:32:56.886+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Warmup took 111.316875ms 2023-10-21T04:32:56.887+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 7 WriterThreads 1 2023-10-21T04:32:56.887+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Stopped Billing WUs 2023-10-21T04:32:56.887+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index Index: 8732144615635911668 Partition: 7 2023-10-21T04:32:56.887+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:56.887+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 8 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:56.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.887+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.887+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:32:56.888+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.888+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.888+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.889+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:32:56.919+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:56.919+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:56.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:56.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.920+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:56.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:32:56.920+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.920+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:56.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.921+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:56.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:32:56.922+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:32:56.922+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:32:56.922+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:32:56.922+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:56.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.922+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.925+05:30 [Info] switched currmeta from 203 -> 204 force true 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:32:56.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:56.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:32:56.937+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:32:56.937+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:56.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:56.937+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:56.938+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:56.938+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:56.954+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:56.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:32:56.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:32:56.955+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:56.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:32:56.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:32:56.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:32:56.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:32:56.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:32:56.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:32:56.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.969+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.971+05:30 [Error] ScanCoordinator: Unable to compute index items_count for bucket_%2/idx_partitioned_defer/8732144615635911668 state 0 (Index not found) 2023-10-21T04:32:56.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:56.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.974+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:56.974+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:56.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:56.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:56.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:56.979+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:56.979+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:32:56.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:32:56.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:56.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:56.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:56.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:56.987+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:56.987+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:32:56.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:32:56.988+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Warmup took 100.549136ms 2023-10-21T04:32:56.988+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:56.988+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 8 WriterThreads 1 2023-10-21T04:32:56.988+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Stopped Billing WUs 2023-10-21T04:32:56.988+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index Index: 8732144615635911668 Partition: 8 2023-10-21T04:32:56.988+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:56.988+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 1 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:56.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:56.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:56.989+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.989+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:56.989+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:32:56.989+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:32:56.994+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:57.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.005+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:57.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:32:57.005+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.005+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:57.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.006+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:57.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:32:57.006+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:32:57.006+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:57.006+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.006+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.006+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.006+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:32:57.006+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:57.007+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:32:57.007+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:32:57.007+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:57.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:32:57.007+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.007+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.007+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.007+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.007+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.014+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:32:57.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:32:57.014+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:57.014+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.014+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.014+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:32:57.015+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:57.015+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:32:57.015+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:32:57.015+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:57.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:32:57.015+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.015+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.015+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.015+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.027+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:32:57.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:32:57.027+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:57.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:32:57.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:32:57.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:32:57.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:32:57.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:32:57.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:32:57.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:57.042+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.042+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:57.047+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.048+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:57.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.050+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:57.050+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:32:57.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:32:57.050+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.050+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:57.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.057+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:57.057+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:32:57.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:32:57.057+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Warmup took 69.156156ms 2023-10-21T04:32:57.058+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.058+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.058+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 1 WriterThreads 1 2023-10-21T04:32:57.059+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Stopped Billing WUs 2023-10-21T04:32:57.059+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index Index: 8732144615635911668 Partition: 1 2023-10-21T04:32:57.061+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:57.063+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 2 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:57.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:57.063+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:57.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:57.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:32:57.064+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:57.064+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:32:57.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:57.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.073+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:57.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:32:57.073+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.073+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:57.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.074+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:57.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:32:57.075+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:32:57.075+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:57.075+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.075+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.076+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.075+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.075+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:32:57.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:57.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:32:57.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:32:57.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:57.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:32:57.078+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.079+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.079+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.079+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.087+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:32:57.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:32:57.088+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:32:57.088+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:57.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.097+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:32:57.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:32:57.097+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:57.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:32:57.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:32:57.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:32:57.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:32:57.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:32:57.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:32:57.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:57.115+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.115+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:57.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:57.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.131+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:57.131+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:32:57.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:32:57.131+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.131+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.137+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.137+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.144+05:30 [Info] switched currmeta from 204 -> 205 force true 2023-10-21T04:32:57.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:57.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.149+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:57.149+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:32:57.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:32:57.149+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Warmup took 85.693634ms 2023-10-21T04:32:57.149+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 2 WriterThreads 1 2023-10-21T04:32:57.149+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Stopped Billing WUs 2023-10-21T04:32:57.149+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index Index: 8732144615635911668 Partition: 2 2023-10-21T04:32:57.149+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:57.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.150+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 3 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:57.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:57.152+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:57.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:57.152+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:57.152+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 37236 recovery UsedSpace 8236 2023-10-21T04:32:57.152+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:32:57.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:57.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.163+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:57.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:32:57.163+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.163+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:57.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.169+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:57.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.169+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:32:57.169+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:32:57.169+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:32:57.169+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:57.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:32:57.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:32:57.186+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:32:57.186+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:57.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.195+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:32:57.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:32:57.195+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:57.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:32:57.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:32:57.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:32:57.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:32:57.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:32:57.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:32:57.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:57.209+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.209+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:57.213+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.213+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:57.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.217+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:57.217+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:32:57.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:32:57.217+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.217+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:57.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.220+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:57.220+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:32:57.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:32:57.221+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.221+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Warmup took 70.597068ms 2023-10-21T04:32:57.221+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.221+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 3 WriterThreads 1 2023-10-21T04:32:57.221+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Stopped Billing WUs 2023-10-21T04:32:57.221+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index Index: 8732144615635911668 Partition: 3 2023-10-21T04:32:57.221+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] 2023-10-21T04:32:57.221+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8732144615635911668 PartitionId: 4 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:32:57.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:57.221+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:57.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:57.222+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:32:57.222+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 37236 recovery UsedSpace 8236 2023-10-21T04:32:57.222+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.233+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:32:57.233+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:57.233+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.233+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:57.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:32:57.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:32:57.234+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:32:57.234+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:57.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.234+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:32:57.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:32:57.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:32:57.257+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:32:57.257+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:57.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:32:57.257+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:32:57.269+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:32:57.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:32:57.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:32:57.269+05:30 [Info] bucket_%2/idx_secondary/Backstore#2670343739772083055:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:32:57.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:32:57.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:32:57.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:32:57.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:32:57.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:32:57.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.280+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:32:57.286+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[10032435514095102923:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:32:57.286+05:30 [Info] Indexer::handleMergePartition Source 4708891605960345271 Target 10032435514095102923 2023-10-21T04:32:57.286+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007c14540 2023-10-21T04:32:57.287+05:30 [Info] MergePartition: sent async request to update index instance 10032435514095102923 rstate moved to ACTIVE 2023-10-21T04:32:57.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:32:57.288+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.288+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:32:57.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:32:57.294+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:32:57.294+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:32:57.296+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 4708891605960345271, realInst: 10032435514095102923 2023-10-21T04:32:57.296+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [16961575949607441659 4790214984019693894] belonging to collectionId: c, ttid: ShardTokenb2:f8:fb:ca:53:2e:1c:c4. Initiating build 2023-10-21T04:32:57.295+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.847943ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:57.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:57.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:32:57.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:32:57.306+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:57.306+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:32:57.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:32:57.307+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.307+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.319+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.319+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.321+05:30 [Info] switched currmeta from 205 -> 206 force true 2023-10-21T04:32:57.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:57.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:32:57.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:32:57.323+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:57.323+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:32:57.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:32:57.323+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Warmup took 101.729836ms 2023-10-21T04:32:57.323+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 4 WriterThreads 1 2023-10-21T04:32:57.323+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Stopped Billing WUs 2023-10-21T04:32:57.323+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index Index: 8732144615635911668 Partition: 4 2023-10-21T04:32:57.323+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8732144615635911668, shardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]] err: 2023-10-21T04:32:57.323+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8732144615635911668, partnShardIdMap: map[1:[17406544375239214575 10170029854712240877] 2:[17406544375239214575 10170029854712240877] 3:[17406544375239214575 10170029854712240877] 4:[17406544375239214575 10170029854712240877] 5:[17406544375239214575 10170029854712240877] 6:[17406544375239214575 10170029854712240877] 7:[17406544375239214575 10170029854712240877] 8:[17406544375239214575 10170029854712240877]], err: 2023-10-21T04:32:57.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:32:57.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:32:57.323+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:57.325+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8732144615635911668 2023-10-21T04:32:57.325+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8732144615635911668 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:32:57.325+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:32:57.325+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:57.325+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8732144615635911668 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:32:57.325+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:57.325+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Partitions [7 8 1 2 3 4 5 6] 2023-10-21T04:32:57.325+05:30 [Info] StorageMgr::openSnapshot IndexInst:8732144615635911668 Partition:1 No Snapshot Found. 2023-10-21T04:32:57.325+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Adding Nil Snapshot. 2023-10-21T04:32:57.351+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [15864358084358521096 4083560586502645495] &{2} 2023-10-21T04:32:57.352+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.352+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.351+05:30 [Info] Indexer::handleBuildRecoveredIndexes [15864358084358521096 4083560586502645495] 2023-10-21T04:32:57.353+05:30 [Info] bulkUpdateRState: Index instance 15864358084358521096 rstate moved to PENDING 2023-10-21T04:32:57.353+05:30 [Info] bulkUpdateRState: Index instance 4083560586502645495 rstate moved to PENDING 2023-10-21T04:32:57.353+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [15864358084358521096 4083560586502645495] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:32:57.357+05:30 [Info] switched currmeta from 206 -> 208 force true 2023-10-21T04:32:57.358+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:57.360+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:57.362+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:57.372+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.372+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.378+05:30 [Info] switched currmeta from 208 -> 208 force true 2023-10-21T04:32:57.415+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:32:57.419+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 55765 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:57.420+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:32:57.420+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:32:57.421+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:32:57.423+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 55765 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:57.423+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:32:57.423+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:32:57.423+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:32:57.424+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId 55766 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:32:57.426+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:32:57.428+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:32:57.428+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15864358084358521096 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:32:57.429+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_1:_default:c1:2]] 2023-10-21T04:32:57.429+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4083560586502645495 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:32:57.429+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:32:57.429+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 55766 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:32:57.439+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:32:57.439+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:32:57.440+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:32:57.440+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:57.441+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 89.545081ms 2023-10-21T04:32:57.441+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 89.586029ms 2023-10-21T04:32:57.441+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [15864358084358521096 4083560586502645495] 2023-10-21T04:32:57.441+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 143.759306ms len(expediates) 3 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:57.443+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4083560586502645495:true 15864358084358521096:true] 2023-10-21T04:32:57.444+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:32:57.460+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60064" doAuth() ... 2023-10-21T04:32:57.460+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60064" auth successful 2023-10-21T04:32:57.460+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:60064" +1 2023-10-21T04:32:57.479+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 37.457804ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:57.485+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:57.494+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.494+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.494+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.663396ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:57.502+05:30 [Info] switched currmeta from 208 -> 210 force true 2023-10-21T04:32:57.513+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.131532ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:57.513+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 111.615µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:32:57.596+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.602+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8732144615635911668:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:32:57.603+05:30 [Info] Indexer::handleMergePartition Source 3283688418228810121 Target 8732144615635911668 2023-10-21T04:32:57.603+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005083d40 2023-10-21T04:32:57.605+05:30 [Info] MergePartition: sent async request to update index instance 8732144615635911668 rstate moved to ACTIVE 2023-10-21T04:32:57.648+05:30 [Info] switched currmeta from 210 -> 212 force true 2023-10-21T04:32:57.740+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.741+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.749+05:30 [Info] switched currmeta from 212 -> 212 force true 2023-10-21T04:32:57.749+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 144.516683ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:57.749+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 3283688418228810121, realInst: 8732144615635911668 2023-10-21T04:32:57.750+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [13268712595133993155 8910459001587546151] belonging to collectionId: c, ttid: ShardTokena2:b2:c5:e2:b2:69:50:c0. Initiating build 2023-10-21T04:32:57.775+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.776+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.794+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 99.93178717598494 InstId: 15864358084358521096 RealInstId: 15864358084358521096 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:32:57.824+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 99.93178717598494 InstId: 4083560586502645495 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:32:57.794+05:30 [Info] switched currmeta from 212 -> 213 force true 2023-10-21T04:32:57.807+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [2670343739772083055 8965923595543532942] &{2} 2023-10-21T04:32:57.827+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 99.7271487039543 InstId: 2670343739772083055 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:32:57.839+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 99.7271487039543 InstId: 8965923595543532942 RealInstId: 8965923595543532942 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:32:57.838+05:30 [Info] Indexer::handleBuildRecoveredIndexes [2670343739772083055 8965923595543532942] 2023-10-21T04:32:57.841+05:30 [Info] bulkUpdateRState: Index instance 2670343739772083055 rstate moved to PENDING 2023-10-21T04:32:57.841+05:30 [Info] bulkUpdateRState: Index instance 8965923595543532942 rstate moved to PENDING 2023-10-21T04:32:57.841+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [2670343739772083055 8965923595543532942] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:32:57.841+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 99.7271487039543 InstId: 4238649781004957942 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:32:57.844+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 99.7271487039543 InstId: 6043349427976659583 RealInstId: 6043349427976659583 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:32:57.844+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 10402379389576076017 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:32:57.844+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 3345094951175719474 RealInstId: 3345094951175719474 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:32:57.844+05:30 [Info] ShardRebalancer::computeProgress 0.7029988489085949 2023-10-21T04:32:57.845+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:57.845+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:57.846+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:57.877+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.877+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:57.885+05:30 [Info] switched currmeta from 213 -> 216 force true 2023-10-21T04:32:57.937+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:32:57.946+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:57.948+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:32:57.946+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:57.950+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 21066 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:57.951+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:32:57.952+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:32:57.952+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:32:57.952+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 21066 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:57.952+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:_default:c1:{0xc00abe3b00 }] 2023-10-21T04:32:57.952+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:57.952+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:32:57.952+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:32:57.952+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:32:57.952+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId 21067 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:32:57.952+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:57.952+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:32:57.952+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:32:57.952+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_%2:_default:c1:1 bucket_1:_default:c1:2]] 2023-10-21T04:32:57.952+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 2670343739772083055 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:32:57.952+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_%2:_default:c1:2 bucket_1:_default:c1:2]] 2023-10-21T04:32:57.952+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 8965923595543532942 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:32:57.952+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:32:57.952+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=18,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 21067 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:32:57.953+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc00048a780 } bucket_1:_default:c1:{0xc00abe3b00 }] 2023-10-21T04:32:57.953+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:57.954+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 115.98222ms 2023-10-21T04:32:57.955+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [2670343739772083055 8965923595543532942] 2023-10-21T04:32:57.955+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 203.84693ms len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:32:57.957+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:32:57.958+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[2670343739772083055:true 8965923595543532942:true] 2023-10-21T04:32:57.958+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 120.205425ms 2023-10-21T04:32:57.973+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:32:57.974+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.90599ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:57.987+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:58.003+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 28.440233ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:58.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.042+05:30 [Info] switched currmeta from 216 -> 222 force true 2023-10-21T04:32:58.049+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.049+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.051+05:30 [Info] switched currmeta from 222 -> 222 force true 2023-10-21T04:32:58.106+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.106+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.111+05:30 [Info] switched currmeta from 222 -> 223 force true 2023-10-21T04:32:58.146+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.149+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.154+05:30 [Info] switched currmeta from 223 -> 224 force true 2023-10-21T04:32:58.195+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1f784fc0142ee312f0d92d98bd188c9a Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/1f784fc0142ee312f0d92d98bd188c9a Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029988489085949 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]}]} 2023-10-21T04:32:58.195+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1f784fc0142ee312f0d92d98bd188c9a Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/1f784fc0142ee312f0d92d98bd188c9a Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029988489085949 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]}]} 2023-10-21T04:32:58.195+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:32:58.195+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:32:58.200+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 6] 2023-10-21T04:32:58.202+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 6] 2023-10-21T04:32:58.267+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.267+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.271+05:30 [Info] switched currmeta from 224 -> 226 force true 2023-10-21T04:32:58.284+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.284+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.295+05:30 [Info] switched currmeta from 226 -> 226 force true 2023-10-21T04:32:58.490+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:58.548+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 1949031200982 23 55766 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:58.551+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 245782287380001 25 55766 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:32:58.551+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 52522903947059 28 55766 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:32:58.553+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 93079113162334 18 55766 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:32:58.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 205201940883916 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.559+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 244791485011286 23 55766 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:58.561+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 177815184127237 17 55766 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:32:58.563+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 167580527121081 23 55766 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:58.563+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 54236768606397 20 55766 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:58.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 134409829206048 26 55766 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:32:58.567+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 58522600746999 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.567+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 160381752747972 18 55766 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:32:58.569+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 142387448675642 30 55766 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:32:58.572+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 90115353725987 21 55766 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:58.573+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 161230548567164 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.575+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 85302001459467 21 55766 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:58.575+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 123957315902602 28 55766 172.31.5.112:9000. HWT [27-28,28]. 2023-10-21T04:32:58.577+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 213108110408291 23 55766 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:58.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 163460438181870 34 55766 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:32:58.584+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 151273673850546 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.584+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 150690956328758 26 55766 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:32:58.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 258671969827141 19 55766 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:58.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 19989312661772 15 55766 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:32:58.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 53829314402679 32 55766 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:32:58.587+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 147946970349859 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.588+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 114823206239184 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 25510109770064 28 55766 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:32:58.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 232028101514735 19 55766 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:58.594+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 124811039198764 18 55766 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:32:58.594+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 68847799765163 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.594+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 8305763409275 21 55766 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:32:58.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 33310272161722 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 207489375637709 12 55766 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:32:58.601+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 107407811917992 20 55766 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:58.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 117298435979365 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 100746255343609 19 55766 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:58.606+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 228857796432725 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.606+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 203503670520010 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.608+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 158357249947806 21 55766 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:58.608+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 47156288300713 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.611+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 236904976023311 22 55766 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:32:58.611+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 104136033125373 29 55766 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:32:58.613+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 196269910725961 26 55766 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:32:58.613+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 124310544527292 21 55766 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:58.618+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 246959094981553 34 55766 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:32:58.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 276463747117997 23 55766 172.31.5.112:9000. HWT [22-23,23]. 2023-10-21T04:32:58.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 65885584526743 20 55766 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:58.624+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 172365063201867 21 55766 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:58.626+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 79077905592013 17 55766 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:32:58.629+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 212884822297038 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.629+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 246582249071123 20 55766 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:58.631+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 142324128597367 22 55766 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:32:58.631+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 213334533059126 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.632+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 55145461136105 16 55766 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:32:58.632+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 254944616021293 24 55766 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:58.636+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 219528837936411 25 55766 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:58.638+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 194568213794435 29 55766 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:32:58.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 25511316571199 16 55766 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:32:58.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 36300629322478 32 55766 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:32:58.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 80145536651832 17 55766 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:32:58.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 223881084081370 17 55766 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:32:58.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 279733370822238 23 55766 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:58.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 106620692996001 23 55766 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:58.654+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 InstanceIds [15864358084358521096 4083560586502645495] 2023-10-21T04:32:58.654+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 55766 2023-10-21T04:32:58.654+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 55766 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 15864358084358521096 INDEX_STATE_CATCHUP 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 4083560586502645495 INDEX_STATE_CATCHUP 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 15864358084358521096 true 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 4083560586502645495 true 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 15864358084358521096 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 55766 BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} , fetchKVSeq false 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:32:58.654+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:32:58.654+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 55766 STREAM_ACTIVE 2023-10-21T04:32:58.654+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 55766 2023-10-21T04:32:58.654+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1:_default:c1 55766 2023-10-21T04:32:58.655+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:58.655+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:58.655+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:58.655+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:58.656+05:30 [Info] send progress stats to clients 2023-10-21T04:32:58.656+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 55766 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:58.656+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15864358084358521096 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:32:58.656+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4083560586502645495 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:32:58.656+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:32:58.656+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:32:58.656+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:32:58.656+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 55766 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:58.656+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_default:c1:{0xc00048a780 }] 2023-10-21T04:32:58.656+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:58.656+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:32:58.656+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:58.656+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:32:58.657+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:32:58.657+05:30 [Error] MutationStreamReader::updateVbuuidInFilter MissingkeyspaceId bucket_1:_default:c1 vb 63 vbuuid 25657368013413 in Filter for Stream INIT_STREAM 2023-10-21T04:32:58.657+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:32:58.658+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1:_default:c1 2023-10-21T04:32:58.658+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 25657368013413 17 55766 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:32:58.658+05:30 [Warn] Timekeeper::handleStreamBegin Received StreamBegin for Inactive KeyspaceId bucket_1:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:32:58.665+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.517753ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:58.672+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.994704ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:58.702+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.704+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.713+05:30 [Info] switched currmeta from 226 -> 228 force true 2023-10-21T04:32:58.728+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.728+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.730+05:30 [Info] switched currmeta from 228 -> 228 force true 2023-10-21T04:32:58.818+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.818+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.833+05:30 [Info] switched currmeta from 228 -> 229 force true 2023-10-21T04:32:58.842+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.842+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:58.846+05:30 [Info] switched currmeta from 229 -> 230 force true 2023-10-21T04:32:58.988+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:32:58.999+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:58.999+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.006+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:59.012+05:30 [Info] switched currmeta from 230 -> 230 force true 2023-10-21T04:32:59.058+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:59.058+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.064+05:30 [Info] switched currmeta from 230 -> 231 force true 2023-10-21T04:32:59.120+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:59.122+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.128+05:30 [Info] switched currmeta from 231 -> 234 force true 2023-10-21T04:32:59.138+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:59.139+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.149+05:30 [Info] switched currmeta from 234 -> 234 force true 2023-10-21T04:32:59.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 274143649547516 29 21067 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:32:59.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 162049908746825 25 21067 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:32:59.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 206082810619393 24 21067 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:59.268+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 143687065777286 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.268+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 120349894144442 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 212967951483349 28 21067 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:32:59.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 116493903186721 22 21067 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:32:59.274+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 263088038522959 18 21067 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:32:59.274+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 78446412189949 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.280+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 136568163328749 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.283+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 173457131919400 12 21067 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:32:59.284+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 127746572208200 19 21067 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:59.284+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 261951617336488 35 21067 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:32:59.284+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 188740876924377 22 21067 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:32:59.284+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 58733680850130 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.285+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 277338544220217 21 21067 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:59.287+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 245247859472006 18 21067 172.31.5.112:9000. HWT [17-18,18]. 2023-10-21T04:32:59.289+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 206088069380492 13 21067 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:32:59.292+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 34510046292135 16 21067 172.31.5.112:9000. HWT [16-16,16]. 2023-10-21T04:32:59.296+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 250426459207023 28 21067 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:32:59.300+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 274158831830257 20 21067 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:59.300+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 125696667341112 27 21067 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:32:59.311+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 35716737784090 24 21067 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:59.317+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 191500187207978 21 21067 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:59.317+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 106231614376481 21 21067 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:59.318+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 118920785117980 21 21067 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:32:59.323+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 137256376714252 22 21067 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:32:59.326+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5906787184229 20 21067 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:59.331+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 146375746634523 17 21067 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:32:59.335+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 112482982607415 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.338+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 241391494503904 28 21067 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:32:59.338+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 41101832875819 19 21067 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:59.341+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 121450709505223 25 21067 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:59.343+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 216238759475962 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 168446351781198 25 21067 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:59.347+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 127702263378100 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.352+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 169268048765249 19 21067 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:59.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 245900669851618 19 21067 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:59.368+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 149033089785397 28 21067 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:32:59.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 160399698726237 27 21067 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:32:59.377+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 217237947818243 20 21067 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:59.377+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 100008922352935 17 21067 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:32:59.378+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 88718752743571 25 21067 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:59.378+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 110286285845649 31 21067 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:32:59.378+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 106297365959267 27 21067 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:32:59.378+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 39045953044628 18 21067 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:32:59.391+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 264001907139144 22 21067 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:32:59.391+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 266604622117081 26 21067 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:32:59.391+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 162870112426804 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.391+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 132470209691538 26 21067 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:32:59.393+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 63737850123332 19 21067 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:59.396+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 32274016651656 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.401+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 225662991941187 26 21067 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:32:59.402+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 84581802233088 22 21067 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:32:59.406+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 241607113601772 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.406+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 272523968790649 20 21067 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:32:59.410+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 197957220074784 24 21067 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:32:59.423+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 97677619676504 31 21067 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:32:59.423+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 136038310030103 28 21067 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:32:59.427+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 105415986286631 18 21067 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:32:59.427+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 27479155597282 29 21067 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:32:59.431+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 161870047388121 25 21067 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:32:59.444+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 72072509155168 23 21067 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:32:59.444+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 77598347829810 19 21067 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:32:59.453+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 InstanceIds [4083560586502645495 15864358084358521096 2670343739772083055 8965923595543532942] 2023-10-21T04:32:59.453+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 21067 2023-10-21T04:32:59.453+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 21067 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 2670343739772083055 INDEX_STATE_CATCHUP 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 8965923595543532942 INDEX_STATE_CATCHUP 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 2670343739772083055 true 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 8965923595543532942 true 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 2670343739772083055 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 21067 BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=18,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} , fetchKVSeq false 2023-10-21T04:32:59.453+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:32:59.453+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 21067 STREAM_ACTIVE 2023-10-21T04:32:59.454+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 21067 2023-10-21T04:32:59.454+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2:_default:c1 21067 2023-10-21T04:32:59.454+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:59.454+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:32:59.454+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:32:59.455+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:32:59.456+05:30 [Info] send progress stats to clients 2023-10-21T04:32:59.456+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 21067 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:59.456+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 2670343739772083055 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:32:59.457+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 8965923595543532942 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:32:59.457+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:32:59.457+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:32:59.457+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:32:59.468+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 21067 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:59.468+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:32:59.469+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:32:59.469+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:32:59.469+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:32:59.469+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:32:59.470+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:32:59.470+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:60064: use of closed network connection 2023-10-21T04:32:59.470+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60064" closed ! 2023-10-21T04:32:59.470+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:32:59.470+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:60064" already gone 2023-10-21T04:32:59.470+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c1 4 93079113162334 18 2023-10-21T04:32:59.470+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:32:59.470+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c1 2 52522903947059 28 2023-10-21T04:32:59.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:32:59.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c1 3 205201940883916 24 2023-10-21T04:32:59.470+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:60064" exit: read tcp4 127.0.0.1:9121->127.0.0.1:60064: use of closed network connection 2023-10-21T04:32:59.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:32:59.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c1 5 244791485011286 23 2023-10-21T04:32:59.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:32:59.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c1 0 1949031200982 23 2023-10-21T04:32:59.471+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:32:59.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:32:59.471+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_default:c1 1 245782287380001 25 2023-10-21T04:32:59.471+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2:_default:c1 2023-10-21T04:32:59.471+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_default:c1 Stream INIT_STREAM. Ignored. 2023-10-21T04:32:59.485+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:59.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.493+05:30 [Info] switched currmeta from 234 -> 235 force true 2023-10-21T04:32:59.495+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 38.166273ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:59.519+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:32:59.522+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:59.522+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.524+05:30 [Info] switched currmeta from 235 -> 236 force true 2023-10-21T04:32:59.549+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 53.613304ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:32:59.577+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:59.577+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.580+05:30 [Info] switched currmeta from 236 -> 238 force true 2023-10-21T04:32:59.591+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:32:59.591+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:32:59.596+05:30 [Info] switched currmeta from 238 -> 238 force true 2023-10-21T04:32:59.624+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1]. Err projector.topicMissing 2023-10-21T04:32:59.624+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:32:59.624+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating projector.topicMissing As Success 2023-10-21T04:32:59.624+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 21067 2023-10-21T04:32:59.624+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:32:59.624+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:32:59.624+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:59.624+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:32:59.624+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:32:59.624+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:32:59.624+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 36450 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:59.624+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:32:59.625+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:32:59.625+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:32:59.625+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 36451 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:32:59.626+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:32:59.626+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1] INIT_STREAM:map[]] 2023-10-21T04:32:59.626+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2] INIT_STREAM:map[]] 2023-10-21T04:32:59.626+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:32:59.626+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 36451 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:32:59.628+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:32:59.628+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:32:59.628+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:32:59.629+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 55766 2023-10-21T04:32:59.629+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1:_default:c1 2023-10-21T04:32:59.629+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:32:59.629+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:59.629+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:32:59.630+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 21886 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:59.630+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:32:59.630+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:32:59.630+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:32:59.630+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 21886 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:32:59.630+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc007fe4480 }] 2023-10-21T04:32:59.630+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:59.630+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:32:59.630+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:32:59.630+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:32:59.631+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 21887 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:32:59.631+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:32:59.631+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:32:59.631+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:32:59.631+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:32:59.631+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:32:59.631+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:32:59.633+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:32:59.631+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 21887 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:32:59.641+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc007fe4480 } bucket_1:{0xc007fe46c0 }] 2023-10-21T04:32:59.647+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:32:59.657+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:32:59.697+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:56042" doAuth() ... 2023-10-21T04:32:59.697+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:56042" auth successful 2023-10-21T04:32:59.699+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:56042" +1 2023-10-21T04:32:59.874+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6043349427976659583:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:32:59.874+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4238649781004957942:[0] 6043349427976659583:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:33:00.010+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:00.010+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006076540 2023-10-21T04:33:00.010+05:30 [Info] handleUpdateIndexRState: Index instance 2670343739772083055 rstate moved to ACTIVE 2023-10-21T04:33:00.022+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.849897ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.022+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 17434304260397056806. Retrying... 2023-10-21T04:33:00.022+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[2670343739772083055:true 8965923595543532942:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:33:00.022+05:30 [Info] Indexer::handleMergePartition Source 16711923683289406198 Target 8965923595543532942 2023-10-21T04:33:00.022+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00e6baa20 2023-10-21T04:33:00.045+05:30 [Info] MergePartition: sent async request to update index instance 8965923595543532942 rstate moved to ACTIVE 2023-10-21T04:33:00.061+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.532787ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.061+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 16711923683289406198, realInst: 8965923595543532942 2023-10-21T04:33:00.066+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 3381524789655566900, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:33:00.066+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 3381524789655566900, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:33:00.072+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:00.072+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:00.086+05:30 [Info] switched currmeta from 238 -> 240 force true 2023-10-21T04:33:00.166+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17434304260397056806,realInstId: 0, indexDefn DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:33:00.168+05:30 [Info] Indexer::handleRecoverIndex InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:33:00.169+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 485.497µs 2023-10-21T04:33:00.169+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 738.256µs 2023-10-21T04:33:00.169+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17434304260397056806, indexDefn DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:33:00.170+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 103.849463ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.169+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17434304260397056806 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[17406544375239214575 10170029854712240877]] 2023-10-21T04:33:00.171+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17434304260397056806 PartitionId: 0 using shardIds: [17406544375239214575 10170029854712240877], rebalance running: true 2023-10-21T04:33:00.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:00.171+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:33:00.171+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 9 count: 6 metaSz: 2493 rp [1] sn: 360005 count: 6 metaSz: 2493 2023-10-21T04:33:00.172+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: doInit: data UsedSpace 66989 recovery UsedSpace 74828 2023-10-21T04:33:00.185+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:00.185+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:00.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:33:00.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:00.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:00.189+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:33:00.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex started 2023-10-21T04:33:00.189+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:33:00.189+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:00.189+05:30 [Info] bucket_%2/idx_secondary/Mainstore#2670343739772083055:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:00.206+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:00.211+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Warmup took 40.563823ms 2023-10-21T04:33:00.211+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17434304260397056806 partitionId 0 WriterThreads 4 2023-10-21T04:33:00.211+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17434304260397056806 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index Index: 17434304260397056806 Partition: 0 2023-10-21T04:33:00.212+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17434304260397056806, shardIdMap: map[0:[17406544375239214575]] err: 2023-10-21T04:33:00.212+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17434304260397056806, partnShardIdMap: map[0:[17406544375239214575]], err: 2023-10-21T04:33:00.212+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:00.212+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17434304260397056806 2023-10-21T04:33:00.212+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:00.213+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:33:00.213+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:00.213+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:00.220+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.541856ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.231+05:30 [Info] switched currmeta from 240 -> 244 force true 2023-10-21T04:33:00.247+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:00.247+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:00.213+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:00.256+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 79.26µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.256+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17434304260397056806 Partitions [0] 2023-10-21T04:33:00.257+05:30 [Info] StorageMgr::openSnapshot IndexInst:17434304260397056806 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:33:00.266+05:30 [Info] switched currmeta from 244 -> 244 force true 2023-10-21T04:33:00.308+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 245782287380001 25 21887 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:33:00.308+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 1949031200982 23 21887 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:00.310+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 205201940883916 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.311+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 52522903947059 28 21887 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:00.314+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 93079113162334 18 21887 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:00.321+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 177815184127237 17 21887 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:00.321+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 167580527121081 23 21887 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:00.321+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 244791485011286 23 21887 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:00.323+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 160381752747972 18 21887 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:00.323+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 54236768606397 20 21887 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:00.325+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 58522600746999 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.326+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 142387448675642 30 21887 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:33:00.331+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 90115353725987 21 21887 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:00.331+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 134409829206048 26 21887 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:00.331+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 161230548567164 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.333+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 85302001459467 21 21887 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:00.334+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 123957315902602 28 21887 172.31.5.112:9000. HWT [27-28,28]. 2023-10-21T04:33:00.338+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 150690956328758 26 21887 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:00.338+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 163460438181870 34 21887 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:33:00.338+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 213108110408291 23 21887 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:00.341+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 151273673850546 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.343+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 147946970349859 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.346+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 258671969827141 19 21887 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:00.348+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 53829314402679 32 21887 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:33:00.349+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 19989312661772 15 21887 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:33:00.354+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 68847799765163 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.354+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 114823206239184 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.355+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 232028101514735 19 21887 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:00.358+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 25510109770064 28 21887 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:00.358+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 8305763409275 21 21887 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:33:00.361+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 124811039198764 18 21887 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:00.361+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 207489375637709 12 21887 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:33:00.364+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 33310272161722 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.369+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 100746255343609 19 21887 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:00.369+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 107407811917992 20 21887 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:00.374+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 228857796432725 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.374+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 117298435979365 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.378+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 47156288300713 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.381+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 158357249947806 21 21887 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:00.381+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 203503670520010 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.383+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 104136033125373 29 21887 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:00.386+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 124310544527292 21 21887 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:00.386+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 236904976023311 22 21887 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:00.391+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 246959094981553 34 21887 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:33:00.391+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 196269910725961 26 21887 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:00.393+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 276463747117997 23 21887 172.31.5.112:9000. HWT [22-23,23]. 2023-10-21T04:33:00.396+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 172365063201867 21 21887 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:00.396+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 65885584526743 20 21887 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:00.399+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 212884822297038 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.399+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 79077905592013 17 21887 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:00.403+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 213334533059126 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.403+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 246582249071123 20 21887 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:00.408+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 254944616021293 24 21887 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:00.408+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 142324128597367 22 21887 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:00.409+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 55145461136105 16 21887 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:33:00.411+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 219528837936411 25 21887 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:00.413+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 194568213794435 29 21887 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:00.415+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 80145536651832 17 21887 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:00.419+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 106620692996001 23 21887 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:00.419+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 25511316571199 16 21887 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:33:00.422+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 279733370822238 23 21887 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:00.424+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 36300629322478 32 21887 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:33:00.424+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 223881084081370 17 21887 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:00.428+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 25657368013413 17 21887 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:00.430+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 InstanceIds [15864358084358521096 4083560586502645495] 2023-10-21T04:33:00.430+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 21887 2023-10-21T04:33:00.430+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 21887 2023-10-21T04:33:00.430+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:33:00.430+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:33:00.430+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:33:00.430+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:33:00.430+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 21887 2023-10-21T04:33:00.430+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 21887 STREAM_ACTIVE 2023-10-21T04:33:00.430+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:33:00.441+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2670343739772083055:[0]]. New rollbacktime: 1697842885353317281, Old rollbacktime: 0 2023-10-21T04:33:00.441+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2670343739772083055:[0] 8965923595543532942:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842885353317281, Old rollbacktime: 0 2023-10-21T04:33:00.458+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:33:00.525+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:00.525+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00687b080 2023-10-21T04:33:00.526+05:30 [Info] handleUpdateIndexRState: Index instance 4083560586502645495 rstate moved to ACTIVE 2023-10-21T04:33:00.526+05:30 [Info] Indexer::handleMergePartition Source 11304674148443950662 Target 15864358084358521096 2023-10-21T04:33:00.526+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0047556e0 2023-10-21T04:33:00.526+05:30 [Info] MergePartition: sent async request to update index instance 15864358084358521096 rstate moved to ACTIVE 2023-10-21T04:33:00.549+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 22.85845ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.551+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 3464125101225762400. Retrying... 2023-10-21T04:33:00.551+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[4083560586502645495:true 15864358084358521096:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:33:00.561+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.673262ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.561+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 11304674148443950662, realInst: 15864358084358521096 2023-10-21T04:33:00.561+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:33:00.561+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:33:00.564+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:00.564+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:00.568+05:30 [Info] switched currmeta from 244 -> 246 force true 2023-10-21T04:33:00.588+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:00.588+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:00.607+05:30 [Info] switched currmeta from 246 -> 246 force true 2023-10-21T04:33:00.609+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3464125101225762400,realInstId: 0, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:33:00.609+05:30 [Info] Indexer::handleRecoverIndex InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:33:00.613+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 4.011282ms 2023-10-21T04:33:00.613+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 4.251342ms 2023-10-21T04:33:00.613+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3464125101225762400 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[16787445029868804812 2925063509317026816]] 2023-10-21T04:33:00.613+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3464125101225762400, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:33:00.614+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 52.618788ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.614+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3464125101225762400 PartitionId: 0 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:33:00.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:00.615+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:33:00.615+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 360006 count: 6 metaSz: 2481 rp [1] sn: 360007 count: 5 metaSz: 2482 2023-10-21T04:33:00.616+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: doInit: data UsedSpace 73305 recovery UsedSpace 91456 2023-10-21T04:33:00.619+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:00.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:33:00.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:00.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:00.634+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:33:00.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex started 2023-10-21T04:33:00.635+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:33:00.635+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:00.635+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:00.654+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Warmup took 39.163246ms 2023-10-21T04:33:00.655+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3464125101225762400 partitionId 0 WriterThreads 4 2023-10-21T04:33:00.656+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3464125101225762400 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index Index: 3464125101225762400 Partition: 0 2023-10-21T04:33:00.656+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3464125101225762400, shardIdMap: map[0:[16787445029868804812]] err: 2023-10-21T04:33:00.657+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3464125101225762400, partnShardIdMap: map[0:[16787445029868804812]], err: 2023-10-21T04:33:00.657+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:33:00.658+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:00.659+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3464125101225762400 2023-10-21T04:33:00.659+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:00.660+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:33:00.660+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:00.660+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:00.661+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:00.661+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3464125101225762400 Partitions [0] 2023-10-21T04:33:00.664+05:30 [Info] StorageMgr::openSnapshot IndexInst:3464125101225762400 Partition:0 Attempting to open snapshot (SnapshotInfo: count:5 committed:false) 2023-10-21T04:33:00.687+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:00.687+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:00.689+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 30.493088ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:00.690+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 89.337µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:33:00.712+05:30 [Info] switched currmeta from 246 -> 248 force true 2023-10-21T04:33:00.732+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:00.732+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:00.739+05:30 [Info] switched currmeta from 248 -> 248 force true 2023-10-21T04:33:00.940+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:33:00.953+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:33:00.955+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:33:00.956+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4083560586502645495:[0] 15864358084358521096:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697842885760483465, Old rollbacktime: 0 2023-10-21T04:33:01.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9642629151259231, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:33:01.211+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[17434304260397056806:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:33:01.211+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [3381524789655566900] belonging to collectionId: 9, ttid: ShardTokena2:b2:c5:e2:b2:69:50:c0. Initiating build 2023-10-21T04:33:01.225+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [17434304260397056806] &{2} 2023-10-21T04:33:01.225+05:30 [Info] Indexer::handleBuildRecoveredIndexes [17434304260397056806] 2023-10-21T04:33:01.225+05:30 [Info] bulkUpdateRState: Index instance 17434304260397056806 rstate moved to PENDING 2023-10-21T04:33:01.225+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [17434304260397056806] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:33:01.225+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:01.225+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:01.225+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:01.226+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:33:01.226+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.227+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:01.227+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 43684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:01.227+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:33:01.227+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:33:01.227+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:33:01.227+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 43684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:01.227+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:33:01.227+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.227+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:01.227+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_system:_query reset bucket state to NIL 2023-10-21T04:33:01.227+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:33:01.227+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId 43685 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:33:01.227+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.227+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:01.227+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:33:01.227+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:2] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:33:01.227+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17434304260397056806 stream INIT_STREAM keyspaceId bucket_%2:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:33:01.227+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:33:01.228+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=18,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 43685 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:33:01.228+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:33:01.228+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:33:01.233+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:01.233+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 8.372051ms 2023-10-21T04:33:01.233+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 8.411122ms 2023-10-21T04:33:01.233+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [17434304260397056806] 2023-10-21T04:33:01.233+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 21.949692ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:01.237+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[17434304260397056806:true] 2023-10-21T04:33:01.238+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_system:_query Check pending stream merge. 2023-10-21T04:33:01.240+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.563322ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:01.249+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:01.279+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:01.281+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:01.290+05:30 [Info] switched currmeta from 248 -> 250 force true 2023-10-21T04:33:01.310+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:01.311+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:01.325+05:30 [Info] switched currmeta from 250 -> 251 force true 2023-10-21T04:33:01.354+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:01.354+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:01.378+05:30 [Info] switched currmeta from 251 -> 253 force true 2023-10-21T04:33:01.430+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:33:01.432+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:33:01.456+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:01.456+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:01.457+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 212967951483349 28 36451 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:01.457+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 162049908746825 25 36451 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:33:01.457+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 274143649547516 29 36451 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:33:01.458+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 206082810619393 24 36451 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:01.458+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 143687065777286 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.461+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 116493903186721 22 36451 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:01.464+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 263088038522959 18 36451 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:01.467+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 136568163328749 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.468+05:30 [Info] switched currmeta from 253 -> 254 force true 2023-10-21T04:33:01.470+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 261951617336488 35 36451 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:33:01.480+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 58733680850130 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.480+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 188740876924377 22 36451 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:01.482+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 173457131919400 12 36451 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:33:01.487+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 245247859472006 18 36451 172.31.5.112:9000. HWT [17-18,18]. 2023-10-21T04:33:01.489+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:01.489+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:01.496+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 120349894144442 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.496+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 206088069380492 13 36451 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:33:01.497+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 34510046292135 16 36451 172.31.5.112:9000. HWT [16-16,16]. 2023-10-21T04:33:01.497+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 274158831830257 20 36451 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:01.497+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 277338544220217 21 36451 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:01.497+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 35716737784090 24 36451 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:01.499+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 191500187207978 21 36451 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:01.499+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 250426459207023 28 36451 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:01.499+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 125696667341112 27 36451 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:01.499+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 127746572208200 19 36451 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:01.499+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 106231614376481 21 36451 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:01.503+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 118920785117980 21 36451 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:01.503+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 137256376714252 22 36451 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:01.503+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 78446412189949 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.507+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 5906787184229 20 36451 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:01.507+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 146375746634523 17 36451 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:01.507+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 112482982607415 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.508+05:30 [Info] switched currmeta from 254 -> 254 force true 2023-10-21T04:33:01.511+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 41101832875819 19 36451 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:01.521+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 241391494503904 28 36451 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:01.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 216238759475962 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 100008922352935 17 36451 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:01.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 169268048765249 19 36451 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:01.531+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 127702263378100 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 168446351781198 25 36451 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:01.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 149033089785397 28 36451 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:01.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 245900669851618 19 36451 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:01.533+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 121450709505223 25 36451 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:01.533+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 106297365959267 27 36451 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:01.533+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 88718752743571 25 36451 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:01.536+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 39045953044628 18 36451 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:01.538+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 217237947818243 20 36451 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:01.538+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 160399698726237 27 36451 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:01.541+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 110286285845649 31 36451 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:33:01.543+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 162870112426804 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.549+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 264001907139144 22 36451 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:01.549+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 132470209691538 26 36451 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:01.552+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 266604622117081 26 36451 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:01.552+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 63737850123332 19 36451 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:01.556+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 32274016651656 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.558+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 84581802233088 22 36451 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:01.560+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 225662991941187 26 36451 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:01.571+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 241607113601772 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.573+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 97677619676504 31 36451 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:33:01.573+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 272523968790649 20 36451 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:01.573+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 77598347829810 19 36451 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:01.573+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 136038310030103 28 36451 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:01.573+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 197957220074784 24 36451 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:01.579+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 161870047388121 25 36451 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:01.585+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 105415986286631 18 36451 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:01.585+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 72072509155168 23 36451 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:01.585+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 27479155597282 29 36451 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:01.590+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 InstanceIds [15864358084358521096 4083560586502645495 8965923595543532942 2670343739772083055] 2023-10-21T04:33:01.590+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 36451 2023-10-21T04:33:01.590+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 36451 2023-10-21T04:33:01.590+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:33:01.590+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:33:01.590+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM MTR In Progress. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:33:01.590+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:33:01.590+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 36451 2023-10-21T04:33:01.590+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 36451 STREAM_ACTIVE 2023-10-21T04:33:01.626+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3464125101225762400:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:33:01.626+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [11276338709617092948] belonging to collectionId: 9, ttid: ShardTokenb2:f8:fb:ca:53:2e:1c:c4. Initiating build 2023-10-21T04:33:01.639+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [3464125101225762400] &{2} 2023-10-21T04:33:01.639+05:30 [Info] Indexer::handleBuildRecoveredIndexes [3464125101225762400] 2023-10-21T04:33:01.639+05:30 [Info] bulkUpdateRState: Index instance 3464125101225762400 rstate moved to PENDING 2023-10-21T04:33:01.639+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [3464125101225762400] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:33:01.639+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:01.639+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:01.640+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:01.641+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:33:01.641+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.641+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:01.641+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.641+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:33:01.641+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 31407 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:01.642+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:33:01.642+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:33:01.642+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:33:01.642+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 31407 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:01.642+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_system:_query:{0xc003752f00 }] 2023-10-21T04:33:01.642+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.642+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:01.642+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.642+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:33:01.642+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:33:01.642+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:33:01.642+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 31408 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:33:01.642+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.642+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:01.642+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:01.642+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:33:01.642+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:33:01.642+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:2] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:33:01.642+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3464125101225762400 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:33:01.642+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:33:01.652+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:01.652+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:01.652+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:33:01.655+05:30 [Info] switched currmeta from 254 -> 255 force true 2023-10-21T04:33:01.642+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 31408 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:33:01.664+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_system:_query:{0xc003752f00 } bucket_1:_system:_query:{0xc007b87140 }] 2023-10-21T04:33:01.666+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:01.670+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 30.573934ms 2023-10-21T04:33:01.669+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:33:01.670+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [3464125101225762400] 2023-10-21T04:33:01.670+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 31.294502ms 2023-10-21T04:33:01.678+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[3464125101225762400:true] 2023-10-21T04:33:01.677+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 50.782015ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:01.684+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60440" doAuth() ... 2023-10-21T04:33:01.684+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60440" auth successful 2023-10-21T04:33:01.686+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:60440" +1 2023-10-21T04:33:01.687+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.541427ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:01.697+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:01.697+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:01.700+05:30 [Info] switched currmeta from 255 -> 256 force true 2023-10-21T04:33:01.701+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:01.875+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10402379389576076017:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:33:01.875+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1] 10402379389576076017:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:33:02.001+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 0 1949031200982 23 31408 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.003+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 1 245782287380001 25 31408 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:33:02.005+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 2 52522903947059 28 31408 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:02.008+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 4 93079113162334 18 31408 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:02.008+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 3 205201940883916 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.010+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 6 167580527121081 23 31408 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.010+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 5 244791485011286 23 31408 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.013+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 7 177815184127237 17 31408 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:02.016+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 9 134409829206048 26 31408 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:02.016+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 8 54236768606397 20 31408 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.018+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 10 160381752747972 18 31408 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:02.020+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 11 58522600746999 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.022+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_system:_query setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:33:02.023+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 12 142387448675642 30 31408 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:33:02.025+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 13 90115353725987 21 31408 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.026+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 14 161230548567164 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.027+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:33:02.027+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3464125101225762400 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:6 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.075µs SnapOpenDur 121.284µs 2023-10-21T04:33:02.027+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:33:02.028+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 15 85302001459467 21 31408 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.033+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 18 163460438181870 34 31408 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:33:02.033+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 17 213108110408291 23 31408 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.037+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 19 150690956328758 26 31408 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:02.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 22 258671969827141 19 31408 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 20 151273673850546 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 21 147946970349859 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.040+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 16 123957315902602 28 31408 172.31.5.112:9000. HWT [27-28,28]. 2023-10-21T04:33:02.040+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 SnapshotId 2 Created recovery point (took 12.889343ms) 2023-10-21T04:33:02.042+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 23 53829314402679 32 31408 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:33:02.042+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:33:02.044+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 24 19989312661772 15 31408 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:33:02.046+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 25 114823206239184 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 26 68847799765163 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.050+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 28 25510109770064 28 31408 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:02.051+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 27 232028101514735 19 31408 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.053+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 29 8305763409275 21 31408 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:33:02.054+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 30 124811039198764 18 31408 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:02.056+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 31 207489375637709 12 31408 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:33:02.058+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 32 33310272161722 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 33 107407811917992 20 31408 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.062+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9339740212826804, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:33:02.064+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 35 117298435979365 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.064+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 34 100746255343609 19 31408 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.067+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 37 203503670520010 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.067+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 36 228857796432725 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.068+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 38 47156288300713 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.071+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 40 104136033125373 29 31408 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:02.071+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 39 158357249947806 21 31408 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.074+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 41 236904976023311 22 31408 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:02.077+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 42 124310544527292 21 31408 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.080+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 44 246959094981553 34 31408 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:33:02.080+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 43 196269910725961 26 31408 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:02.082+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 45 276463747117997 23 31408 172.31.5.112:9000. HWT [22-23,23]. 2023-10-21T04:33:02.083+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 46 65885584526743 20 31408 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.085+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 47 172365063201867 21 31408 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.087+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 48 79077905592013 17 31408 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:02.089+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 49 212884822297038 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.091+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 50 246582249071123 20 31408 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.093+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 51 213334533059126 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.097+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 53 254944616021293 24 31408 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.097+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 52 142324128597367 22 31408 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:02.099+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 54 55145461136105 16 31408 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:33:02.101+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 55 219528837936411 25 31408 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.102+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 56 194568213794435 29 31408 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:02.104+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 57 80145536651832 17 31408 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:02.107+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 58 25511316571199 16 31408 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:33:02.109+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 60 279733370822238 23 31408 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.109+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 59 106620692996001 23 31408 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.112+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 61 223881084081370 17 31408 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:02.112+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 62 36300629322478 32 31408 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:33:02.113+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 63 25657368013413 17 31408 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:02.116+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query InstanceIds [3464125101225762400] 2023-10-21T04:33:02.116+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 31408 2023-10-21T04:33:02.116+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31408 2023-10-21T04:33:02.116+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:33:02.116+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:33:02.116+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 3464125101225762400 INDEX_STATE_CATCHUP 2023-10-21T04:33:02.116+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 3464125101225762400 true 2023-10-21T04:33:02.116+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 3464125101225762400 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query Session: 31408 BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] 2023-10-21T04:33:02.116+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_system:_query, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} , fetchKVSeq false 2023-10-21T04:33:02.117+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:33:02.117+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:33:02.117+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:33:02.117+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31408 STREAM_ACTIVE 2023-10-21T04:33:02.117+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 31408 2023-10-21T04:33:02.117+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_system:_query 31408 2023-10-21T04:33:02.117+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:02.117+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:02.117+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:02.117+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:02.118+05:30 [Info] send progress stats to clients 2023-10-21T04:33:02.118+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:33:02.118+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:33:02.118+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:3] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:33:02.118+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 31408 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:33:02.118+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=4,1=25,2=4,3=4,4=18,5=4,6=4,7=4,8=21,9=4,10=4,11=4,12=4,13=4,14=4,15=4,16=28,17=4,18=4,19=4,20=4,21=4,22=4,23=4,24=4,25=4,26=4,27=4,28=4,29=21,30=4,31=4,32=4,33=4,34=4,35=4,36=4,37=4,38=4,39=4,40=4,41=4,42=4,43=26,44=4,45=23,46=4,47=4,48=4,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=4,59=4,60=4,61=4,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:02.120+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:33:02.125+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 2023-10-21T04:33:02.125+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31408 2023-10-21T04:33:02.125+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31408 2023-10-21T04:33:02.125+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:33:02.125+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.331657ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:02.125+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:33:02.125+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 3464125101225762400 false 2023-10-21T04:33:02.125+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_system:_query 3464125101225762400 2023-10-21T04:33:02.125+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_system:_query. 2023-10-21T04:33:02.125+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 31408 2023-10-21T04:33:02.146+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:02.146+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:02.149+05:30 [Info] switched currmeta from 256 -> 257 force true 2023-10-21T04:33:02.220+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 10402379389576076017 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 3345094951175719474 RealInstId: 3345094951175719474 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 10601152754683910907 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_partitioned Progress: 100 InstId: 15864358084358521096 RealInstId: 15864358084358521096 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 4083560586502645495 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 3464125101225762400 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 2670343739772083055 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 8965923595543532942 RealInstId: 8965923595543532942 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 17434304260397056806 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 4238649781004957942 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 6043349427976659583 RealInstId: 6043349427976659583 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 2535881715450691825 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:33:02.221+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:33:02.237+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:33:02.266+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:02.435+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:33:02.459+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:02.459+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:02.463+05:30 [Info] switched currmeta from 257 -> 258 force true 2023-10-21T04:33:02.486+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:02.496+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:02.512+05:30 [Info] switched currmeta from 258 -> 259 force true 2023-10-21T04:33:02.591+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:33:02.591+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:33:02.706+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:02.724+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 1 162049908746825 25 43685 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:33:02.725+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 0 274143649547516 29 43685 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:33:02.728+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_system:_query setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:33:02.728+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:33:02.729+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17434304260397056806 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:6 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.432µs SnapOpenDur 111.636µs 2023-10-21T04:33:02.729+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:33:02.730+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 4 143687065777286 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.731+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 2 206082810619393 24 43685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.733+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 5 212967951483349 28 43685 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:02.733+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 3 120349894144442 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.736+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 7 78446412189949 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.736+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 6 116493903186721 22 43685 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:02.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 8 263088038522959 18 43685 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:02.742+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 9 136568163328749 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.742+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 2 Created recovery point (took 13.106259ms) 2023-10-21T04:33:02.743+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:33:02.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 10 261951617336488 35 43685 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:33:02.747+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 13 188740876924377 22 43685 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:02.747+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 12 58733680850130 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.747+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 11 127746572208200 19 43685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.752+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 14 173457131919400 12 43685 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:33:02.754+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 15 277338544220217 21 43685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.754+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 16 245247859472006 18 43685 172.31.5.112:9000. HWT [17-18,18]. 2023-10-21T04:33:02.761+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 19 250426459207023 28 43685 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:02.761+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 17 206088069380492 13 43685 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:33:02.773+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:02.773+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:02.781+05:30 [Info] switched currmeta from 259 -> 260 force true 2023-10-21T04:33:02.787+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 18 34510046292135 16 43685 172.31.5.112:9000. HWT [16-16,16]. 2023-10-21T04:33:02.795+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 22 35716737784090 24 43685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.797+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 23 191500187207978 21 43685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.798+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 24 106231614376481 21 43685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.798+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 20 274158831830257 20 43685 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.798+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 21 125696667341112 27 43685 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:02.801+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 25 118920785117980 21 43685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:02.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 29 112482982607415 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 27 5906787184229 20 43685 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 28 146375746634523 17 43685 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:02.810+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 26 137256376714252 22 43685 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:02.811+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 30 41101832875819 19 43685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 33 216238759475962 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 32 121450709505223 25 43685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 31 241391494503904 28 43685 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:02.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 34 168446351781198 25 43685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 35 127702263378100 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.822+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 37 169268048765249 19 43685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.822+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 36 100008922352935 17 43685 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:02.824+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 38 245900669851618 19 43685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 40 106297365959267 27 43685 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:02.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 39 149033089785397 28 43685 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:02.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 41 88718752743571 25 43685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.832+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 43 160399698726237 27 43685 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:02.832+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 42 39045953044628 18 43685 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:02.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 45 110286285845649 31 43685 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:33:02.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 44 217237947818243 20 43685 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.837+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 46 162870112426804 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.840+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 47 132470209691538 26 43685 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:02.844+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 48 264001907139144 22 43685 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:02.845+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 49 63737850123332 19 43685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 53 225662991941187 26 43685 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:02.852+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 51 32274016651656 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.854+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 50 266604622117081 26 43685 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:02.854+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 55 272523968790649 20 43685 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:02.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 52 84581802233088 22 43685 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:02.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 54 241607113601772 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 57 197957220074784 24 43685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:02.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 56 77598347829810 19 43685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:02.859+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 58 136038310030103 28 43685 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:02.860+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 61 105415986286631 18 43685 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:02.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 59 97677619676504 31 43685 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:33:02.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 60 72072509155168 23 43685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:02.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 62 27479155597282 29 43685 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:02.868+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 63 161870047388121 25 43685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:02.877+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_system:_query InstanceIds [3464125101225762400 17434304260397056806] 2023-10-21T04:33:02.878+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query Cid 9 SessionId 43685 2023-10-21T04:33:02.878+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43685 2023-10-21T04:33:02.878+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:33:02.878+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:33:02.878+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 17434304260397056806 INDEX_STATE_CATCHUP 2023-10-21T04:33:02.878+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 17434304260397056806 true 2023-10-21T04:33:02.878+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 17434304260397056806 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query Session: 43685 BuildTS: [0=29,1=25,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=18,17=13,18=16,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=27,44=20,45=31,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] 2023-10-21T04:33:02.878+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_system:_query, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 26 25 26} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 18 16 18} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 32 31 32} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} , fetchKVSeq false 2023-10-21T04:33:02.878+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:33:02.878+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43685 STREAM_ACTIVE 2023-10-21T04:33:02.878+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 43685 2023-10-21T04:33:02.878+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_system:_query 43685 2023-10-21T04:33:02.878+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:02.878+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:02.879+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:02.879+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:02.879+05:30 [Info] send progress stats to clients 2023-10-21T04:33:02.879+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:33:02.879+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:33:02.879+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_%2:_system:_query:1 bucket_1:_system:_query:1]] 2023-10-21T04:33:02.879+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 43685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:33:02.879+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=29,1=26,2=4,3=4,4=4,5=4,6=4,7=4,8=4,9=4,10=4,11=4,12=4,13=4,14=4,15=4,16=18,17=4,18=18,19=4,20=4,21=4,22=4,23=4,24=4,25=4,26=4,27=4,28=4,29=4,30=4,31=4,32=4,33=4,34=4,35=4,36=4,37=4,38=4,39=4,40=4,41=4,42=4,43=27,44=4,45=32,46=4,47=4,48=4,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=4,59=4,60=4,61=4,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:02.881+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:33:02.890+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 2023-10-21T04:33:02.890+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43685 2023-10-21T04:33:02.890+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43685 2023-10-21T04:33:02.890+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:33:02.890+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:33:02.890+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 17434304260397056806 false 2023-10-21T04:33:02.890+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_system:_query 17434304260397056806 2023-10-21T04:33:02.890+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:33:02.890+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 43685 2023-10-21T04:33:02.891+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.244749ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:02.908+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:02.909+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:02.913+05:30 [Info] switched currmeta from 260 -> 261 force true 2023-10-21T04:33:03.118+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:33:03.118+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_system:_query 2023-10-21T04:33:03.118+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 3464125101225762400 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId: 31408, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:33:03.118+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 3464125101225762400 INDEX_STATE_ACTIVE 2023-10-21T04:33:03.118+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_system:_query State Changed to INACTIVE 2023-10-21T04:33:03.118+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:33:03.118+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:33:03.118+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:33:03.118+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 31408 2023-10-21T04:33:03.118+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:03.118+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:03.119+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:33:03.119+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 31408 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:03.119+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3464125101225762400 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:33:03.119+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:33:03.119+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:33:03.119+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:33:03.119+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 31408 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:03.119+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:_system:_query:{0xc003752f00 }] 2023-10-21T04:33:03.119+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:03.119+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:03.119+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:03.119+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:33:03.119+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:03.119+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:03.119+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:03.119+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream INIT_STREAM Succceed 2023-10-21T04:33:03.119+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:33:03.119+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:03.120+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId 31408 2023-10-21T04:33:03.121+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:33:03.124+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 6 167580527121081 23 2023-10-21T04:33:03.124+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.124+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 7 177815184127237 17 2023-10-21T04:33:03.124+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.124+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 0 1949031200982 23 2023-10-21T04:33:03.124+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.125+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 1 245782287380001 25 2023-10-21T04:33:03.125+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.125+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 2 52522903947059 28 2023-10-21T04:33:03.125+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.125+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 3 205201940883916 24 2023-10-21T04:33:03.125+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.125+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 4 93079113162334 18 2023-10-21T04:33:03.125+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.125+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 5 244791485011286 23 2023-10-21T04:33:03.125+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.131+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 26 68847799765163 25 2023-10-21T04:33:03.131+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.131+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 18 163460438181870 34 2023-10-21T04:33:03.131+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 23 53829314402679 32 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 20 151273673850546 25 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 27 232028101514735 19 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 28 25510109770064 28 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 32 33310272161722 24 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 29 8305763409275 21 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 9 134409829206048 26 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 10 160381752747972 18 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 31 207489375637709 12 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 12 142387448675642 30 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 14 161230548567164 25 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 21 147946970349859 24 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 8 54236768606397 21 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 22 258671969827141 19 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 24 19989312661772 15 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 11 58522600746999 24 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.132+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 19 150690956328758 26 2023-10-21T04:33:03.132+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.133+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 13 90115353725987 21 2023-10-21T04:33:03.133+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.133+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 25 114823206239184 25 2023-10-21T04:33:03.133+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.133+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 30 124811039198764 18 2023-10-21T04:33:03.133+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.133+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 17 213108110408291 23 2023-10-21T04:33:03.133+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.133+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 16 123957315902602 28 2023-10-21T04:33:03.133+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.136+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 34 100746255343609 19 2023-10-21T04:33:03.136+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.136+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 37 203503670520010 24 2023-10-21T04:33:03.136+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.136+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 33 107407811917992 20 2023-10-21T04:33:03.136+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.136+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 35 117298435979365 24 2023-10-21T04:33:03.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.137+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 36 228857796432725 25 2023-10-21T04:33:03.137+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.137+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 15 85302001459467 21 2023-10-21T04:33:03.137+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.329044ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:03.138+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.144+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:03.145+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:03.145+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 41 236904976023311 22 2023-10-21T04:33:03.145+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.145+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 42 124310544527292 21 2023-10-21T04:33:03.145+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.145+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 43 196269910725961 26 2023-10-21T04:33:03.145+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.145+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 44 246959094981553 34 2023-10-21T04:33:03.145+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.145+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 38 47156288300713 24 2023-10-21T04:33:03.145+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.145+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 39 158357249947806 21 2023-10-21T04:33:03.145+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.145+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 40 104136033125373 29 2023-10-21T04:33:03.145+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.154+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:33:03.154+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31408 2023-10-21T04:33:03.154+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 31408 2023-10-21T04:33:03.155+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 46 65885584526743 20 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 52 142324128597367 22 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 56 194568213794435 29 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 63 25657368013413 17 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 50 246582249071123 20 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 51 213334533059126 25 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 62 36300629322478 32 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 55 219528837936411 25 2023-10-21T04:33:03.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.155+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 57 80145536651832 17 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 58 25511316571199 16 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 59 106620692996001 23 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 60 279733370822238 23 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 45 276463747117997 23 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 48 79077905592013 17 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 49 212884822297038 25 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 61 223881084081370 17 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 47 172365063201867 21 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 53 254944616021293 24 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.156+05:30 [Info] TK StreamEnd INIT_STREAM bucket_1:_system:_query 54 55145461136105 16 2023-10-21T04:33:03.156+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1:_system:_query Stream INIT_STREAM. Ignored. 2023-10-21T04:33:03.157+05:30 [Info] switched currmeta from 261 -> 262 force true 2023-10-21T04:33:03.195+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 6] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/1f784fc0142ee312f0d92d98bd188c9a Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]} {Rev:[0 0 0 0 0 0 0 2] ID:rebalance/1f784fc0142ee312f0d92d98bd188c9a Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:1f784fc0142ee312f0d92d98bd188c9a]}]} 2023-10-21T04:33:03.195+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:33:03.197+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 7] 2023-10-21T04:33:03.197+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 7] 2023-10-21T04:33:03.271+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:03.271+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:03.276+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:03.276+05:30 [Info] switched currmeta from 262 -> 263 force true 2023-10-21T04:33:03.282+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is enabled 2023-10-21T04:33:03.710+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:03.710+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc008344060 2023-10-21T04:33:03.711+05:30 [Info] handleUpdateIndexRState: Index instance 3464125101225762400 rstate moved to ACTIVE 2023-10-21T04:33:03.721+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.549838ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:03.721+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[3464125101225762400:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:33:03.721+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardTokenb2:f8:fb:ca:53:2e:1c:c4, elapsed(sec): 9.665648335 2023-10-21T04:33:03.727+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:33:03.729+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 2023-10-21T04:33:03.773+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:03.773+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:03.778+05:30 [Info] TestCode::TestActionAtTag: currTag: 7, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:03.779+05:30 [Info] switched currmeta from 263 -> 264 force true 2023-10-21T04:33:03.816+05:30 [Info] memstats {"Alloc":225052136, "TotalAlloc":332230104, "Sys":252598584, "Lookups":0, "Mallocs":2030668,"Frees":1130079, "HeapAlloc":225052136, "HeapSys":230522880, "HeapIdle":2449408, "HeapInuse":228073472,"HeapReleased":2449408, "HeapObjects":900589,"MSpanInuse":1707360, "MSpanSys": 1713600, "StackInuse": 4358144,"GCSys":13425328, "LastGC":1697842889686922904,"PauseTotalNs":4212178, "PauseNs":[124844, 1651431], "NumGC":7} 2023-10-21T04:33:03.838+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:03.838+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:03.843+05:30 [Info] switched currmeta from 264 -> 265 force true 2023-10-21T04:33:03.878+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:33:03.878+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_system:_query 2023-10-21T04:33:03.878+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 17434304260397056806 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId: 43685, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 26 25 26} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 18 16 18} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 32 31 32} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:33:03.878+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 17434304260397056806 INDEX_STATE_ACTIVE 2023-10-21T04:33:03.879+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query State Changed to INACTIVE 2023-10-21T04:33:03.879+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:33:03.879+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:33:03.879+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:33:03.879+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 43685 2023-10-21T04:33:03.879+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:03.879+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842885353317281 bucket_1:1697842885760483465] 2023-10-21T04:33:03.879+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:33:03.879+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 43685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:03.879+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17434304260397056806 from stream INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:33:03.879+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:33:03.879+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:33:03.879+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:33:03.880+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 43685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:03.880+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:33:03.880+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query total received key-versions: map[0:0 1:1 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:2 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:1 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:33:03.880+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query latest sequence numbers: map[0:0 1:26 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:18 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:32 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:33:03.880+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:33:03.882+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:60440: use of closed network connection 2023-10-21T04:33:03.882+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:60440" exit: read tcp4 127.0.0.1:9121->127.0.0.1:60440: use of closed network connection 2023-10-21T04:33:03.882+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60440" closed ! 2023-10-21T04:33:03.882+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:33:03.882+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:60440" already gone 2023-10-21T04:33:03.882+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:03.883+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:03.883+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:03.883+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:03.883+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:33:03.883+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:03.883+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId 43685 2023-10-21T04:33:03.883+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:33:03.907+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_system:_query]. Err genServer.closed 2023-10-21T04:33:03.907+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:33:03.907+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Treating genServer.closed As Success 2023-10-21T04:33:03.907+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43685 2023-10-21T04:33:03.907+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 43685 2023-10-21T04:33:03.907+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:33:03.931+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 47.807043ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:03.963+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:03.965+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:03.970+05:30 [Info] switched currmeta from 265 -> 266 force true 2023-10-21T04:33:04.000+05:30 [Info] connected with 4 indexers 2023-10-21T04:33:04.001+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:33:04.047+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:04.052+05:30 [Info] num concurrent scans {0} 2023-10-21T04:33:04.052+05:30 [Info] average scan response {0 ms} 2023-10-21T04:33:04.053+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:04.053+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:04.055+05:30 [Info] switched currmeta from 266 -> 266 force true 2023-10-21T04:33:04.097+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:33:04.110+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:33:04.117+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:33:04.148+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:33:04.155+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:33:04.235+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:04.237+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:04.237+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken7f:de:a2:ba:54:17:8e:fe MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 Shards: [7434639758576776593 1249143295319901191] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8248238408530887194 RealInstId: 6225851069678382014 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4238649781004957942 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2332250308932951681 RealInstId: 6043349427976659583 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2535881715450691825 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:33:04.255+05:30 [Info] switched currmeta from 266 -> 267 force true 2023-10-21T04:33:04.266+05:30 [Info] TestCode::TestActionAtTag: currTag: 7, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:04.312+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:04.312+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00af5a060 2023-10-21T04:33:04.312+05:30 [Info] handleUpdateIndexRState: Index instance 17434304260397056806 rstate moved to ACTIVE 2023-10-21T04:33:04.329+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.297713ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:04.329+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[17434304260397056806:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:33:04.329+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardTokena2:b2:c5:e2:b2:69:50:c0, elapsed(sec): 10.116871931 2023-10-21T04:33:04.335+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardTokena2:b2:c5:e2:b2:69:50:c0 2023-10-21T04:33:04.341+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena2:b2:c5:e2:b2:69:50:c0 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2670343739772083055 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16711923683289406198 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3283688418228810121 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:33:04.368+05:30 [Info] TestCode::TestActionAtTag: currTag: 7, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:33:04.399+05:30 [Info] TestCode::TestActionAtTag: currTag: 8, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 panic: TestCode::TestActionAtTag - Inducing artificial panic at tag: 8 as wished goroutine 5434 [running]: github.com/couchbase/indexing/secondary/testcode.TestActionAtTag(0xc00c8f4240?, 0x8) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/testcode/test_action_ci.go:45 +0x5e5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferTokenAsMaster(0xc009968480, {0xc004d9a814, 0x21}, 0xc00d4a0800) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:856 +0x673 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferToken(0xc009968480, {0xc004d9a814, 0x21}, 0xc00d4a0800) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:724 +0x2a5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTokens(0xc009968480, {{0xc004d9a800, 0x35}, {0xc00d8c0000, 0x144c, 0x144c}, {0x176e520, 0xc00ccf90b0}, 0x0}) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:644 +0x134 github.com/couchbase/cbauth/metakv.RunObserveChildren.func1({{0xc004d9a800, 0x35}, {0xc00d8c0000, 0x144c, 0x144c}, {0xc00ce38180, 0x77, 0x78}, 0x0}) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:400 +0x149 github.com/couchbase/cbauth/metakv.doRunObserveChildren(0x1a8c321?, {0x1aa523a, 0x14}, 0xc00b5dbf00, 0xc007154660) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:316 +0x4eb github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(...) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:268 github.com/couchbase/cbauth/metakv.RunObserveChildren({0x1aa523a?, 0xc0062c0004?}, 0x1aee743?, 0x2e?) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:398 +0x58 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).observeRebalance(0xc009968480) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:235 +0x111 created by github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).doRebalance /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:2689 +0x14a 2023-10-21T04:33:07.001+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=18194302e3f10df59bd076e1c4142d0b -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:33:07.001+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:33:07.002+05:30 [Info] Setting ipv6=false 2023-10-21T04:33:07.034+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:33:07.063+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:33:07.069+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:33:07.069+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:33:07.069+05:30 [Info] Setting log level to Info 2023-10-21T04:33:07.069+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:33:07.069+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:33:07.070+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:33:07.070+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:33:07.105+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:33:07 audit: created new audit service 2023-10-21T04:33:07.296+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:33:07.301+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:33:07.301+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:33:07.302+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:33:07.302+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:33:07.302+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:33:07.420+05:30 [Info] security context initialized 2023-10-21T04:33:07.421+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:33:07.421+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:33:07.421+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:33:07.421+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:33:07.421+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:33:07.430+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:33:07.434+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:33:07.437+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:33:07.440+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:33:07.434+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:33:07.442+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:33:07.471+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:33:07.471+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:33:07.477+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:33:07.481+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:33:07.482+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:33:07.482+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:33:07.482+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:07.523+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:33:07.524+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:33:07.524+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:33:07.528+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:33:07.528+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:33:07.532+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:33:07.533+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:33:07.533+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:33:07.534+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:33:07.535+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:33:07.536+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 172 us 2023-10-21T04:33:07.538+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:33:07.556+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:33:07.557+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:33:07.574+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:33:07.574+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:33:07.575+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:33:07.575+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:33:07.575+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:33:07.575+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:33:07.575+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:33:07.575+05:30 [Info] Indexer Id 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:33:07.579+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:33:07.581+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:33:07.587+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 201] updating cluster rev epoch 0 -> 1 2023-10-21T04:33:07.587+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 201] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:33:07.594+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 216] bucketNames has been updated (was map[], now map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43]) 2023-10-21T04:33:07.594+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 217] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:33:07.594+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 218] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:33:07.599+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to false 2023-10-21T04:33:07.600+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:33:07.600+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:33:07.600+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:33:07.603+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:33:07.604+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:33:07.604+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:33:07.604+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:33:07.606+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning true RebalanceToken &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:33:07.607+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:33:07.607+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:33:07.608+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:33:07.609+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:33:07.612+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:59992 2023-10-21T04:33:07.629+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:33:07.630+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:33:07.633+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:33:07.635+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:33:07.635+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:33:07.655+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 InstanceId: 8732144615635911668 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_READY Stream: NIL_STREAM RState: RebalActive Version: 1 ReplicaId: 0 InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_READY Stream: NIL_STREAM RState: RebalActive Version: 1 ReplicaId: 0 InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 , elapsed: 45.690118ms 2023-10-21T04:33:07.655+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:33:07.656+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:33:07.656+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:33:07.656+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager true &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:33:07.656+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:33:07.656+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:33:07.656+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:33:07.657+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:33:07.693+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:33:07.693+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:33:07.697+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:33:07.697+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:false Messages:[]} 2023-10-21T04:33:07.698+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:33:07.698+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:33:07.721+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err genServer.closed 2023-10-21T04:33:07.721+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:33:07.721+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating genServer.closed As Success 2023-10-21T04:33:07.721+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:33:07.722+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:33:07.722+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:33:07.722+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:33:07.722+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:07.725+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:33:07.725+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:33:07.727+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:07.728+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:33:07.730+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:33:07.731+05:30 [Info] : Recover shard shard10170029854712240877 from metadata completed. 2023-10-21T04:33:07.733+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:33:07.733+05:30 [Info] : Recover shard shard16787445029868804812 from metadata completed. 2023-10-21T04:33:07.733+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:33:07.734+05:30 [Info] : Recover shard shard17406544375239214575 from metadata completed. 2023-10-21T04:33:07.734+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:33:07.734+05:30 [Info] : Recover shard shard2925063509317026816 from metadata completed. 2023-10-21T04:33:07.735+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:33:07.735+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:33:07.736+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner initialized 2023-10-21T04:33:07.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:07.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:07.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner initialized 2023-10-21T04:33:07.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:07.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:07.738+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCleaner initialized for recovery 2023-10-21T04:33:07.738+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:07.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery 2023-10-21T04:33:07.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:07.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true] 2023-10-21T04:33:07.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1196032] Data log Head [0] Tail [1282048] 2023-10-21T04:33:07.738+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1196032] 2023-10-21T04:33:07.739+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCleaner initialized for recovery 2023-10-21T04:33:07.739+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:07.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery 2023-10-21T04:33:07.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:07.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true] 2023-10-21T04:33:07.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1355776] Data log Head [0] Tail [1421312] 2023-10-21T04:33:07.740+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1355776] 2023-10-21T04:33:07.741+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.742+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:33:07.742+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:33:07.742+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.743+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:33:07.743+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.744+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:33:07.744+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.744+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:33:07.744+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.744+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:33:07.744+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.744+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:33:07.744+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.744+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:33:07.744+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.744+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:33:07.745+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.745+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:33:07.743+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.746+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:33:07.746+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.746+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:33:07.746+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.748+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:33:07.748+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.748+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:33:07.748+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:33:07.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:33:07.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:33:07.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:33:07.752+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.752+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:33:07.754+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.754+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.754+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.754+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:07.754+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.754+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:07.754+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.754+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:07.755+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.755+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:07.755+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.755+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:07.755+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.755+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:07.755+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.755+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.755+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : replayOffset: 1269760 (RP:1269760) (CP:991232) 2023-10-21T04:33:07.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1269760] took [17.475796ms] 2023-10-21T04:33:07.755+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.755+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.755+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9501, memSzIndex 76, numRecords 127 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1056160, resident 1, memSz 66636, memSzIndex 1023, numRecords 1000 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.756+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053288, resident 1, memSz 10500, memSzIndex 77, numRecords 136 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: request quota after header replay 1053364, resident 1, memSz 9269, memSzIndex 92, numRecords 125 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 36359957 controller mem 4378 2023-10-21T04:33:07.757+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1282048] replayOffset [1269760] 2023-10-21T04:33:07.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [19.173456ms] 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.757+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true]. Built [17] plasmas, took [20.143466ms] 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:07.758+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 24094 recovery UsedSpace 4118 2023-10-21T04:33:07.748+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.758+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:33:07.760+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.760+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.760+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.760+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:07.760+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.760+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:07.760+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.760+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:07.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.761+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:07.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.761+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:07.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.761+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:07.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.761+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.761+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : replayOffset: 1413120 (RP:1400832) (CP:1413120) 2023-10-21T04:33:07.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1413120] took [21.769575ms] 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12433, memSzIndex 0, numRecords 123 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13717, memSzIndex 0, numRecords 136 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 10752, memSzIndex 0, numRecords 106 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.761+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12809, memSzIndex 0, numRecords 127 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12643, memSzIndex 0, numRecords 125 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 12321, memSzIndex 0, numRecords 122 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057276, resident 1, memSz 13698, memSzIndex 0, numRecords 136 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1056344, resident 1, memSz 75000, memSzIndex 297, numRecords 1000 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1055176, resident 0.11, memSz 347, memSzIndex 0, numRecords 9 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: request quota after header replay 1057426, resident 1, memSz 12641, memSzIndex 0, numRecords 125 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44754573 controller mem 5108 2023-10-21T04:33:07.762+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1421312] replayOffset [1413120] 2023-10-21T04:33:07.762+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [22.620952ms] 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.762+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.763+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.764+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.764+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.764+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:07.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true]. Built [18] plasmas, took [24.107659ms] 2023-10-21T04:33:07.764+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:07.764+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:07.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:07.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:07.774+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:33:07.774+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:33:07.774+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.775+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:07.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:07.775+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:33:07.775+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:33:07.776+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:07.776+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:07.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.776+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:07.777+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:07.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:07.777+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.777+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:07.777+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:07.777+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:33:07.777+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:07.777+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:07.777+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:07.777+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:07.791+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:07.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:07.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:33:07.792+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:07.792+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:07.792+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:07.792+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:07.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:07.792+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.792+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:07.875+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:07.875+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:33:07.875+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:07.876+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:07.876+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:07.876+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:07.882+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:07.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:07.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:33:07.882+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:07.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:33:07.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:33:07.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:33:07.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:33:07.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:33:07.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:33:07.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:07.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.897+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:07.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:07.900+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.900+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:07.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:07.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:07.902+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.902+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:07.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:33:07.903+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.903+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:07.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:07.906+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.906+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:07.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:33:07.906+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Warmup took 178.689432ms 2023-10-21T04:33:07.906+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 6 WriterThreads 1 2023-10-21T04:33:07.907+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index Index: 10032435514095102923 Partition: 6 2023-10-21T04:33:07.907+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:07.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:07.907+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:07.907+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:33:07.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:07.907+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:07.908+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:07.908+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.908+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:07.915+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:07.915+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:33:07.915+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:33:07.915+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:07.915+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.915+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:07.915+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:07.915+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:07.915+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:07.920+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:07.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:07.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:33:07.920+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:07.920+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:07.920+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:07.920+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:07.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:07.920+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.920+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:07.920+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:07.921+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:33:07.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:07.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:07.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:07.921+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:07.925+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:07.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:07.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:33:07.926+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:07.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:33:07.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:33:07.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:33:07.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:33:07.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:33:07.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:33:07.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:07.936+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.937+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:07.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:07.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:07.939+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:07.939+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:07.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:07.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:07.945+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.945+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:07.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:33:07.945+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.945+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:07.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:07.948+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.948+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:07.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:33:07.948+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.948+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Warmup took 41.723752ms 2023-10-21T04:33:07.949+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.949+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 7 WriterThreads 1 2023-10-21T04:33:07.949+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index Index: 10032435514095102923 Partition: 7 2023-10-21T04:33:07.949+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:07.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:07.950+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:07.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:07.950+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:07.950+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:33:07.950+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:07.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:07.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:07.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:07.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:07.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:33:07.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:07.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:07.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:07.961+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:07.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:33:07.961+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:33:07.961+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:07.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:07.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:07.962+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:07.962+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:07.962+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:07.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:07.962+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:07.962+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:07.962+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:07.962+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:07.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:33:07.962+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:07.962+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:07.962+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:07.962+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:07.976+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:07.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:07.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:33:07.976+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:07.976+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:07.976+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:07.976+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:07.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:07.976+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:07.976+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:07.976+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:07.976+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:07.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:33:07.976+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:07.976+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:07.977+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:07.977+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:07.994+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:07.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:07.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:33:07.994+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:07.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:33:07.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:33:07.997+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 53] will report aggregate recorder stats every 5m0s 2023-10-21T04:33:08.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:33:08.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:33:08.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:33:08.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:33:08.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.012+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.012+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.026+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.026+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:08.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.029+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:08.029+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:08.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:33:08.029+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.029+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:08.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.033+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:08.033+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:08.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:33:08.033+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.033+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.033+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Warmup took 83.741808ms 2023-10-21T04:33:08.033+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 8 WriterThreads 1 2023-10-21T04:33:08.033+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index Index: 10032435514095102923 Partition: 8 2023-10-21T04:33:08.033+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.034+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.034+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:33:08.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.034+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:08.038+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.039+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.039+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:33:08.039+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:33:08.039+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:08.039+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:08.039+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:08.039+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:08.039+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:08.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.039+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:08.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:08.047+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:08.047+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:08.047+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:08.047+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:08.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.047+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.052+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:08.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:33:08.052+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:08.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:33:08.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:33:08.054+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:33:08.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:33:08.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:33:08.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:33:08.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.061+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.061+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.065+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.065+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.065+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:08.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.067+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:08.067+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:08.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:33:08.067+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.067+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:08.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.071+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:08.071+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:08.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:33:08.071+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Warmup took 37.577127ms 2023-10-21T04:33:08.072+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 1 WriterThreads 1 2023-10-21T04:33:08.072+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index Index: 10032435514095102923 Partition: 1 2023-10-21T04:33:08.072+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.072+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:33:08.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.073+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.073+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:08.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:08.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.079+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:08.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:33:08.079+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.079+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:08.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.080+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:08.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:33:08.080+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:33:08.080+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:08.080+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:08.080+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:08.080+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:08.080+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:08.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.080+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:08.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:08.085+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:08.085+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:08.085+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:08.085+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:08.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.085+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.090+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:08.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:33:08.090+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:08.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:33:08.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:33:08.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:33:08.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:33:08.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:33:08.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:33:08.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.107+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.107+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.110+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.111+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:08.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.115+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:08.115+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:08.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:33:08.115+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.115+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:08.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.122+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:08.122+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:08.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:33:08.122+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Warmup took 49.913145ms 2023-10-21T04:33:08.122+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.122+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.122+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 2 WriterThreads 1 2023-10-21T04:33:08.122+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index Index: 10032435514095102923 Partition: 2 2023-10-21T04:33:08.122+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.122+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.122+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:33:08.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.123+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.123+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:08.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:08.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.138+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:08.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:33:08.138+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.138+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:08.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.140+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:08.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:33:08.141+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:33:08.141+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:08.141+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:08.141+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:08.141+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:08.141+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:08.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.141+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.146+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:08.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:33:08.146+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:08.146+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.146+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.146+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:08.146+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:08.147+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:08.147+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:08.147+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:08.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:33:08.147+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.147+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.147+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.147+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.160+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:08.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:33:08.161+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:08.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:33:08.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:33:08.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:33:08.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:33:08.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:33:08.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:33:08.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.174+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.177+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.177+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:08.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.183+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:08.183+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:08.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:33:08.183+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.183+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:08.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.183+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.183+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:08.183+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:08.184+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:33:08.184+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.184+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.184+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Warmup took 61.498921ms 2023-10-21T04:33:08.184+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 3 WriterThreads 1 2023-10-21T04:33:08.184+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index Index: 10032435514095102923 Partition: 3 2023-10-21T04:33:08.184+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.185+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.185+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:08.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.185+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.192+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:33:08.192+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.192+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.192+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:08.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:33:08.192+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.193+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:08.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:08.193+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:08.193+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:08.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:08.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.193+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:08.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:08.200+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:08.200+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:08.200+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:08.200+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:08.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.200+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.206+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:08.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:33:08.206+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:08.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:33:08.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:33:08.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:33:08.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:33:08.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:33:08.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:33:08.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.216+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.216+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.219+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.219+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:08.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.222+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:08.222+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:08.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:33:08.222+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.222+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:08.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.233+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:08.233+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:08.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:33:08.234+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Warmup took 49.201408ms 2023-10-21T04:33:08.234+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.234+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 4 WriterThreads 1 2023-10-21T04:33:08.234+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index Index: 10032435514095102923 Partition: 4 2023-10-21T04:33:08.234+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.234+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.234+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:08.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.235+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:33:08.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:08.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.246+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:08.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:33:08.246+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.246+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:08.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:08.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:33:08.249+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:33:08.249+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:08.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:08.249+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:08.249+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:08.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:08.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.249+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:08.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:08.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:08.261+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:08.261+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:08.261+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:08.261+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:08.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:08.261+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:08.275+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:08.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:08.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:33:08.275+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:08.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:33:08.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:33:08.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:33:08.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:33:08.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:33:08.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:33:08.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.290+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.291+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:08.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.296+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.296+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:08.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:08.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.303+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:08.303+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:08.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:33:08.303+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.303+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:08.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.307+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:08.307+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:08.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:33:08.307+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Warmup took 73.308336ms 2023-10-21T04:33:08.308+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 5 WriterThreads 1 2023-10-21T04:33:08.308+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index Index: 10032435514095102923 Partition: 5 2023-10-21T04:33:08.308+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10032435514095102923 2023-10-21T04:33:08.308+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:33:08.308+05:30 [Info] StorageMgr::openSnapshot IndexInst:10032435514095102923 Partition:3 No Snapshot Found. 2023-10-21T04:33:08.308+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Adding Nil Snapshot. 2023-10-21T04:33:08.308+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {10032435514095102923 {16337157693634799600 idx_partitioned_defer plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 false [`balance`] N1QL KEY [false] true true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 0 0 0 0xc005af2d50 [] 1 0 false plasma 0} map[1:{{1 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc0059e0fb0} 2:{{2 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005b0cb70} 3:{{3 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005bc7790} 4:{{4 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005ca4070} 5:{{5 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005ca4600} 6:{{6 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc004c48e00} 7:{{7 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc004f375e0} 8:{{8 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005a7d720}] 0xc00505f200 0xc005aeec00} 2023-10-21T04:33:08.309+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.309+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 240 2023-10-21T04:33:08.309+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 12 count: 125 metaSz: 2512 rp [1] sn: 360005 count: 125 metaSz: 2512 rp [2] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:33:08.309+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 94862 recovery UsedSpace 108100 2023-10-21T04:33:08.310+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.310+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.310+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:33:08.310+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 12 count: 125 metaSz: 2512 rp [1] sn: 360005 count: 125 metaSz: 2512 rp [2] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:33:08.310+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 99100 recovery UsedSpace 103614 2023-10-21T04:33:08.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:33:08.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.318+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:33:08.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex started 2023-10-21T04:33:08.318+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.318+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:33:08.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.319+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:33:08.319+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex started 2023-10-21T04:33:08.319+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:33:08.319+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.319+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.335+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Warmup took 25.985463ms 2023-10-21T04:33:08.335+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 4 WriterThreads 1 2023-10-21T04:33:08.335+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index Index: 15864358084358521096 Partition: 4 2023-10-21T04:33:08.336+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.336+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:33:08.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.336+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 123 metaSz: 2512 rp [1] sn: 360006 count: 123 metaSz: 2513 2023-10-21T04:33:08.336+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:33:08.336+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 123 metaSz: 2512 rp [1] sn: 360006 count: 123 metaSz: 2513 2023-10-21T04:33:08.336+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 98886 recovery UsedSpace 111828 2023-10-21T04:33:08.336+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 92943 recovery UsedSpace 115924 2023-10-21T04:33:08.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:33:08.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.346+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:33:08.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex started 2023-10-21T04:33:08.346+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.346+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:33:08.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.347+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:33:08.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex started 2023-10-21T04:33:08.347+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:33:08.347+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.347+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.387+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Warmup took 51.699627ms 2023-10-21T04:33:08.390+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 5 WriterThreads 1 2023-10-21T04:33:08.390+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index Index: 15864358084358521096 Partition: 5 2023-10-21T04:33:08.390+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 122 metaSz: 2512 rp [1] sn: 360006 count: 122 metaSz: 2513 2023-10-21T04:33:08.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 98680 recovery UsedSpace 111828 2023-10-21T04:33:08.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.395+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:33:08.396+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 12 count: 122 metaSz: 2512 rp [1] sn: 360005 count: 122 metaSz: 2512 rp [2] sn: 360006 count: 122 metaSz: 2513 2023-10-21T04:33:08.398+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 92500 recovery UsedSpace 107710 2023-10-21T04:33:08.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:33:08.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.398+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:33:08.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex started 2023-10-21T04:33:08.399+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.399+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.409+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:33:08.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.414+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:33:08.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex started 2023-10-21T04:33:08.416+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.416+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.417+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:33:08.453+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Warmup took 60.2085ms 2023-10-21T04:33:08.453+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 6 WriterThreads 1 2023-10-21T04:33:08.453+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index Index: 15864358084358521096 Partition: 6 2023-10-21T04:33:08.453+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.454+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:33:08.454+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 12 count: 136 metaSz: 2512 rp [1] sn: 360005 count: 136 metaSz: 2512 rp [2] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:33:08.454+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 103980 recovery UsedSpace 108288 2023-10-21T04:33:08.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 136 metaSz: 2512 rp [1] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:33:08.454+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 100346 recovery UsedSpace 111828 2023-10-21T04:33:08.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:33:08.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.463+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:33:08.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex started 2023-10-21T04:33:08.463+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.463+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:33:08.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.465+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:33:08.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex started 2023-10-21T04:33:08.465+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:33:08.465+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.466+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.477+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Warmup took 23.903307ms 2023-10-21T04:33:08.478+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 7 WriterThreads 1 2023-10-21T04:33:08.478+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index Index: 15864358084358521096 Partition: 7 2023-10-21T04:33:08.478+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.478+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:33:08.478+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 12 count: 106 metaSz: 2511 rp [1] sn: 360005 count: 106 metaSz: 2511 rp [2] sn: 360006 count: 106 metaSz: 2512 2023-10-21T04:33:08.479+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 96686 recovery UsedSpace 107710 2023-10-21T04:33:08.479+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.479+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:33:08.479+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 106 metaSz: 2511 rp [1] sn: 360006 count: 106 metaSz: 2512 2023-10-21T04:33:08.479+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 103507 recovery UsedSpace 111828 2023-10-21T04:33:08.486+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:33:08.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:33:08.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.487+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:33:08.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex started 2023-10-21T04:33:08.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.487+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:33:08.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.488+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:33:08.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex started 2023-10-21T04:33:08.488+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:33:08.489+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.489+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.499+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Warmup took 21.072135ms 2023-10-21T04:33:08.499+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 8 WriterThreads 1 2023-10-21T04:33:08.500+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index Index: 15864358084358521096 Partition: 8 2023-10-21T04:33:08.500+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.501+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.501+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:33:08.501+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 13 count: 127 metaSz: 2512 rp [1] sn: 360005 count: 127 metaSz: 2512 rp [2] sn: 360006 count: 127 metaSz: 2513 2023-10-21T04:33:08.501+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 127 metaSz: 2512 rp [1] sn: 360006 count: 127 metaSz: 2513 2023-10-21T04:33:08.501+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 102160 recovery UsedSpace 108194 2023-10-21T04:33:08.501+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 106285 recovery UsedSpace 111828 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.507+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex started 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.507+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:33:08.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex started 2023-10-21T04:33:08.507+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.507+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.507+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.507+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:33:08.507+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.523+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Warmup took 22.535456ms 2023-10-21T04:33:08.523+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 1 WriterThreads 1 2023-10-21T04:33:08.523+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index Index: 15864358084358521096 Partition: 1 2023-10-21T04:33:08.523+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.523+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:33:08.523+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 12 count: 136 metaSz: 2512 rp [1] sn: 360005 count: 136 metaSz: 2512 rp [2] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:33:08.523+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 104253 recovery UsedSpace 108296 2023-10-21T04:33:08.523+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.524+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.524+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 136 metaSz: 2512 rp [1] sn: 360006 count: 136 metaSz: 2513 2023-10-21T04:33:08.524+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 107435 recovery UsedSpace 111828 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.536+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex started 2023-10-21T04:33:08.536+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.536+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:33:08.536+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex started 2023-10-21T04:33:08.536+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.537+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:33:08.537+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.537+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.552+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Warmup took 28.698626ms 2023-10-21T04:33:08.552+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 2 WriterThreads 1 2023-10-21T04:33:08.552+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index Index: 15864358084358521096 Partition: 2 2023-10-21T04:33:08.552+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.553+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:33:08.553+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 125 metaSz: 2512 rp [1] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:33:08.553+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 101720 recovery UsedSpace 116320 2023-10-21T04:33:08.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.553+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:33:08.553+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360005 count: 125 metaSz: 2512 rp [1] sn: 360006 count: 125 metaSz: 2513 2023-10-21T04:33:08.554+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 106292 recovery UsedSpace 111828 2023-10-21T04:33:08.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:33:08.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.563+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:33:08.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex started 2023-10-21T04:33:08.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:33:08.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.564+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:33:08.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex started 2023-10-21T04:33:08.564+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:33:08.564+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.564+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.585+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Warmup took 32.705482ms 2023-10-21T04:33:08.586+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 3 WriterThreads 1 2023-10-21T04:33:08.586+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index Index: 15864358084358521096 Partition: 3 2023-10-21T04:33:08.586+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15864358084358521096 2023-10-21T04:33:08.586+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15864358084358521096 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:33:08.588+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:1 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:33:08.589+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:2 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:33:08.590+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:3 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:33:08.591+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:4 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:33:08.592+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:5 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:33:08.593+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:6 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:33:08.594+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:7 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:33:08.595+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:8 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:33:08.595+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {15864358084358521096 {16961575949607441659 idx_partitioned plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 false [`emailid`] N1QL KEY [false] false true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc005f2fd10 [] 1 0 false plasma 0} map[1:{{1 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005ee4e50} 2:{{2 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005e3de60} 3:{{3 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005ee52e0} 4:{{4 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005afc250} 5:{{5 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005075b70} 6:{{6 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005e3c480} 7:{{7 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005075ea0} 8:{{8 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc005ca52e0}] 0xc005f76580 0xc005f24d20} 2023-10-21T04:33:08.595+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17434304260397056806 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.596+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': added 2023-10-21T04:33:08.597+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner initialized 2023-10-21T04:33:08.597+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:08.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.598+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCleaner initialized for recovery 2023-10-21T04:33:08.598+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:08.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery 2023-10-21T04:33:08.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true] 2023-10-21T04:33:08.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1200128] Data log Head [0] Tail [1273856] 2023-10-21T04:33:08.599+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1200128] 2023-10-21T04:33:08.600+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.600+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:33:08.600+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:33:08.601+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.601+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:33:08.601+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.601+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:33:08.601+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.601+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:33:08.601+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.601+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:33:08.602+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.602+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:33:08.602+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.602+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:33:08.602+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.602+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:33:08.602+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.602+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:33:08.605+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.605+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:33:08.606+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.606+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:08.606+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.606+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:08.606+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.606+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:08.607+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.607+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:08.607+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.607+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:08.607+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.607+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:08.607+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.607+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:08.607+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.607+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:08.608+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : replayOffset: 1265664 (RP:1257472) (CP:1265664) 2023-10-21T04:33:08.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1265664] took [9.096922ms] 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12446, memSzIndex 0, numRecords 123 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11919, memSzIndex 0, numRecords 118 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13881, memSzIndex 0, numRecords 138 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1055176, resident 0.2, memSz 438, memSzIndex 0, numRecords 10 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057276, resident 1, memSz 11323, memSzIndex 0, numRecords 112 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12465, memSzIndex 0, numRecords 123 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13215, memSzIndex 0, numRecords 131 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12819, memSzIndex 0, numRecords 127 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12901, memSzIndex 0, numRecords 128 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1055940, resident 1, memSz 70469, memSzIndex 352, numRecords 1000 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.608+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 27934622 controller mem 1456 2023-10-21T04:33:08.608+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1273856] replayOffset [1265664] 2023-10-21T04:33:08.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [9.920506ms] 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.609+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.610+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.610+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true]. Built [18] plasmas, took [11.023093ms] 2023-10-21T04:33:08.610+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:33:08.610+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 12 numRPs: 3 rp [0] sn: 9 count: 6 metaSz: 2493 rp [1] sn: 360005 count: 6 metaSz: 2493 rp [2] sn: 720006 count: 6 metaSz: 2493 2023-10-21T04:33:08.610+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 70966 recovery UsedSpace 103788 2023-10-21T04:33:08.611+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 238] aggregate storage info: map[bucket_%2:map[kv:2304192 index:4350220] bucket_1:map[kv:2369728 index:10023159]] 2023-10-21T04:33:08.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:33:08.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.615+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper started 2023-10-21T04:33:08.616+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:33:08.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex started 2023-10-21T04:33:08.616+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:33:08.616+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.616+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.629+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Warmup took 32.951854ms 2023-10-21T04:33:08.629+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17434304260397056806 partitionId 0 WriterThreads 4 2023-10-21T04:33:08.629+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17434304260397056806 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index Index: 17434304260397056806 Partition: 0 2023-10-21T04:33:08.629+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17434304260397056806 2023-10-21T04:33:08.629+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17434304260397056806 Partitions [0] 2023-10-21T04:33:08.631+05:30 [Info] StorageMgr::openSnapshot IndexInst:17434304260397056806 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:33:08.631+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {17434304260397056806 {3381524789655566900 #primary plasma bucket_%2 b9538b29d7e025c819e734b646db039c true [] N1QL SINGLE [] false false [] false false 1 [] false 0 {true 1 0 0} _system _query 8 9 false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc0066376e0 [] 1 1 false plasma 0} map[0:{{0 1 [:9123] [17406544375239214575]} 0xc006126390}] 0xc006141200 0xc006674120} 2023-10-21T04:33:08.632+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4083560586502645495 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:33:08.632+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1056160 memSz 2390 2023-10-21T04:33:08.632+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 16 count: 1000 metaSz: 2508 rp [1] sn: 17 count: 1000 metaSz: 2514 2023-10-21T04:33:08.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:08.632+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1056344 memSz 698 2023-10-21T04:33:08.632+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 192546 recovery UsedSpace 47903 2023-10-21T04:33:08.632+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 16 count: 1000 metaSz: 2508 rp [1] sn: 17 count: 1000 metaSz: 2514 2023-10-21T04:33:08.632+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 189100 recovery UsedSpace 43178 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:33:08.638+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex started 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:08.638+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:33:08.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex started 2023-10-21T04:33:08.638+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.638+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.638+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4083560586502645495 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:33:08.638+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.638+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.656+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4083560586502645495 PartitionId 0 Warmup took 24.744433ms 2023-10-21T04:33:08.657+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4083560586502645495 partitionId 0 WriterThreads 4 2023-10-21T04:33:08.657+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4083560586502645495 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index Index: 4083560586502645495 Partition: 0 2023-10-21T04:33:08.657+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4083560586502645495 2023-10-21T04:33:08.657+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4083560586502645495 Partitions [0] 2023-10-21T04:33:08.658+05:30 [Info] StorageMgr::openSnapshot IndexInst:4083560586502645495 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:33:08.658+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {4083560586502645495 {4790214984019693894 idx_secondary plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 false [`age`] N1QL SINGLE [false] false false [] false false 1 [] false 0 {true 1 0 0} _default c1 0 c false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc005db13b0 [] 1 1 false plasma 0} map[0:{{0 1 [:9123] [16787445029868804812 2925063509317026816]} 0xc006689060}] 0xc005dd1900 0xc005aeff20} 2023-10-21T04:33:08.659+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.659+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.659+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 16 count: 112 metaSz: 2520 rp [1] sn: 360005 count: 112 metaSz: 2520 2023-10-21T04:33:08.660+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 81205 recovery UsedSpace 95258 2023-10-21T04:33:08.661+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner initialized 2023-10-21T04:33:08.661+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:08.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.662+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCleaner initialized for recovery 2023-10-21T04:33:08.662+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:33:08.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery 2023-10-21T04:33:08.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true] 2023-10-21T04:33:08.662+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1069056] Data log Head [0] Tail [1150976] 2023-10-21T04:33:08.662+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1069056] 2023-10-21T04:33:08.664+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.664+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:33:08.664+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.664+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:33:08.664+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.664+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:33:08.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:33:08.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.665+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:33:08.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex started 2023-10-21T04:33:08.665+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.665+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.665+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:33:08.665+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.665+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.665+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:33:08.666+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.666+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:33:08.666+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.666+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:33:08.666+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.666+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:33:08.668+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.668+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:33:08.670+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.670+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:08.671+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.671+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:08.671+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.671+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:08.671+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.671+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:08.671+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.671+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:08.672+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.672+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:08.672+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.672+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:08.672+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:08.672+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:08.673+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : replayOffset: 1138688 (RP:1138688) (CP:794624) 2023-10-21T04:33:08.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1138688] took [10.158156ms] 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053364, resident 1, memSz 9834, memSzIndex 0, numRecords 123 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9287, memSzIndex 0, numRecords 118 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053288, resident 1, memSz 10687, memSzIndex 75, numRecords 138 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9511, memSzIndex 76, numRecords 127 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9997, memSzIndex 77, numRecords 131 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9669, memSzIndex 76, numRecords 128 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053364, resident 1, memSz 9853, memSzIndex 0, numRecords 123 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1053288, resident 1, memSz 8667, memSzIndex 0, numRecords 112 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1056196, resident 1, memSz 67249, memSzIndex 994, numRecords 1000 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:33:08.673+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 43705138 controller mem 5004 2023-10-21T04:33:08.673+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1150976] replayOffset [1138688] 2023-10-21T04:33:08.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [11.201862ms] 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.674+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.675+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:33:08.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true]. Built [17] plasmas, took [12.111485ms] 2023-10-21T04:33:08.675+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:33:08.675+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 14 count: 112 metaSz: 2514 rp [1] sn: 15 count: 112 metaSz: 2518 rp [2] sn: 16 count: 112 metaSz: 2520 rp [3] sn: 360005 count: 112 metaSz: 2520 2023-10-21T04:33:08.675+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 84986 recovery UsedSpace 91118 2023-10-21T04:33:08.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:33:08.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.689+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:33:08.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex started 2023-10-21T04:33:08.689+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:33:08.689+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.690+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.738+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Warmup took 79.141132ms 2023-10-21T04:33:08.739+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 8 WriterThreads 1 2023-10-21T04:33:08.739+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index Index: 8965923595543532942 Partition: 8 2023-10-21T04:33:08.739+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.739+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:33:08.739+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 13 count: 128 metaSz: 2514 rp [1] sn: 14 count: 128 metaSz: 2518 rp [2] sn: 15 count: 128 metaSz: 2520 rp [3] sn: 360005 count: 128 metaSz: 2520 2023-10-21T04:33:08.740+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 93042 recovery UsedSpace 91696 2023-10-21T04:33:08.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.740+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.740+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 15 count: 128 metaSz: 2520 rp [1] sn: 360005 count: 128 metaSz: 2520 2023-10-21T04:33:08.740+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 82744 recovery UsedSpace 95258 2023-10-21T04:33:08.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:33:08.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.755+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:33:08.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex started 2023-10-21T04:33:08.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.756+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:33:08.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.759+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:33:08.759+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex started 2023-10-21T04:33:08.759+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:33:08.760+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.760+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.782+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Warmup took 43.022467ms 2023-10-21T04:33:08.782+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 1 WriterThreads 1 2023-10-21T04:33:08.783+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index Index: 8965923595543532942 Partition: 1 2023-10-21T04:33:08.783+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.783+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:33:08.783+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 14 count: 123 metaSz: 2514 rp [1] sn: 15 count: 123 metaSz: 2517 rp [2] sn: 16 count: 123 metaSz: 2520 rp [3] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:33:08.783+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 90280 recovery UsedSpace 91118 2023-10-21T04:33:08.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.783+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:33:08.783+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 16 count: 123 metaSz: 2520 rp [1] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:33:08.783+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 87241 recovery UsedSpace 95258 2023-10-21T04:33:08.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:33:08.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.790+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:33:08.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex started 2023-10-21T04:33:08.790+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.790+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:33:08.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.791+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:33:08.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex started 2023-10-21T04:33:08.791+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.791+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.791+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:33:08.808+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Warmup took 25.174884ms 2023-10-21T04:33:08.808+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 2 WriterThreads 1 2023-10-21T04:33:08.808+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index Index: 8965923595543532942 Partition: 2 2023-10-21T04:33:08.809+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.809+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:33:08.809+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 13 count: 123 metaSz: 2514 rp [1] sn: 14 count: 123 metaSz: 2517 rp [2] sn: 15 count: 123 metaSz: 2520 rp [3] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:33:08.809+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 94610 recovery UsedSpace 91118 2023-10-21T04:33:08.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.810+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:33:08.810+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 15 count: 123 metaSz: 2520 rp [1] sn: 360005 count: 123 metaSz: 2520 2023-10-21T04:33:08.810+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 87276 recovery UsedSpace 95258 2023-10-21T04:33:08.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:33:08.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.817+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:33:08.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex started 2023-10-21T04:33:08.817+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.817+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:33:08.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.818+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:33:08.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex started 2023-10-21T04:33:08.818+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.818+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.819+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:33:08.831+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Warmup took 22.51324ms 2023-10-21T04:33:08.832+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 3 WriterThreads 1 2023-10-21T04:33:08.832+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index Index: 8965923595543532942 Partition: 3 2023-10-21T04:33:08.832+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.832+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:33:08.832+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 14 count: 118 metaSz: 2514 rp [1] sn: 15 count: 118 metaSz: 2517 rp [2] sn: 16 count: 118 metaSz: 2520 rp [3] sn: 360005 count: 118 metaSz: 2520 2023-10-21T04:33:08.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.832+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.832+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 16 count: 118 metaSz: 2520 rp [1] sn: 360005 count: 118 metaSz: 2520 2023-10-21T04:33:08.832+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 94090 recovery UsedSpace 91118 2023-10-21T04:33:08.833+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 86711 recovery UsedSpace 95258 2023-10-21T04:33:08.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:33:08.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.845+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:33:08.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex started 2023-10-21T04:33:08.845+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.845+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:33:08.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.850+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:33:08.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex started 2023-10-21T04:33:08.850+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:33:08.850+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.850+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.894+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Warmup took 62.256296ms 2023-10-21T04:33:08.894+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 4 WriterThreads 1 2023-10-21T04:33:08.894+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index Index: 8965923595543532942 Partition: 4 2023-10-21T04:33:08.894+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 238 2023-10-21T04:33:08.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 14 count: 138 metaSz: 2514 rp [1] sn: 15 count: 138 metaSz: 2518 rp [2] sn: 16 count: 138 metaSz: 2520 rp [3] sn: 360005 count: 138 metaSz: 2520 2023-10-21T04:33:08.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 103889 recovery UsedSpace 91688 2023-10-21T04:33:08.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.895+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.895+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 16 count: 138 metaSz: 2520 rp [1] sn: 360005 count: 138 metaSz: 2520 2023-10-21T04:33:08.895+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 88652 recovery UsedSpace 99354 2023-10-21T04:33:08.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:33:08.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.903+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:33:08.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex started 2023-10-21T04:33:08.903+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.903+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:33:08.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.906+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:33:08.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex started 2023-10-21T04:33:08.906+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:33:08.907+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.907+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.928+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Warmup took 33.7715ms 2023-10-21T04:33:08.929+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 5 WriterThreads 1 2023-10-21T04:33:08.929+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index Index: 8965923595543532942 Partition: 5 2023-10-21T04:33:08.929+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.930+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.930+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 16 count: 127 metaSz: 2520 rp [1] sn: 360005 count: 127 metaSz: 2520 2023-10-21T04:33:08.930+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 92956 recovery UsedSpace 95258 2023-10-21T04:33:08.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.930+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:33:08.930+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 14 count: 127 metaSz: 2514 rp [1] sn: 15 count: 127 metaSz: 2518 rp [2] sn: 16 count: 127 metaSz: 2520 rp [3] sn: 360005 count: 127 metaSz: 2520 2023-10-21T04:33:08.930+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 102257 recovery UsedSpace 91602 2023-10-21T04:33:08.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:33:08.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.938+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:33:08.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex started 2023-10-21T04:33:08.938+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.938+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:33:08.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.939+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:33:08.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex started 2023-10-21T04:33:08.939+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:33:08.939+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.939+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.951+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Warmup took 22.083891ms 2023-10-21T04:33:08.952+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 6 WriterThreads 1 2023-10-21T04:33:08.952+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index Index: 8965923595543532942 Partition: 6 2023-10-21T04:33:08.952+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.952+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:33:08.952+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 9 numRPs: 4 rp [0] sn: 14 count: 131 metaSz: 2514 rp [1] sn: 15 count: 131 metaSz: 2518 rp [2] sn: 16 count: 131 metaSz: 2520 rp [3] sn: 360005 count: 131 metaSz: 2520 2023-10-21T04:33:08.952+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 103142 recovery UsedSpace 91704 2023-10-21T04:33:08.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.952+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:33:08.953+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 11 numRPs: 2 rp [0] sn: 16 count: 131 metaSz: 2520 rp [1] sn: 360005 count: 131 metaSz: 2520 2023-10-21T04:33:08.953+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 93706 recovery UsedSpace 95258 2023-10-21T04:33:08.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:33:08.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:08.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:08.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:33:08.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:08.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:08.960+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:33:08.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex started 2023-10-21T04:33:08.961+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.960+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:33:08.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex started 2023-10-21T04:33:08.961+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:33:08.962+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.961+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:08.961+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:08.979+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Warmup took 27.2791ms 2023-10-21T04:33:08.979+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 7 WriterThreads 1 2023-10-21T04:33:08.979+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index Index: 8965923595543532942 Partition: 7 2023-10-21T04:33:08.980+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8965923595543532942 2023-10-21T04:33:08.980+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8965923595543532942 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:33:08.981+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:1 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:33:08.984+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:2 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:33:08.985+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:3 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:33:08.987+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:4 Attempting to open snapshot (SnapshotInfo: count:118 committed:false) 2023-10-21T04:33:08.988+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:5 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:33:08.989+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:33:08.990+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:7 Attempting to open snapshot (SnapshotInfo: count:131 committed:false) 2023-10-21T04:33:08.992+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:8 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:33:08.992+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {8965923595543532942 {8910459001587546151 idx_partitioned plasma bucket_%2 b9538b29d7e025c819e734b646db039c false [`emailid`] N1QL KEY [false] false true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc006e3ca20 [] 1 1 false plasma 0} map[1:{{1 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc006710620} 2:{{2 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc006d10060} 3:{{3 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc006ce0190} 4:{{4 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc006ce0620} 5:{{5 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc006d632a0} 6:{{6 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc006c99ac0} 7:{{7 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc006eb72f0} 8:{{8 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005afdfa0}] 0xc006e65900 0xc006053a40} 2023-10-21T04:33:08.992+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:08.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:08.993+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.993+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:33:08.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:08.993+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:08.993+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:09.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:09.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.003+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:09.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:33:09.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:09.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.004+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:09.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:33:09.004+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.004+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:33:09.004+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.004+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.004+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.004+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.004+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:33:09.004+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:09.005+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:33:09.005+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:33:09.005+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:09.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:33:09.005+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.005+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.005+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.005+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.015+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:33:09.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:33:09.015+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.015+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:33:09.016+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:33:09.016+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:09.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.016+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.025+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:33:09.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:33:09.025+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:33:09.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:33:09.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:33:09.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:33:09.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:33:09.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.039+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.039+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.045+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.045+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:09.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.049+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:09.049+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:33:09.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:33:09.049+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.049+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:09.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.052+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:09.052+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:33:09.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:33:09.052+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Warmup took 60.018564ms 2023-10-21T04:33:09.053+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.053+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 1 WriterThreads 1 2023-10-21T04:33:09.053+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.053+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index Index: 8732144615635911668 Partition: 1 2023-10-21T04:33:09.053+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.053+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.053+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.053+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:33:09.054+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:09.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:09.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.062+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:09.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:33:09.062+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:09.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.063+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:09.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:33:09.063+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.064+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:33:09.064+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:33:09.064+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:33:09.064+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:09.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.064+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.065+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.071+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:33:09.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:33:09.072+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.072+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.072+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.072+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:33:09.073+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:09.073+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:33:09.073+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:33:09.073+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:09.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:33:09.074+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.074+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.074+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.074+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.084+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:33:09.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:33:09.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:33:09.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:33:09.087+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:09.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:33:09.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:33:09.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:33:09.099+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.100+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.100+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.105+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.105+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:09.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.106+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:09.106+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:33:09.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:33:09.106+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.106+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:09.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.116+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:09.116+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:33:09.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:33:09.116+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.116+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Warmup took 63.433451ms 2023-10-21T04:33:09.117+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 2 WriterThreads 1 2023-10-21T04:33:09.117+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index Index: 8732144615635911668 Partition: 2 2023-10-21T04:33:09.117+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.117+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.117+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:33:09.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.118+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.118+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:09.118+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:09.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.131+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:09.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:33:09.131+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.131+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:09.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.135+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:09.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:33:09.136+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:33:09.136+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:33:09.136+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:33:09.136+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:09.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.136+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:33:09.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:33:09.148+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:33:09.148+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:09.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.160+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:33:09.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:33:09.161+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:33:09.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:33:09.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:33:09.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:33:09.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:33:09.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.174+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.177+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:09.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:09.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:33:09.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:33:09.181+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.181+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:09.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.185+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:09.185+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:33:09.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:33:09.185+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Warmup took 67.938414ms 2023-10-21T04:33:09.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.185+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 3 WriterThreads 1 2023-10-21T04:33:09.185+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index Index: 8732144615635911668 Partition: 3 2023-10-21T04:33:09.185+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.186+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.186+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:33:09.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.186+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.186+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 22755 recovery UsedSpace 4118 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.195+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.195+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:33:09.195+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:33:09.195+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.195+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.195+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.195+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.195+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.195+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.195+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:33:09.195+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:09.196+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:33:09.196+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:33:09.196+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:09.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:33:09.196+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.196+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.196+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.196+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.196+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.200+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:33:09.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:33:09.200+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:33:09.201+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:33:09.201+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:09.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.201+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.206+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:33:09.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:33:09.207+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:33:09.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:33:09.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:33:09.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:33:09.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:33:09.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.216+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.216+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.219+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.219+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:09.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.223+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:09.223+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:33:09.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:33:09.223+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:09.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.227+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:09.227+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:33:09.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:33:09.228+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Warmup took 42.135329ms 2023-10-21T04:33:09.228+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 4 WriterThreads 1 2023-10-21T04:33:09.228+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index Index: 8732144615635911668 Partition: 4 2023-10-21T04:33:09.228+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.228+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.228+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.229+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:33:09.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.229+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.229+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.229+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.234+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:09.234+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:33:09.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:33:09.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.234+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.234+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.234+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.234+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.234+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.234+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:33:09.234+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.235+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:09.235+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:33:09.235+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:33:09.235+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:09.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:33:09.235+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.235+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.235+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.235+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:33:09.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:33:09.240+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:33:09.240+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:09.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.252+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:33:09.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:33:09.252+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:33:09.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:33:09.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:33:09.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:33:09.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:33:09.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.290+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.290+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.296+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:09.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.300+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:09.300+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:33:09.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:33:09.301+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.301+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:09.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.308+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:09.308+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:33:09.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:33:09.308+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Warmup took 79.962642ms 2023-10-21T04:33:09.308+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 5 WriterThreads 1 2023-10-21T04:33:09.308+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index Index: 8732144615635911668 Partition: 5 2023-10-21T04:33:09.308+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.309+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.309+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:33:09.309+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.309+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.310+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.310+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:33:09.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:09.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.317+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:09.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:33:09.317+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.317+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:09.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.321+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:09.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:33:09.321+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:33:09.321+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:33:09.321+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:33:09.321+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:09.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.321+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.322+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.348+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:33:09.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:33:09.349+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.350+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.350+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.350+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:33:09.350+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:09.350+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:33:09.350+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:33:09.350+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:09.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:33:09.351+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.351+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.351+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.351+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.361+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:33:09.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:33:09.361+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:33:09.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:33:09.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:33:09.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:33:09.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:33:09.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.384+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.384+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.401+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.401+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.401+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:09.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.406+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:09.406+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:33:09.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:33:09.406+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:09.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.407+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:09.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.407+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:33:09.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:33:09.407+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Warmup took 98.074675ms 2023-10-21T04:33:09.407+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 6 WriterThreads 1 2023-10-21T04:33:09.407+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index Index: 8732144615635911668 Partition: 6 2023-10-21T04:33:09.407+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.408+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.408+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:33:09.408+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.408+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.408+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.409+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:33:09.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:09.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.413+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:09.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:33:09.413+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:09.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.414+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:09.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:33:09.414+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:33:09.414+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:33:09.414+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:33:09.414+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:09.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.414+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.415+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:33:09.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:33:09.419+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:33:09.419+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:09.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.419+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.423+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.423+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:33:09.423+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:33:09.423+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.423+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:33:09.423+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:33:09.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:33:09.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:33:09.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:33:09.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.434+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.434+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.441+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.442+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:09.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.445+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:09.445+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:33:09.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:33:09.445+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.445+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:09.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.451+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:09.451+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:33:09.451+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:33:09.451+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.451+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Warmup took 43.673938ms 2023-10-21T04:33:09.451+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.451+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 7 WriterThreads 1 2023-10-21T04:33:09.451+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index Index: 8732144615635911668 Partition: 7 2023-10-21T04:33:09.451+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.452+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.452+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:33:09.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.453+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:33:09.453+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:33:09.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:09.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.470+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:09.470+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:33:09.470+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.470+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:09.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.472+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:09.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:33:09.472+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:33:09.472+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:33:09.472+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:33:09.472+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:09.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.472+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.473+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.480+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:33:09.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:33:09.480+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:09.480+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:09.480+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:09.480+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:09.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:33:09.481+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:09.481+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:33:09.481+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:33:09.481+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:09.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:33:09.481+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:09.481+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:09.481+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:09.481+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:09.491+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:09.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:33:09.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:33:09.491+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:09.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:33:09.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:33:09.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:33:09.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:33:09.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:33:09.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:33:09.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.500+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.500+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:09.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.504+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:33:09.504+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:33:09.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:09.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.506+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:09.506+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:33:09.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:33:09.506+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.506+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:09.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.509+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:09.509+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:33:09.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:33:09.509+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Warmup took 57.449182ms 2023-10-21T04:33:09.509+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 8 WriterThreads 1 2023-10-21T04:33:09.509+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index Index: 8732144615635911668 Partition: 8 2023-10-21T04:33:09.510+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.510+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.510+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8732144615635911668 2023-10-21T04:33:09.510+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:33:09.510+05:30 [Info] StorageMgr::openSnapshot IndexInst:8732144615635911668 Partition:6 No Snapshot Found. 2023-10-21T04:33:09.510+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Adding Nil Snapshot. 2023-10-21T04:33:09.510+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {8732144615635911668 {10606939961365971246 idx_partitioned_defer plasma bucket_%2 b9538b29d7e025c819e734b646db039c false [`balance`] N1QL KEY [false] true true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 0 0 0 0xc0075a3350 [] 1 0 false plasma 0} map[1:{{1 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc007106db0} 2:{{2 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc007314330} 3:{{3 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc0070d1ad0} 4:{{4 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc0071627c0} 5:{{5 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc007315f00} 6:{{6 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc0071636a0} 7:{{7 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc0074a2680} 8:{{8 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc00752ebb0}] 0xc0075c8000 0xc00758d440} 2023-10-21T04:33:09.510+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3464125101225762400 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.511+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:33:09.511+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:33:09.511+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: RP version: 14 numRPs: 3 rp [0] sn: 360006 count: 6 metaSz: 2481 rp [1] sn: 360007 count: 5 metaSz: 2482 rp [2] sn: 720006 count: 6 metaSz: 2482 2023-10-21T04:33:09.511+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 89921 recovery UsedSpace 120474 2023-10-21T04:33:09.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:33:09.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:33:09.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:33:09.516+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:33:09.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex started 2023-10-21T04:33:09.516+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.516+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:33:09.518+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.530+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Warmup took 19.351263ms 2023-10-21T04:33:09.530+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3464125101225762400 partitionId 0 WriterThreads 4 2023-10-21T04:33:09.530+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3464125101225762400 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index Index: 3464125101225762400 Partition: 0 2023-10-21T04:33:09.530+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3464125101225762400 2023-10-21T04:33:09.531+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3464125101225762400 Partitions [0] 2023-10-21T04:33:09.532+05:30 [Info] StorageMgr::openSnapshot IndexInst:3464125101225762400 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:33:09.532+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {3464125101225762400 {11276338709617092948 #primary plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 true [] N1QL SINGLE [] false false [] false false 1 [] false 0 {true 1 0 0} _system _query 8 9 false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc00761bdd0 [] 1 0 false plasma 0} map[0:{{0 1 [:9123] [16787445029868804812]} 0xc007582ef0}] 0xc00765a580 0xc007620780} 2023-10-21T04:33:09.532+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2670343739772083055 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:33:09.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:33:09.533+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: after recovery: resident ratio 0 requestQuota 1056196 memSz 2212 2023-10-21T04:33:09.533+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 22 count: 1000 metaSz: 2517 rp [1] sn: 23 count: 1000 metaSz: 2525 2023-10-21T04:33:09.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:33:09.533+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: after recovery: resident ratio 0 requestQuota 1055940 memSz 880 2023-10-21T04:33:09.533+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 22 count: 1000 metaSz: 2517 rp [1] sn: 23 count: 1000 metaSz: 2525 2023-10-21T04:33:09.533+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: doInit: data UsedSpace 167805 recovery UsedSpace 47257 2023-10-21T04:33:09.533+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: doInit: data UsedSpace 180806 recovery UsedSpace 43786 2023-10-21T04:33:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:33:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:33:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:33:09.541+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:33:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex started 2023-10-21T04:33:09.541+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.541+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:33:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:33:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:33:09.541+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:33:09.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex started 2023-10-21T04:33:09.542+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2670343739772083055 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:33:09.542+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:33:09.542+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:33:09.561+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2670343739772083055 PartitionId 0 Warmup took 28.200304ms 2023-10-21T04:33:09.561+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2670343739772083055 partitionId 0 WriterThreads 4 2023-10-21T04:33:09.561+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2670343739772083055 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index Index: 2670343739772083055 Partition: 0 2023-10-21T04:33:09.562+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2670343739772083055 2023-10-21T04:33:09.562+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2670343739772083055 Partitions [0] 2023-10-21T04:33:09.563+05:30 [Info] StorageMgr::openSnapshot IndexInst:2670343739772083055 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:33:09.563+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc00004c780 ScanCoordinator {2670343739772083055 {13268712595133993155 idx_secondary plasma bucket_%2 b9538b29d7e025c819e734b646db039c false [`age`] N1QL SINGLE [false] false false [] false false 1 [] false 0 {true 1 0 0} _default c1 0 c false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc00753dfb0 [] 1 1 false plasma 0} map[0:{{0 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc007667790}] 0xc007716580 0xc007556960} 2023-10-21T04:33:09.565+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:33:09.565+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:33:09.565+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:33:09.566+05:30 [Info] RebalanceServiceManager::recoverRebalance: Init Pending Cleanup 2023-10-21T04:33:09.567+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:33:09.571+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:33:09.574+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:33:09.574+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:33:09.580+05:30 [Info] RebalanceServiceManager::doRecoverRebalance Found Global Rebalance Token &{18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:33:09.580+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:33:09.584+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:33:09.588+05:30 [Info] Setting log level to Info 2023-10-21T04:33:09.592+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:33:09.591+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {18194302e3f10df59bd076e1c4142d0b 1f784fc0142ee312f0d92d98bd188c9a Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:33:09.598+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 62731 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:33:09.598+05:30 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2023-10-21T04:33:09.599+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:33:09.599+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1]] 2023-10-21T04:33:09.599+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2]] 2023-10-21T04:33:09.599+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3]] 2023-10-21T04:33:09.599+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:33:09.600+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] [17406544375239214575 10170029854712240877]} 2:{2 1 [:9123] [17406544375239214575 10170029854712240877]} 3:{3 1 [:9123] [17406544375239214575 10170029854712240877]} 4:{4 1 [:9123] [17406544375239214575 10170029854712240877]} 5:{5 1 [:9123] [17406544375239214575 10170029854712240877]} 6:{6 1 [:9123] [17406544375239214575 10170029854712240877]} 7:{7 1 [:9123] [17406544375239214575 10170029854712240877]} 8:{8 1 [:9123] [17406544375239214575 10170029854712240877]}] 8 KEY 0} InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [17406544375239214575 10170029854712240877]}] 1 SINGLE 0} InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [17406544375239214575]}] 1 SINGLE 0}] Async: true SessionId: 62731 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 25 25 25} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 17 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 16 16 16} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 31 31 31} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:33:09.603+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:33:09.603+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:33:09.607+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 61684 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:33:09.607+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:09.607+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:09.608+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:33:09.608+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:1]] 2023-10-21T04:33:09.608+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:2]] 2023-10-21T04:33:09.609+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3]] 2023-10-21T04:33:09.610+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:33:09.610+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] [16787445029868804812 2925063509317026816]} 2:{2 1 [:9123] [16787445029868804812 2925063509317026816]} 3:{3 1 [:9123] [16787445029868804812 2925063509317026816]} 4:{4 1 [:9123] [16787445029868804812 2925063509317026816]} 5:{5 1 [:9123] [16787445029868804812 2925063509317026816]} 6:{6 1 [:9123] [16787445029868804812 2925063509317026816]} 7:{7 1 [:9123] [16787445029868804812 2925063509317026816]} 8:{8 1 [:9123] [16787445029868804812 2925063509317026816]}] 8 KEY 0} InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [16787445029868804812 2925063509317026816]}] 1 SINGLE 0} InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] [16787445029868804812]}] 1 SINGLE 0}] Async: true SessionId: 61684 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 20 0 20} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:33:09.611+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0046acf00 } bucket_1:{0xc0046ad140 }] 2023-10-21T04:33:09.613+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : all deamons started 2023-10-21T04:33:09.613+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : all deamons started 2023-10-21T04:33:09.613+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : all deamons started 2023-10-21T04:33:09.613+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : all deamons started 2023-10-21T04:33:09.613+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : all deamons started 2023-10-21T04:33:09.614+05:30 [Info] ScanCoordinator::initialize rollback times on indexer resume: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:09.614+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.615+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.614+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : all deamons started 2023-10-21T04:33:09.620+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : all deamons started 2023-10-21T04:33:09.620+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : all deamons started 2023-10-21T04:33:09.620+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.614+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.621+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.621+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.621+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:33:09.621+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:33:09.622+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:33:09.622+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:33:09.622+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:33:09.622+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:33:09.623+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:33:09.637+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:33:09.638+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:33:09.642+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:33:09.648+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:33:09.648+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:33:09.648+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:33:09.651+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:33:09.654+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:33:09.693+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:56540" doAuth() ... 2023-10-21T04:33:09.694+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:56540" auth successful 2023-10-21T04:33:09.694+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:56540" +1 2023-10-21T04:33:09.714+05:30 [Info] security context initialized 2023-10-21T04:33:09.714+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:33:09.728+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:33:09.744+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:33:09.744+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:09.744+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:09.744+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:33:09.753+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:33:09.762+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:33:09.762+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:33:09.764+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:33:09.768+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:33:09.768+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:33:09.792+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:33:09.792+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:33:09.810+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:33:09.844+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:33:09.844+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:33:09.867+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60080 2023-10-21T04:33:09.868+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:33:09.919+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 35.269934ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:09.921+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:33:09.923+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:33:09.921+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 174.705µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:09.944+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:33:10.210+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:33:10.214+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:33:10.214+05:30 [Info] initialized currmeta 33 force true 2023-10-21T04:33:10.224+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:10.224+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:10.227+05:30 [Info] switched currmeta from 33 -> 33 force true 2023-10-21T04:33:10.227+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:33:10.228+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[4 5 6 7 8 1 2 3] 10402379389576076017:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:33:10.228+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[4 5 6 7 8 1 2 3] 10402379389576076017:[0] 10601152754683910907:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:33:10.233+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:33:10.233+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:33:10.233+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:33:10.236+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:33:10.248+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:33:10.248+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:33:10.248+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:33:10.249+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:33:10.272+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:33:10.272+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:33:10.272+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:33:10.273+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:33:10.274+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43480" doAuth() ... 2023-10-21T04:33:10.274+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43480" doReceive() ... 2023-10-21T04:33:10.285+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:33:10.285+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:33:10.285+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:33:10.288+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:33:10.297+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:33:10.300+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:33:10.300+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:33:10.301+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:33:10.301+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:33:10.301+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:33:10.302+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:33:10.315+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:33:10.317+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:33:10.318+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenff:34:4:b4:54:ed:73:4a MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10402379389576076017 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10465155089296381829 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2062526591622067910 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:33:10.318+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken13:82:a:ff:71:77:a0:d9 MasterId: SourceId: DestId: RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokena2:b2:c5:e2:b2:69:50:c0 SiblingTokenId: ShardToken7f:de:a2:ba:54:17:8e:fe Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:33:10.318+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenb2:f8:fb:ca:53:2e:1c:c4 MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: 1f784fc0142ee312f0d92d98bd188c9a ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenff:34:4:b4:54:ed:73:4a Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11304674148443950662 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4083560586502645495 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 4708891605960345271 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:33:10.318+05:30 [Info] RebalanceServiceManager::cleanupShardTokenForDest Fetching transfer tokens again from metaKV before attempting to drop token on destination 2023-10-21T04:33:10.331+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 16.596839ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:10.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6043349427976659583:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:33:10.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4238649781004957942:[0] 6043349427976659583:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:33:10.374+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 4238649781004957942:[0] 6043349427976659583:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:33:10.390+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno cache..., established connections: map[172.31.5.112:49592:172.31.5.112:12000] 2023-10-21T04:33:10.422+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 274143649547516 29 62731 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:33:10.425+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 162049908746825 25 62731 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:33:10.427+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 206082810619393 24 62731 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:10.430+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 120349894144442 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.432+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 143687065777286 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.431+05:30 [Info] NewTsVbuuidCached: bucket_1, 64 TsVbuuidPoolMap is updated from: map[] to: map[64:0xc004e803f0] 2023-10-21T04:33:10.437+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 116493903186721 22 62731 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:10.438+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 212967951483349 28 62731 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:10.439+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 78446412189949 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.441+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 136568163328749 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.441+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 263088038522959 18 62731 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:10.443+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 261951617336488 35 62731 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:33:10.445+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 127746572208200 19 62731 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:10.449+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 58733680850130 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.450+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 173457131919400 12 62731 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:33:10.456+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 245247859472006 18 62731 172.31.5.112:9000. HWT [17-18,18]. 2023-10-21T04:33:10.456+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 277338544220217 21 62731 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:10.456+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 188740876924377 22 62731 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:10.458+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 34510046292135 16 62731 172.31.5.112:9000. HWT [16-16,16]. 2023-10-21T04:33:10.458+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 206088069380492 13 62731 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:33:10.465+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 250426459207023 28 62731 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:10.466+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 35716737784090 24 62731 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:10.468+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 191500187207978 21 62731 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:10.483+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 106231614376481 21 62731 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:10.483+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 274158831830257 20 62731 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:10.483+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 137256376714252 22 62731 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:10.486+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 146375746634523 17 62731 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:10.486+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 5906787184229 20 62731 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:10.493+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 41101832875819 19 62731 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:10.496+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 121450709505223 25 62731 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:10.517+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 241391494503904 28 62731 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 168446351781198 25 62731 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 127702263378100 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 100008922352935 17 62731 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 118920785117980 21 62731 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 125696667341112 27 62731 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 216238759475962 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 245900669851618 19 62731 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 169268048765249 19 62731 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:10.518+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 112482982607415 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.519+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 149033089785397 28 62731 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:10.520+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 106297365959267 27 62731 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:10.523+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 88718752743571 25 62731 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:10.524+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 39045953044628 18 62731 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:10.527+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 160399698726237 27 62731 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:33:10.529+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 217237947818243 20 62731 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:10.530+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 110286285845649 31 62731 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:33:10.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 162870112426804 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.534+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 132470209691538 26 62731 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:10.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 63737850123332 19 62731 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:10.540+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 264001907139144 22 62731 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:10.542+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 266604622117081 26 62731 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:10.543+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 32274016651656 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.546+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 225662991941187 26 62731 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:10.546+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 84581802233088 22 62731 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:10.550+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 272523968790649 20 62731 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:10.550+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 241607113601772 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.553+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 197957220074784 24 62731 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:10.553+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 77598347829810 19 62731 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:10.556+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 136038310030103 28 62731 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:10.556+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 97677619676504 31 62731 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:33:10.558+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 72072509155168 23 62731 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:10.564+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 105415986286631 18 62731 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:10.564+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 27479155597282 29 62731 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:10.565+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 InstanceIds [17434304260397056806 8965923595543532942 2670343739772083055] 2023-10-21T04:33:10.565+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 62731 2023-10-21T04:33:10.565+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 62731 2023-10-21T04:33:10.565+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:33:10.565+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:33:10.565+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:33:10.565+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:33:10.565+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 62731 2023-10-21T04:33:10.565+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 62731 STREAM_ACTIVE 2023-10-21T04:33:10.565+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:33:10.568+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 161870047388121 25 62731 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:10.593+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49618:172.31.5.112:12000] 2023-10-21T04:33:10.615+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49622:172.31.5.112:12000] 2023-10-21T04:33:10.642+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49624:172.31.5.112:12000] 2023-10-21T04:33:10.661+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49626:172.31.5.112:12000] 2023-10-21T04:33:10.701+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49632:172.31.5.112:12000] 2023-10-21T04:33:10.713+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49630:172.31.5.112:12000] 2023-10-21T04:33:10.738+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49634:172.31.5.112:12000] 2023-10-21T04:33:10.743+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is enabled 2023-10-21T04:33:10.749+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49620:172.31.5.112:12000] 2023-10-21T04:33:10.776+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49628:172.31.5.112:12000] 2023-10-21T04:33:10.794+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49636:172.31.5.112:12000] 2023-10-21T04:33:10.929+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:49648:172.31.5.112:12000] 2023-10-21T04:33:11.026+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49670:172.31.5.112:12000] 2023-10-21T04:33:11.029+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49672:172.31.5.112:12000] 2023-10-21T04:33:11.066+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49674:172.31.5.112:12000] 2023-10-21T04:33:11.076+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49676:172.31.5.112:12000] 2023-10-21T04:33:11.101+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49678:172.31.5.112:12000] 2023-10-21T04:33:11.112+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49680:172.31.5.112:12000] 2023-10-21T04:33:11.178+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49686:172.31.5.112:12000] 2023-10-21T04:33:11.190+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49688:172.31.5.112:12000] 2023-10-21T04:33:11.217+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49690:172.31.5.112:12000] 2023-10-21T04:33:11.237+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49692:172.31.5.112:12000] 2023-10-21T04:33:11.240+05:30 [Info] progress stats collection done. 2023-10-21T04:33:11.240+05:30 [Info] send progress stats to clients 2023-10-21T04:33:11.243+05:30 [Info] Indexer::ReadMemstats Time Taken 204.626µs 2023-10-21T04:33:11.362+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 1949031200982 23 61684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:11.365+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 52522903947059 28 61684 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:11.365+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 245782287380001 25 61684 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:33:11.369+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 205201940883916 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.374+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 93079113162334 18 61684 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:11.374+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 244791485011286 23 61684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:11.376+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 167580527121081 23 61684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:11.376+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 177815184127237 17 61684 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:11.385+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 134409829206048 26 61684 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:11.385+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 160381752747972 18 61684 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:11.386+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 58522600746999 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.386+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 142387448675642 30 61684 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:33:11.386+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 54236768606397 20 61684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:11.389+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 90115353725987 21 61684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:11.392+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 161230548567164 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.394+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 85302001459467 21 61684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:11.398+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 213108110408291 23 61684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:11.398+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 123957315902602 28 61684 172.31.5.112:9000. HWT [27-28,28]. 2023-10-21T04:33:11.401+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 163460438181870 34 61684 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:33:11.403+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 151273673850546 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.403+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 150690956328758 26 61684 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:11.405+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 147946970349859 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.408+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 258671969827141 19 61684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:11.410+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 53829314402679 32 61684 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:33:11.412+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 68847799765163 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.412+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 19989312661772 15 61684 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:33:11.415+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 232028101514735 19 61684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:11.415+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 114823206239184 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.419+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 8305763409275 21 61684 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:33:11.419+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 25510109770064 28 61684 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:33:11.421+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 124811039198764 18 61684 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:33:11.421+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:33:11.425+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 207489375637709 12 61684 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:33:11.428+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 33310272161722 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.430+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 100746255343609 19 61684 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:33:11.430+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 107407811917992 20 61684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:11.433+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 117298435979365 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.435+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 228857796432725 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.436+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 203503670520010 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.438+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 47156288300713 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.440+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 158357249947806 21 61684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:11.445+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 104136033125373 29 61684 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:11.450+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 236904976023311 22 61684 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:11.450+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 124310544527292 21 61684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:11.452+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 196269910725961 26 61684 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:33:11.452+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 246959094981553 34 61684 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:33:11.455+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 65885584526743 20 61684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:11.455+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 276463747117997 23 61684 172.31.5.112:9000. HWT [22-23,23]. 2023-10-21T04:33:11.458+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 79077905592013 17 61684 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:11.458+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 172365063201867 21 61684 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:33:11.461+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 212884822297038 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.462+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 246582249071123 20 61684 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:33:11.464+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 213334533059126 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.466+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 142324128597367 22 61684 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:33:11.475+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 219528837936411 25 61684 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:33:11.475+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 55145461136105 16 61684 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:33:11.475+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 254944616021293 24 61684 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:33:11.480+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 80145536651832 17 61684 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:11.480+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 194568213794435 29 61684 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:33:11.482+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 25511316571199 16 61684 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:33:11.488+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 36300629322478 32 61684 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:33:11.492+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 InstanceIds [8965923595543532942 2670343739772083055 17434304260397056806 4083560586502645495 3464125101225762400 15864358084358521096] 2023-10-21T04:33:11.492+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 61684 2023-10-21T04:33:11.492+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 61684 2023-10-21T04:33:11.492+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:33:11.492+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:33:11.492+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:33:11.492+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 61684 2023-10-21T04:33:11.493+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 61684 STREAM_ACTIVE 2023-10-21T04:33:11.500+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 223881084081370 17 61684 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:11.500+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 25657368013413 17 61684 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:33:11.500+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 106620692996001 23 61684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:11.501+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 279733370822238 23 61684 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:33:11.566+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:33:11.566+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:33:11.618+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60266 2023-10-21T04:33:11.620+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60268 2023-10-21T04:33:11.630+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60270 2023-10-21T04:33:11.632+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60272 2023-10-21T04:33:11.634+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60274 2023-10-21T04:33:12.493+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:33:12.493+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:33:12.565+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:33:12.657+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:33:12.657+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 4 NodeUUIDs: [7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:33:12.657+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 4 NodeUUIDs: [7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:33:12.657+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:33:12.658+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:33:12.659+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:33:12.659+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:33:13.487+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:33:13.569+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17434304260397056806:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 2023-10-21T04:33:13.569+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[7 8 1 2 3 4 5 6] 17434304260397056806:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 2023-10-21T04:33:13.569+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[7 8 1 2 3 4 5 6] 17434304260397056806:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:33:13.569+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[7 8 1 2 3 4 5 6] 17434304260397056806:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0] 15864358084358521096:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:33:13.570+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8965923595543532942:[7 8 1 2 3 4 5 6] 17434304260397056806:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0] 4083560586502645495:[0] 15864358084358521096:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:33:13.570+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2670343739772083055:[0] 8965923595543532942:[7 8 1 2 3 4 5 6] 17434304260397056806:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 {Index inst : [partitions]}: map[3464125101225762400:[0] 4083560586502645495:[0] 15864358084358521096:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:33:14.101+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:14.108+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:14.109+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:14.111+05:30 [Info] switched currmeta from 33 -> 33 force true 2023-10-21T04:33:14.489+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:33:15.322+05:30 [Info] RebalanceServiceManager::updateShardTokenMap Found 1 tokens. Cleaning up. 2023-10-21T04:33:15.322+05:30 [Info] RebalanceServiceManager::cleanupShardTokenForDest Cleaning up token: ShardTokenb2:f8:fb:ca:53:2e:1c:c4 on dest as ShardTokenDropOnSource is not posted for this token 2023-10-21T04:33:15.322+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenb2:f8:fb:ca:53:2e:1c:c4, dropIndexes: true 2023-10-21T04:33:15.324+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16961575949607441659 instance id 11304674148443950662 real instance id 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:33:15.324+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16961575949607441659 instance 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:33:15.324+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16961575949607441659 instance id 15864358084358521096 2023-10-21T04:33:15.324+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16961575949607441659 2023-10-21T04:33:15.335+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15864358084358521096 &{1} 2023-10-21T04:33:15.335+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15864358084358521096 2023-10-21T04:33:15.335+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16961575949607441659, instId: 15864358084358521096 2023-10-21T04:33:15.336+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:33:15.336+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:15.336+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:33:15.336+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15864358084358521096] 2023-10-21T04:33:15.337+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15864358084358521096] 2023-10-21T04:33:15.337+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:15.337+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15864358084358521096] 2023-10-21T04:33:15.337+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:33:15.337+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:33:15.337+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:33:15.337+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:33:15.337+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:33:15.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex stopped 2023-10-21T04:33:15.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.337+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.340+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.340+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.345+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.341+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.341+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.341+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.341+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.341+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] [16787445029868804812 2925063509317026816]} 2:{2 1 [:9123] [16787445029868804812 2925063509317026816]} 3:{3 1 [:9123] [16787445029868804812 2925063509317026816]} 4:{4 1 [:9123] [16787445029868804812 2925063509317026816]} 5:{5 1 [:9123] [16787445029868804812 2925063509317026816]} 6:{6 1 [:9123] [16787445029868804812 2925063509317026816]} 7:{7 1 [:9123] [16787445029868804812 2925063509317026816]} 8:{8 1 [:9123] [16787445029868804812 2925063509317026816]}] 8 KEY 0}] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:15.342+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.342+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.342+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.346+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:33:15.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:33:15.354+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.357+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.359+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.361+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.361+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.361+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.361+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex closed 2023-10-21T04:33:15.362+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.362+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.363+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.363+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:33:15.363+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:33:15.363+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:33:15.363+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:33:15.363+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:33:15.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex stopped 2023-10-21T04:33:15.363+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.362+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.363+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:33:15.360+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.364+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:33:15.359+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] [16787445029868804812 2925063509317026816]} 2:{2 1 [:9123] [16787445029868804812 2925063509317026816]} 3:{3 1 [:9123] [16787445029868804812 2925063509317026816]} 4:{4 1 [:9123] [16787445029868804812 2925063509317026816]} 5:{5 1 [:9123] [16787445029868804812 2925063509317026816]} 6:{6 1 [:9123] [16787445029868804812 2925063509317026816]} 7:{7 1 [:9123] [16787445029868804812 2925063509317026816]} 8:{8 1 [:9123] [16787445029868804812 2925063509317026816]}] 8 KEY 0}] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:15.364+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 28.868604ms 2023-10-21T04:33:15.364+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15864358084358521096 2023-10-21T04:33:15.369+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [15864358084358521096] 2023-10-21T04:33:15.363+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.369+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.370+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.372+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:33:15.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex closed 2023-10-21T04:33:15.375+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:33:15.375+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 61684 2023-10-21T04:33:15.375+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 11.003572ms 2023-10-21T04:33:15.375+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 39.960256ms 2023-10-21T04:33:15.364+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:33:15.384+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:33:15.385+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:33:15.385+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.385+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:33:15.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex stopped 2023-10-21T04:33:15.386+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.387+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.386+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:33:15.387+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:33:15.387+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.387+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.388+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:33:15.388+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:33:15.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex stopped 2023-10-21T04:33:15.389+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.389+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.390+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.390+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.389+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:33:15.390+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:33:15.391+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:33:15.391+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:33:15.391+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex stopped 2023-10-21T04:33:15.391+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.391+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:33:15.392+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:33:15.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex stopped 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.392+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:33:15.392+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:33:15.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex stopped 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.392+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:33:15.392+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:33:15.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex stopped 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.392+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:33:15.393+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:33:15.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex stopped 2023-10-21T04:33:15.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.392+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex ... 2023-10-21T04:33:15.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex ... 2023-10-21T04:33:15.397+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.397+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.398+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.402+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.400+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.404+05:30 [Info] switched currmeta from 33 -> 34 force true 2023-10-21T04:33:15.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:33:15.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex closed 2023-10-21T04:33:15.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:33:15.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex closed 2023-10-21T04:33:15.405+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.405+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.405+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.405+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:33:15.406+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:33:15.406+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:33:15.406+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:33:15.406+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:33:15.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex stopped 2023-10-21T04:33:15.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.406+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.409+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.409+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:33:15.409+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex closed 2023-10-21T04:33:15.409+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.409+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.410+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.410+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.410+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:33:15.410+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:33:15.410+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:33:15.410+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:33:15.410+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:33:15.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex stopped 2023-10-21T04:33:15.411+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.411+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.411+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:33:15.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex closed 2023-10-21T04:33:15.412+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.412+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.412+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.413+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.413+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:33:15.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex closed 2023-10-21T04:33:15.414+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.414+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.414+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.414+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:33:15.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:33:15.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex ... 2023-10-21T04:33:15.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex ... 2023-10-21T04:33:15.416+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.417+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.417+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.417+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:33:15.411+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.417+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:33:15.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex closed 2023-10-21T04:33:15.419+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.419+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.419+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.419+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:33:15.425+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.427+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.427+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.428+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:33:15.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex closed 2023-10-21T04:33:15.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index destroyed 2023-10-21T04:33:15.428+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.428+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.428+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.428+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 8 Destroy Done 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:33:15.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex stopped 2023-10-21T04:33:15.428+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:33:15.429+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:33:15.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex stopped 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex stopped 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex stopped 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.429+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.429+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:33:15.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:33:15.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex closed 2023-10-21T04:33:15.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:33:15.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex closed 2023-10-21T04:33:15.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.431+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex ... 2023-10-21T04:33:15.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex ... 2023-10-21T04:33:15.432+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex stopped 2023-10-21T04:33:15.432+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.432+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.432+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.432+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.433+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.433+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.433+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 109.265048ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:15.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.433+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex ... 2023-10-21T04:33:15.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex ... 2023-10-21T04:33:15.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex ... 2023-10-21T04:33:15.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex ... 2023-10-21T04:33:15.444+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.447+05:30 [Info] switched currmeta from 34 -> 35 force true 2023-10-21T04:33:15.447+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index destroyed 2023-10-21T04:33:15.450+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.450+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 6 Destroy Done 2023-10-21T04:33:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:33:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex closed 2023-10-21T04:33:15.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:33:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex closed 2023-10-21T04:33:15.450+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.450+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:33:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex closed 2023-10-21T04:33:15.450+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.450+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.452+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:33:15.452+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex closed 2023-10-21T04:33:15.452+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.452+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.453+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:33:15.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex closed 2023-10-21T04:33:15.454+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.454+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.455+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex ... 2023-10-21T04:33:15.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex ... 2023-10-21T04:33:15.458+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4083560586502645495 &{1} 2023-10-21T04:33:15.459+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4083560586502645495 2023-10-21T04:33:15.459+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 4790214984019693894, instId: 4083560586502645495 2023-10-21T04:33:15.460+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:33:15.460+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:15.460+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4083560586502645495 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:33:15.460+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4083560586502645495] 2023-10-21T04:33:15.460+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4083560586502645495] 2023-10-21T04:33:15.461+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:15.461+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4083560586502645495] 2023-10-21T04:33:15.461+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [16787445029868804812 2925063509317026816]}] 1 SINGLE 0}] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:15.461+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4083560586502645495 Defn: DefnId: 4790214984019693894 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [16787445029868804812 2925063509317026816]}] 1 SINGLE 0}] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:15.462+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.358663ms 2023-10-21T04:33:15.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex ... 2023-10-21T04:33:15.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex ... 2023-10-21T04:33:15.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex ... 2023-10-21T04:33:15.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex ... 2023-10-21T04:33:15.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.469+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index destroyed 2023-10-21T04:33:15.469+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.470+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 3 Destroy Done 2023-10-21T04:33:15.470+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4083560586502645495, PartitionId 0, IndexDefnId 4790214984019693894. 2023-10-21T04:33:15.471+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.471+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.471+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex ... 2023-10-21T04:33:15.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex ... 2023-10-21T04:33:15.471+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4083560586502645495 2023-10-21T04:33:15.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex ... 2023-10-21T04:33:15.471+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [4083560586502645495] 2023-10-21T04:33:15.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex] 2023-10-21T04:33:15.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex ... 2023-10-21T04:33:15.478+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:33:15.478+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 61684 2023-10-21T04:33:15.478+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 16.369988ms 2023-10-21T04:33:15.478+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 18.920834ms 2023-10-21T04:33:15.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex ... 2023-10-21T04:33:15.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex ... 2023-10-21T04:33:15.485+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index destroyed 2023-10-21T04:33:15.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex ... 2023-10-21T04:33:15.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex ... 2023-10-21T04:33:15.491+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.492+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 7 Destroy Done 2023-10-21T04:33:15.493+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:33:15.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex ... 2023-10-21T04:33:15.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex ... 2023-10-21T04:33:15.496+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex sucessfully destroyed 2023-10-21T04:33:15.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex ... 2023-10-21T04:33:15.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index destroyed 2023-10-21T04:33:15.501+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.501+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 5 Destroy Done 2023-10-21T04:33:15.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.505+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:33:15.505+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex] 2023-10-21T04:33:15.505+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex] 2023-10-21T04:33:15.505+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex 2023-10-21T04:33:15.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex stopped 2023-10-21T04:33:15.505+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.505+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.505+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.505+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.509+05:30 [Info] switched currmeta from 35 -> 36 force true 2023-10-21T04:33:15.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index destroyed 2023-10-21T04:33:15.515+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.515+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 2 Destroy Done 2023-10-21T04:33:15.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex ... 2023-10-21T04:33:15.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex ... 2023-10-21T04:33:15.517+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex] 2023-10-21T04:33:15.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex closed 2023-10-21T04:33:15.517+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.517+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.517+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.517+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex] 2023-10-21T04:33:15.518+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 69.882329ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:15.519+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.520+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index destroyed 2023-10-21T04:33:15.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex ... 2023-10-21T04:33:15.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex ... 2023-10-21T04:33:15.530+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16337157693634799600 instance id 4708891605960345271 real instance id 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:33:15.530+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16337157693634799600 instance 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:33:15.531+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16337157693634799600 instance id 10032435514095102923 2023-10-21T04:33:15.531+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16337157693634799600 2023-10-21T04:33:15.534+05:30 [Info] switched currmeta from 36 -> 37 force true 2023-10-21T04:33:15.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex sucessfully destroyed 2023-10-21T04:33:15.534+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:33:15.534+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex] 2023-10-21T04:33:15.534+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex] 2023-10-21T04:33:15.534+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex 2023-10-21T04:33:15.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex stopped 2023-10-21T04:33:15.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index destroyed 2023-10-21T04:33:15.538+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.538+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 1 Destroy Done 2023-10-21T04:33:15.539+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:33:15.539+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 4 Destroy Done 2023-10-21T04:33:15.544+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.544+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.547+05:30 [Info] switched currmeta from 37 -> 37 force true 2023-10-21T04:33:15.550+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex] 2023-10-21T04:33:15.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex closed 2023-10-21T04:33:15.550+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.551+05:30 [Info] Indexer::cleanupIndexData IndexInst 4083560586502645495 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex ... 2023-10-21T04:33:15.551+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex ... 2023-10-21T04:33:15.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex ... 2023-10-21T04:33:15.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex ... 2023-10-21T04:33:15.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4083560586502645495_0.index destroyed 2023-10-21T04:33:15.574+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10032435514095102923 &{1} 2023-10-21T04:33:15.574+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10032435514095102923 2023-10-21T04:33:15.575+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16337157693634799600, instId: 10032435514095102923 2023-10-21T04:33:15.576+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4083560586502645495, PartitionId 0, IndexDefnId 4790214984019693894. 2023-10-21T04:33:15.576+05:30 [Info] Indexer::cleanupIndexData IndexInst 4083560586502645495 Partition 0 Destroy Done 2023-10-21T04:33:15.576+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:33:15.577+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:33:15.577+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:15.577+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10032435514095102923] 2023-10-21T04:33:15.578+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.578+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.578+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.578+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:15.580+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.580+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.580+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.580+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.580+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.580+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.580+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.580+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.581+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.581+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:15.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:15.580+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:15.581+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:15.581+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:15.581+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:33:15.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:33:15.581+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.581+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] [16787445029868804812 2925063509317026816]} 2:{2 1 [:9123] [16787445029868804812 2925063509317026816]} 3:{3 1 [:9123] [16787445029868804812 2925063509317026816]} 4:{4 1 [:9123] [16787445029868804812 2925063509317026816]} 5:{5 1 [:9123] [16787445029868804812 2925063509317026816]} 6:{6 1 [:9123] [16787445029868804812 2925063509317026816]} 7:{7 1 [:9123] [16787445029868804812 2925063509317026816]} 8:{8 1 [:9123] [16787445029868804812 2925063509317026816]}] 8 KEY 0} 2023-10-21T04:33:15.582+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 7.964251ms 2023-10-21T04:33:15.582+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.583+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.584+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.584+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:15.582+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.581+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.584+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.584+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.584+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:15.582+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.585+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.582+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.585+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.586+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:15.582+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.589+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.589+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.583+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10032435514095102923 2023-10-21T04:33:15.583+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 8.173878ms 2023-10-21T04:33:15.583+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:15.580+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.591+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.592+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.592+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:15.591+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:15.592+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:15.592+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:15.592+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:33:15.592+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:33:15.592+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.592+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.592+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.592+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.593+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:15.593+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:15.593+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:15.594+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:33:15.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:33:15.594+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.594+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.595+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.595+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.595+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:15.596+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:15.596+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:15.596+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:33:15.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:33:15.596+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.596+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.596+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.596+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.596+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:15.596+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:15.596+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:15.596+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:33:15.596+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:33:15.596+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.597+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:15.597+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:15.597+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:15.597+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:33:15.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.597+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:15.597+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:15.597+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:15.597+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:33:15.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.597+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.598+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.598+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:15.598+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:15.598+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:15.598+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:33:15.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:33:15.598+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.598+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.598+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.598+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.605+05:30 [Info] switched currmeta from 37 -> 38 force true 2023-10-21T04:33:15.632+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:33:15.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:33:15.634+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.635+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.635+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.635+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:15.636+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:15.636+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:15.637+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:15.637+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:33:15.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:33:15.637+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.637+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.637+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.637+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.650+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:33:15.650+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4083560586502645495:[0] 15864358084358521096:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:33:15.671+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 141.338896ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:15.714+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.714+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.718+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:33:15.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:33:15.718+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.718+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:33:15.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:33:15.728+05:30 [Info] switched currmeta from 38 -> 39 force true 2023-10-21T04:33:15.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:33:15.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:33:15.736+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3464125101225762400 &{1} 2023-10-21T04:33:15.741+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3464125101225762400 2023-10-21T04:33:15.742+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 11276338709617092948, instId: 3464125101225762400 2023-10-21T04:33:15.742+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:33:15.743+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:15.744+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:33:15.745+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3464125101225762400] 2023-10-21T04:33:15.745+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3464125101225762400] 2023-10-21T04:33:15.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:33:15.746+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:15.748+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3464125101225762400] 2023-10-21T04:33:15.760+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.762+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.762+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Destroy Done 2023-10-21T04:33:15.761+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.801+05:30 [Info] switched currmeta from 39 -> 40 force true 2023-10-21T04:33:15.804+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:33:15.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:33:15.812+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.812+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.812+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.813+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:15.813+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:15.813+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:15.813+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:15.813+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:33:15.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:33:15.814+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.814+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.814+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.814+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:33:15.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:15.821+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:15.821+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:15.821+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:15.821+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:33:15.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.821+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.830+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:33:15.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:33:15.837+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.837+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:33:15.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:33:15.836+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.839+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.840+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.840+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.840+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.840+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:33:15.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:33:15.846+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.847+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Close Done 2023-10-21T04:33:15.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.844+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.847+05:30 [Info] switched currmeta from 40 -> 40 force true Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:33:15.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:33:15.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:33:15.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:33:15.848+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.848+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.848+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.848+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:33:15.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:33:15.849+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:33:15.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:33:15.849+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.850+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.850+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.850+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Destroy Done 2023-10-21T04:33:15.850+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:15.851+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:15.851+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:15.851+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:33:15.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:33:15.851+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.851+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.851+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.851+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:33:15.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:33:15.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:15.851+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:15.852+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:15.852+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:15.852+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:33:15.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:33:15.852+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.852+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.852+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.852+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.852+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:15.853+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:15.853+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:15.853+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:33:15.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:33:15.853+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.853+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.853+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.853+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.853+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:15.854+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:15.854+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:15.854+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:33:15.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.854+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:15.854+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:15.854+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:15.854+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:33:15.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.854+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:33:15.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:33:15.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:33:15.874+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.874+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Destroy Done 2023-10-21T04:33:15.889+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:33:15.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:33:15.893+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.891+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.892+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:33:15.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:33:15.893+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.893+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:33:15.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:33:15.893+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:33:15.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:33:15.894+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.894+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.894+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:33:15.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:33:15.895+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.895+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.897+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:15.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:33:15.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:33:15.897+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:15.897+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:15.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:33:15.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:33:15.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:33:15.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:33:15.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:33:15.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:33:15.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:33:15.905+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:33:15.905+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Destroy Done 2023-10-21T04:33:15.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:33:15.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:33:15.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:33:15.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:33:15.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:33:15.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:33:15.910+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.910+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Destroy Done 2023-10-21T04:33:15.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:33:15.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:33:15.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:33:15.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:33:15.916+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:33:15.916+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Destroy Done 2023-10-21T04:33:15.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:33:15.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:33:15.922+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.922+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Destroy Done 2023-10-21T04:33:15.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:33:15.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:33:15.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:33:15.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:33:15.927+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:33:15.927+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Destroy Done 2023-10-21T04:33:15.927+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:15.927+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:33:15.927+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:15.927+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:15.927+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:15.928+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:15.928+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0046acf00 }] 2023-10-21T04:33:15.928+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:15.928+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:33:15.928+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:33:15.928+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:33:15.928+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:33:15.928+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:33:15.928+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 186.7187ms 2023-10-21T04:33:15.928+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:15.928+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3464125101225762400 2023-10-21T04:33:15.929+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1] 2023-10-21T04:33:15.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:33:15.929+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:33:15.929+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:33:15.931+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 0 1949031200982 23 2023-10-21T04:33:15.931+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.931+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 1 245782287380001 25 2023-10-21T04:33:15.931+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.932+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 2 52522903947059 28 2023-10-21T04:33:15.932+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.938+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 31 207489375637709 12 2023-10-21T04:33:15.938+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 27 232028101514735 19 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 3 205201940883916 24 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 7 177815184127237 17 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 8 54236768606397 21 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 19 150690956328758 26 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 28 25510109770064 28 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 12 142387448675642 30 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 11 58522600746999 24 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 16 123957315902602 28 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 20 151273673850546 25 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 4 93079113162334 18 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 22 258671969827141 19 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 32 33310272161722 24 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 5 244791485011286 23 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.939+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 15 85302001459467 21 2023-10-21T04:33:15.939+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 6 167580527121081 23 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 17 213108110408291 23 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 9 134409829206048 26 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 18 163460438181870 34 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 10 160381752747972 18 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 25 114823206239184 25 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 30 124811039198764 18 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 26 68847799765163 25 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 14 161230548567164 25 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 23 53829314402679 32 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 33 107407811917992 20 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 13 90115353725987 21 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 24 19989312661772 15 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 21 147946970349859 24 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.940+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 29 8305763409275 21 2023-10-21T04:33:15.940+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1] 2023-10-21T04:33:15.941+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 61684 2023-10-21T04:33:15.941+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 12.724699ms 2023-10-21T04:33:15.941+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 199.542148ms 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 50 246582249071123 20 2023-10-21T04:33:15.941+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 40 104136033125373 29 2023-10-21T04:33:15.941+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 52 142324128597367 22 2023-10-21T04:33:15.941+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 37 203503670520010 24 2023-10-21T04:33:15.941+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 42 124310544527292 21 2023-10-21T04:33:15.941+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 41 236904976023311 22 2023-10-21T04:33:15.941+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 39 158357249947806 21 2023-10-21T04:33:15.941+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.941+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 44 246959094981553 34 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 45 276463747117997 23 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 35 117298435979365 24 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 38 47156288300713 24 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 43 196269910725961 26 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 46 65885584526743 20 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 47 172365063201867 21 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 49 212884822297038 25 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 54 55145461136105 16 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 34 100746255343609 19 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 36 228857796432725 25 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 51 213334533059126 25 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 53 254944616021293 24 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.942+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 55 219528837936411 25 2023-10-21T04:33:15.942+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 56 194568213794435 29 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 48 79077905592013 17 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 59 106620692996001 23 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 60 279733370822238 23 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 61 223881084081370 17 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 62 36300629322478 32 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 63 25657368013413 17 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 57 80145536651832 17 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.943+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 58 25511316571199 16 2023-10-21T04:33:15.943+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:33:15.960+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.960+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.965+05:30 [Info] switched currmeta from 40 -> 40 force true 2023-10-21T04:33:15.973+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 269.567578ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:15.974+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:33:15.987+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:33:15.987+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:33:15.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex stopped 2023-10-21T04:33:15.987+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:15.987+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:15.987+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:15.987+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:15.992+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:15.992+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:15.994+05:30 [Info] switched currmeta from 40 -> 41 force true 2023-10-21T04:33:16.003+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:16.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:33:16.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex closed 2023-10-21T04:33:16.005+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:16.005+05:30 [Info] Indexer::cleanupIndexData IndexInst 3464125101225762400 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:16.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:33:16.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:33:16.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:33:16.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index destroyed 2023-10-21T04:33:16.011+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:33:16.011+05:30 [Info] Indexer::cleanupIndexData IndexInst 3464125101225762400 Partition 0 Destroy Done 2023-10-21T04:33:16.140+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:33:16.494+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:33:16.726+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2925063509317026816 2023-10-21T04:33:16.726+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 16787445029868804812 2023-10-21T04:33:16.726+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 16787445029868804812 2023-10-21T04:33:16.726+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2925063509317026816 2023-10-21T04:33:16.726+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [16787445029868804812 2925063509317026816], elapsed: 752.056081ms 2023-10-21T04:33:16.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:33:16.726+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:33:16.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:33:16.727+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper stopped 2023-10-21T04:33:16.727+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': stopped 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:33:16.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:33:16.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:33:16.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:33:16.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:33:16.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:33:16.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:33:16.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:33:16.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:33:16.732+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:33:16.732+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:33:16.732+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:33:16.732+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:33:16.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:33:16.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shutdown completed 2023-10-21T04:33:16.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:33:16.732+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:33:16.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:33:16.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:33:16.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance destroyed 2023-10-21T04:33:16.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:33:16.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:33:16.734+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:33:16.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:33:16.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:33:16.735+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:33:16.736+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:33:16.736+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:33:16.736+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:33:16.736+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:33:16.736+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:33:16.736+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:33:16.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:33:16.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:33:16.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:33:16.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:33:16.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:33:16.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shutdown completed 2023-10-21T04:33:16.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:33:16.737+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:33:16.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:33:16.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:33:16.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance destroyed 2023-10-21T04:33:16.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:33:16.739+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Cleaned up Tenant with name:bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:33:16.739+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [16787445029868804812 2925063509317026816], elapsed(sec): 0.764677509 2023-10-21T04:33:16.770+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:33:16.770+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:33:16.771+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 2.125µs 2023-10-21T04:33:16.771+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:33:16.771+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:33:16.771+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:33:16.773+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:33:16.773+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:33:16.773+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:33:16.774+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:33:16.775+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 361.956µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:16.775+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:33:16.775+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:33:16.775+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 7.209680552s in RebalanceServiceManager::recoverRebalance: 2023-10-21T04:33:16.775+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{18194302e3f10df59bd076e1c4142d0b 7060000 } 2023-10-21T04:33:17.495+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:33:17.532+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:33:19.114+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:19.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:19.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:19.122+05:30 [Info] switched currmeta from 41 -> 41 force true 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.132+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:28.133+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:33:30.411+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43970" doAuth() ... 2023-10-21T04:33:30.414+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:43970" doReceive() ... 2023-10-21T04:33:32.831+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:32.831+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:32.843+05:30 [Info] switched currmeta from 41 -> 44 force true 2023-10-21T04:33:32.875+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:32.875+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:32.877+05:30 [Info] switched currmeta from 44 -> 45 force true 2023-10-21T04:33:32.924+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:32.924+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:32.968+05:30 [Info] switched currmeta from 45 -> 45 force true 2023-10-21T04:33:34.160+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:34.165+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:34.165+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:34.166+05:30 [Info] switched currmeta from 45 -> 45 force true 2023-10-21T04:33:42.660+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. err: operation_canceled 2023-10-21T04:33:42.660+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:33:42.660+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:33:42.660+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:33:42.660+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:33:42.668+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:33:42.680+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:33:43.455+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 7147407900589480727 2023-10-21T04:33:43.455+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 279.846µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:43.521+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45744 2023-10-21T04:33:43.525+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45744 2023-10-21T04:33:43.551+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45744 2023-10-21T04:33:43.610+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 7147407900589480727 2023-10-21T04:33:43.610+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 304.987µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:43.672+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:43.672+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:43.674+05:30 [Info] switched currmeta from 45 -> 48 force true 2023-10-21T04:33:43.689+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:43.691+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:43.697+05:30 [Info] switched currmeta from 48 -> 48 force true 2023-10-21T04:33:43.839+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:43.839+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:43.842+05:30 [Info] switched currmeta from 48 -> 53 force true 2023-10-21T04:33:43.885+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:43.885+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:43.921+05:30 [Info] switched currmeta from 53 -> 54 force true 2023-10-21T04:33:43.937+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:43.937+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:43.943+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:33:44.089+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60924 2023-10-21T04:33:44.108+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 12.630653ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:44.156+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 7147407900589480727. No update is performed. 2023-10-21T04:33:44.156+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 232.888µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:44.172+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:60924. Reason = EOF. Cleaning Pipe. 2023-10-21T04:33:44.172+05:30 [Info] messageListener.start(): message channel closed. Remove peer 87:93:6c:9e:1e:72:3e:7c as follower. 2023-10-21T04:33:44.262+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:44.273+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:44.273+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:44.277+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:33:44.302+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60938 2023-10-21T04:33:44.318+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 11.677874ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:44.328+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:60938. Reason = EOF. Cleaning Pipe. 2023-10-21T04:33:44.329+05:30 [Info] messageListener.start(): message channel closed. Remove peer 53:86:7d:71:17:c3:34:5b as follower. 2023-10-21T04:33:44.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9574291805315098, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:33:44.609+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:33:44.674+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:33:44.674+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:33:44.680+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:33:44.702+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:33:44.702+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:33:44.702+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:33:44.710+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:33:44.755+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:33:44.755+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:33:44.755+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:33:44.761+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60954 2023-10-21T04:33:44.761+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:33:44.789+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.948192ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:44.790+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:33:44.790+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:33:44.790+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:33:44.802+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:33:44.820+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:33:44.856+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60962 2023-10-21T04:33:44.863+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:33:44.863+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:33:44.863+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:33:44.863+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:33:44.863+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:33:44.863+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:33:44.863+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:33:44.863+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:33:44.863+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:33:44.863+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:33:44.863+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:52200->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:33:44.863+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:52660->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:33:44.864+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:60954->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:33:44.864+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:57330->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:33:44.865+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:60954. Reason = EOF. Cleaning Pipe. 2023-10-21T04:33:44.865+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6e:73:69:6d:0:d0:c3:b0 as follower. 2023-10-21T04:33:44.886+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 21.671673ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:44.953+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:60962. Reason = EOF. Cleaning Pipe. 2023-10-21T04:33:44.953+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5c:f6:93:33:ca:91:c7:a1 as follower. 2023-10-21T04:33:45.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:45.125+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:45.136+05:30 [Info] switched currmeta from 54 -> 55 force true 2023-10-21T04:33:45.186+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:45.187+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:45.190+05:30 [Info] switched currmeta from 55 -> 56 force true 2023-10-21T04:33:45.290+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:33:45.387+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:33:45.487+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8358105493938612, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:33:45.657+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:45.657+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:45.661+05:30 [Info] switched currmeta from 56 -> 57 force true 2023-10-21T04:33:45.692+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:45.692+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:45.696+05:30 [Info] switched currmeta from 57 -> 58 force true 2023-10-21T04:33:46.641+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9570541924043341912:[0]]. New rollbacktime: 1697842771517812402, Old rollbacktime: 0 2023-10-21T04:33:46.776+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:33:46.870+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11645751077768222772:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:33:46.870+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11645751077768222772:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:33:46.870+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11645751077768222772:[0]]. New rollbacktime: 1697842615476970543, Old rollbacktime: 0 2023-10-21T04:33:49.215+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:49.221+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:49.221+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:49.225+05:30 [Info] switched currmeta from 58 -> 58 force true 2023-10-21T04:33:52.815+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 2670343739772083055 &{0} 2023-10-21T04:33:52.825+05:30 [Info] Indexer::handleDropIndex - IndexInstId 2670343739772083055 2023-10-21T04:33:52.825+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:33:52.825+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:52.825+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2670343739772083055 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:33:52.826+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [2670343739772083055] 2023-10-21T04:33:52.826+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [2670343739772083055] 2023-10-21T04:33:52.826+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:33:52.826+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [2670343739772083055] 2023-10-21T04:33:52.826+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [17406544375239214575 10170029854712240877]}] 1 SINGLE 0}] Async: false SessionId: 62731 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:52.826+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 2670343739772083055 Defn: DefnId: 13268712595133993155 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [17406544375239214575 10170029854712240877]}] 1 SINGLE 0}] Async: false SessionId: 62731 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:33:52.827+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.182058ms 2023-10-21T04:33:52.827+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 2670343739772083055, PartitionId 0, IndexDefnId 13268712595133993155. 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:52.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex] 2023-10-21T04:33:52.827+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex] 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex] 2023-10-21T04:33:52.827+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex 2023-10-21T04:33:52.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex stopped 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:52.827+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:52.827+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 2670343739772083055 2023-10-21T04:33:52.828+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [2670343739772083055] 2023-10-21T04:33:52.832+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:33:52.832+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 62731 2023-10-21T04:33:52.832+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.924222ms 2023-10-21T04:33:52.832+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 7.177276ms 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:52.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex] 2023-10-21T04:33:52.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex closed 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2/#primary/Mainstore#17434304260397056806:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:33:52.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex] 2023-10-21T04:33:52.835+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex] 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex] 2023-10-21T04:33:52.835+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex 2023-10-21T04:33:52.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex stopped 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:33:52.835+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:33:52.836+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:33:52.836+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:33:52.844+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:33:52.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex] 2023-10-21T04:33:52.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex closed 2023-10-21T04:33:52.844+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:33:52.844+05:30 [Info] Indexer::cleanupIndexData IndexInst 2670343739772083055 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:33:52.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex ... 2023-10-21T04:33:52.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex ... 2023-10-21T04:33:52.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:33:52.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/docIndex sucessfully destroyed 2023-10-21T04:33:52.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex ... 2023-10-21T04:33:52.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex ... 2023-10-21T04:33:52.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:33:52.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2670343739772083055_0.index destroyed 2023-10-21T04:33:52.854+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 2670343739772083055, PartitionId 0, IndexDefnId 13268712595133993155. 2023-10-21T04:33:52.854+05:30 [Info] Indexer::cleanupIndexData IndexInst 2670343739772083055 Partition 0 Destroy Done 2023-10-21T04:33:52.856+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 54.99795ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:33:52.881+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:52.881+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:52.896+05:30 [Info] switched currmeta from 58 -> 62 force true 2023-10-21T04:33:52.902+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:52.902+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:52.904+05:30 [Info] switched currmeta from 62 -> 62 force true 2023-10-21T04:33:54.246+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:33:54.252+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:33:54.252+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:33:54.254+05:30 [Info] switched currmeta from 62 -> 62 force true 2023-10-21T04:34:03.120+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 238] aggregate storage info: map[bucket_%2:map[kv:2328768 index:9477978] bucket_1:map[kv:2418880 index:7864504]] 2023-10-21T04:34:03.287+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 12599498416578472779 2023-10-21T04:34:03.287+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 134.395µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:03.340+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45744 2023-10-21T04:34:03.350+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45744 2023-10-21T04:34:03.399+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45744 2023-10-21T04:34:03.469+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12599498416578472779, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:34:03.469+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12599498416578472779, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:34:03.491+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 15165282789964834431, indexDefn DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:34:03.499+05:30 [Info] Indexer::handleCreateIndex InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:34:03.500+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15165282789964834431 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:34:03.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:34:03.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:34:03.506+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:03.507+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:03.512+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:34:03.512+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:03.512+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:03.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:34:03.514+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:03.514+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:03.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex 2023-10-21T04:34:03.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:34:03.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex 2023-10-21T04:34:03.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:34:03.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:34:03.527+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex 2023-10-21T04:34:03.527+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex 2023-10-21T04:34:03.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex started 2023-10-21T04:34:03.527+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:03.527+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:03.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:34:03.528+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex 2023-10-21T04:34:03.528+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex 2023-10-21T04:34:03.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex started 2023-10-21T04:34:03.528+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:03.528+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15165282789964834431 partitionId 0 WriterThreads 4 2023-10-21T04:34:03.529+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15165282789964834431 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index Index: 15165282789964834431 Partition: 0 2023-10-21T04:34:03.528+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:03.530+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15165282789964834431 2023-10-21T04:34:03.530+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:03.530+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:34:03.530+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:03.531+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:03.533+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 34.504ms 2023-10-21T04:34:03.534+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 34.550794ms 2023-10-21T04:34:03.534+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 15165282789964834431, partnShardIdMap: map[0:[17406544375239214575 10170029854712240877]], indexDefn DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:34:03.552+05:30 [Info] switched currmeta from 62 -> 67 force true 2023-10-21T04:34:03.554+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [15165282789964834431] &{0} 2023-10-21T04:34:03.554+05:30 [Info] Indexer::handleBuildIndex [15165282789964834431] 2023-10-21T04:34:03.555+05:30 [Info] bulkUpdateRState: Index instance 15165282789964834431 rstate moved to ACTIVE 2023-10-21T04:34:03.555+05:30 [Info] Indexer::handleBuildIndex Added Index: [15165282789964834431] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:34:03.555+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:03.555+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:03.555+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:03.555+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:34:03.555+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:03.555+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:03.555+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:03.556+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:34:03.561+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:34:03.561+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:03.561+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:03.562+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:34:03.562+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:03.562+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:03.562+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:34:03.562+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:34:03.562+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:03.562+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:03.562+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:34:03.562+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:34:03.562+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:34:03.562+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15165282789964834431 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:34:03.562+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:34:03.562+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [ InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 28627 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:34:03.562+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:34:03.562+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:34:03.564+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:03.565+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 10.882859ms 2023-10-21T04:34:03.565+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 11.023638ms 2023-10-21T04:34:03.565+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [15165282789964834431] 2023-10-21T04:34:03.579+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:34:03.579+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 12599498416578472779 2023-10-21T04:34:03.579+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 110.390784ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:03.581+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:34:03.596+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.331282ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:03.743+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:03.743+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:03.766+05:30 [Info] switched currmeta from 67 -> 71 force true 2023-10-21T04:34:03.780+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:33864" doAuth() ... 2023-10-21T04:34:03.780+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:33864" auth successful 2023-10-21T04:34:03.780+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:33864" +1 2023-10-21T04:34:03.850+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:03.850+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:03.864+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:34:03.865+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:34:03.864+05:30 [Info] switched currmeta from 71 -> 71 force true 2023-10-21T04:34:03.886+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:34:03.931+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33102 2023-10-21T04:34:03.953+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 17.117183ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:03.953+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:34:03.953+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:34:03.953+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:34:03.975+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:34:04.034+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:34:04.034+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:34:04.034+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:34:04.049+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33122 2023-10-21T04:34:04.049+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:34:04.106+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33130 2023-10-21T04:34:04.127+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 14.496918ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:04.138+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 86.24734ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:04.138+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:34:04.138+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:34:04.138+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:34:04.158+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:34:04.158+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33142 2023-10-21T04:34:04.179+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:34:04.179+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:34:04.179+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:34:04.179+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:34:04.179+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:34:04.179+05:30 [Info] DDLServiceMgr: processing create index token /indexing/ddl/commandToken/create/12599498416578472779/0 2023-10-21T04:34:04.190+05:30 [Info] DDLServiceMgr: Update Replica Count. Index Defn 12599498416578472779 replica Count {true 1 0 0} 2023-10-21T04:34:04.198+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 26.908537ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:04.201+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 24.897µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:04.240+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 38.776µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:04.257+05:30 [Info] DDLServiceMgr: Remove create index token 12599498416578472779. 2023-10-21T04:34:04.257+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:34:04.257+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:34:04.257+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:34:04.257+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:34:04.257+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:34:04.257+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:33122->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:34:04.257+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:52582->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:34:04.257+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33122. Reason = EOF. Cleaning Pipe. 2023-10-21T04:34:04.258+05:30 [Info] messageListener.start(): message channel closed. Remove peer 3c:b4:31:ae:f5:a7:70:fc as follower. 2023-10-21T04:34:04.259+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:53046->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:34:04.259+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:57746->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:34:04.259+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33130. Reason = EOF. Cleaning Pipe. 2023-10-21T04:34:04.260+05:30 [Info] messageListener.start(): message channel closed. Remove peer 63:f1:f0:f3:b5:6e:d5:e as follower. 2023-10-21T04:34:04.270+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 67.229µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:04.305+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33102. Reason = EOF. Cleaning Pipe. 2023-10-21T04:34:04.305+05:30 [Info] messageListener.start(): message channel closed. Remove peer c5:45:14:12:e0:70:6f:c3 as follower. 2023-10-21T04:34:04.348+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33142. Reason = EOF. Cleaning Pipe. 2023-10-21T04:34:04.348+05:30 [Info] messageListener.start(): message channel closed. Remove peer e4:66:ed:c5:68:98:93:7c as follower. 2023-10-21T04:34:04.454+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:34:04.522+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:04.547+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:04.547+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:04.551+05:30 [Info] switched currmeta from 71 -> 71 force true 2023-10-21T04:34:04.628+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:34:04.638+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:34:04.702+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:34:04.813+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33174 2023-10-21T04:34:04.838+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 21.772089ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:04.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 274143649547516 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 120349894144442 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.869+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 143687065777286 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.876+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 78446412189949 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.877+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 263088038522959 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 162049908746825 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 127746572208200 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 212967951483349 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.885+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 188740876924377 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.885+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 58733680850130 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 136568163328749 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.887+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 206082810619393 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.887+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 116493903186721 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.887+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 261951617336488 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.888+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 173457131919400 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 206088069380492 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 277338544220217 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.895+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33174. Reason = EOF. Cleaning Pipe. 2023-10-21T04:34:04.895+05:30 [Info] messageListener.start(): message channel closed. Remove peer a4:95:11:77:21:e:4e:20 as follower. 2023-10-21T04:34:04.904+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 274158831830257 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.904+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 125696667341112 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.904+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 245247859472006 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.904+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 250426459207023 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.909+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 191500187207978 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.909+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 118920785117980 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.909+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 106231614376481 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 35716737784090 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 34510046292135 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5906787184229 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.915+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 112482982607415 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.916+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 146375746634523 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.917+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 41101832875819 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.918+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 241391494503904 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.918+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 137256376714252 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.920+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 216238759475962 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.920+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 121450709505223 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.923+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 127702263378100 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.923+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 168446351781198 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.925+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 100008922352935 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.933+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 169268048765249 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.933+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 245900669851618 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.943+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 39045953044628 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.943+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 160399698726237 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.943+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 149033089785397 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.954+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 110286285845649 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.954+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 106297365959267 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.954+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 88718752743571 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 32274016651656 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 241607113601772 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 63737850123332 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 217237947818243 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 132470209691538 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 225662991941187 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 162870112426804 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 264001907139144 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 266604622117081 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 84581802233088 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.962+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 272523968790649 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.963+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 77598347829810 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.969+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 136038310030103 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.969+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 197957220074784 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.979+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 97677619676504 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.979+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 105415986286631 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.979+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 72072509155168 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.981+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 27479155597282 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:04.996+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 161870047388121 0 28627 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:34:05.002+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 InstanceIds [15165282789964834431] 2023-10-21T04:34:05.002+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 28627 2023-10-21T04:34:05.002+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:34:05.002+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:34:05.002+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_%2:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:34:05.002+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 63 Crc64: 1658324778457208291 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 0 29} { 1 93622f3f2a49 2 26 0 26} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 20 0 35} { 11 742f510ec848 0 0 0 0} { 12 356b002ef4d2 2 0 0 0} { 13 aba8a8ec75d9 2 0 0 0} { 14 9dc222bb5828 0 0 0 0} { 15 fc3ce9692839 0 0 0 0} { 16 df0d378bb686 0 0 0 0} { 17 bb6f9e2af98c 0 0 0 0} { 18 1f62ff0ce8a7 0 0 0 0} { 19 e3c2f424b56f 0 0 0 0} { 20 f95893c884f1 0 0 0 0} { 21 72520934d138 0 0 0 0} { 22 207bf377d91a 0 0 0 0} { 23 ae2b1ca2092a 0 0 0 0} { 24 609df9a38a21 0 0 0 0} { 25 6c286727431c 0 0 0 0} { 26 7cd57d93c40c 0 0 0 0} { 27 55f47f3ae65 0 0 0 0} { 28 8520c261c31b 0 0 0 0} { 29 664d7c38ba37 0 0 0 0} { 30 2561c425ef2b 0 0 0 0} { 31 db8b56481de0 0 0 0 0} { 32 6e757265d4c7 0 0 0 0} { 33 c4ab0271fafa 0 0 0 0} { 34 993378f62d4e 0 0 0 0} { 35 7425000bb0b4 0 0 0 0} { 36 5af5244a8d27 0 0 0 0} { 37 99f2c9eb2141 0 0 0 0} { 38 dfa5361397e2 0 0 0 0} { 39 878b78605e35 0 0 0 0} { 40 60ad48bd1263 0 0 0 0} { 41 50b071be0093 0 0 0 0} { 42 238318294094 0 0 0 0} { 43 91e1f7108d5d 0 0 0 0} { 44 c593a6b62103 0 0 0 0} { 45 644e06e43491 0 0 0 0} { 46 942127321b34 0 0 0 0} { 47 787b1fcabf92 0 0 0 0} { 48 f01bbbe52248 0 0 0 0} { 49 39f81fea5444 0 0 0 0} { 50 f279b9cbf8d9 0 0 0 0} { 51 1d5a614c4988 0 0 0 0} { 52 4ced3c46cd00 0 0 0 0} { 53 cd3d42793e43 0 0 0 0} { 54 dbbd8a2e72ec 0 0 0 0} { 55 f7dbee5a4079 0 0 0 0} { 56 469345f52232 0 0 0 0} { 57 b40a81c88520 0 0 0 0} { 58 7bb9e3252b17 0 0 0 0} { 59 58d657d2c558 0 0 0 0} { 60 418cb04e6760 0 0 0 0} { 61 5fe0126a9027 0 0 0 0} { 62 18fdfd4edfe2 0 0 0 0} HWT bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 0 29} { 1 93622f3f2a49 2 26 0 26} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 20 0 35} { 11 742f510ec848 0 0 0 0} { 12 356b002ef4d2 2 13 0 23} { 13 aba8a8ec75d9 2 14 0 22} { 14 9dc222bb5828 0 0 0 0} { 15 fc3ce9692839 0 0 0 0} { 16 df0d378bb686 0 0 0 0} { 17 bb6f9e2af98c 0 0 0 0} { 18 1f62ff0ce8a7 0 0 0 0} { 19 e3c2f424b56f 0 0 0 0} { 20 f95893c884f1 0 0 0 0} { 21 72520934d138 0 0 0 0} { 22 207bf377d91a 0 0 0 0} { 23 ae2b1ca2092a 0 0 0 0} { 24 609df9a38a21 0 0 0 0} { 25 6c286727431c 0 0 0 0} { 26 7cd57d93c40c 0 0 0 0} { 27 55f47f3ae65 0 0 0 0} { 28 8520c261c31b 0 0 0 0} { 29 664d7c38ba37 0 0 0 0} { 30 2561c425ef2b 0 0 0 0} { 31 db8b56481de0 0 0 0 0} { 32 6e757265d4c7 0 0 0 0} { 33 c4ab0271fafa 0 0 0 0} { 34 993378f62d4e 0 0 0 0} { 35 7425000bb0b4 0 0 0 0} { 36 5af5244a8d27 0 0 0 0} { 37 99f2c9eb2141 0 0 0 0} { 38 dfa5361397e2 0 0 0 0} { 39 878b78605e35 0 0 0 0} { 40 60ad48bd1263 0 0 0 0} { 41 50b071be0093 0 0 0 0} { 42 238318294094 0 0 0 0} { 43 91e1f7108d5d 0 0 0 0} { 44 c593a6b62103 0 0 0 0} { 45 644e06e43491 0 0 0 0} { 46 942127321b34 0 0 0 0} { 47 787b1fcabf92 0 0 0 0} { 48 f01bbbe52248 0 0 0 0} { 49 39f81fea5444 0 0 0 0} { 50 f279b9cbf8d9 0 0 0 0} { 51 1d5a614c4988 0 0 0 0} { 52 4ced3c46cd00 0 0 0 0} { 53 cd3d42793e43 0 0 0 0} { 54 dbbd8a2e72ec 0 0 0 0} { 55 f7dbee5a4079 0 0 0 0} { 56 469345f52232 0 0 0 0} { 57 b40a81c88520 0 0 0 0} { 58 7bb9e3252b17 0 0 0 0} { 59 58d657d2c558 0 0 0 0} { 60 418cb04e6760 0 0 0 0} { 61 5fe0126a9027 0 0 0 0} { 62 18fdfd4edfe2 0 0 0 0} { 63 93384eaca1d9 0 0 0 0} 2023-10-21T04:34:05.002+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:34:05.002+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:34:05.004+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:34:05.004+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:34:05.006+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:05.339+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:34:05.352+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:34:05.360+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15165282789964834431 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:34:05.360+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15165282789964834431 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.359µs SnapOpenDur 162.476µs 2023-10-21T04:34:05.360+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 15165282789964834431 INDEX_STATE_CATCHUP 2023-10-21T04:34:05.360+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 15165282789964834431 true 2023-10-21T04:34:05.360+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 15165282789964834431 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 28627 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] 2023-10-21T04:34:05.360+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15165282789964834431, PartitionId 0 SnapshotId 16 Creating recovery point ... 2023-10-21T04:34:05.361+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 28627 2023-10-21T04:34:05.361+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 28627 2023-10-21T04:34:05.362+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:05.362+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:05.362+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:05.363+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:05.364+05:30 [Info] send progress stats to clients 2023-10-21T04:34:05.364+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:34:05.364+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:34:05.364+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:34:05.364+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28627 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:34:05.367+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:34:05.374+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 2023-10-21T04:34:05.374+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28627 2023-10-21T04:34:05.376+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28627 2023-10-21T04:34:05.376+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:34:05.376+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:34:05.376+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 15165282789964834431 false 2023-10-21T04:34:05.377+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 15165282789964834431 2023-10-21T04:34:05.377+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:34:05.377+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 28627 2023-10-21T04:34:05.398+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:05.398+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:05.402+05:30 [Info] switched currmeta from 71 -> 72 force true 2023-10-21T04:34:05.422+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 58.259732ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:05.436+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:05.436+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:05.448+05:30 [Info] switched currmeta from 72 -> 73 force true 2023-10-21T04:34:05.449+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15165282789964834431, PartitionId 0 SnapshotId 16 Created recovery point (took 88.349644ms) 2023-10-21T04:34:05.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9610427639867062, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:34:06.004+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:34:06.005+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:34:06.005+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 15165282789964834431 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 28627, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 0 29} { 1 93622f3f2a49 2 26 0 26} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 18 0 18} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 18 0 18} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 32 0 32} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 20 0 20} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:34:06.005+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 15165282789964834431 INDEX_STATE_ACTIVE 2023-10-21T04:34:06.005+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:34:06.005+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:34:06.005+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:34:06.005+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 3 CurrMaint 0 New 0 2023-10-21T04:34:06.005+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 28627 2023-10-21T04:34:06.006+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:06.006+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:06.006+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:06.006+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 28627 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:06.006+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15165282789964834431 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:34:06.006+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:34:06.006+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:34:06.006+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:06.006+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 28627 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:06.006+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:34:06.007+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:20 1:15 2:17 3:16 4:16 5:21 6:15 7:16 8:11 9:16 10:28 11:12 12:16 13:15 14:5 15:14 16:8 17:6 18:7 19:21 20:13 21:20 22:17 23:14 24:14 25:14 26:15 27:13 28:10 29:16 30:12 31:21 32:18 33:16 34:18 35:16 36:10 37:12 38:12 39:21 40:20 41:18 42:11 43:19 44:13 45:21 46:16 47:19 48:15 49:12 50:19 51:16 52:15 53:19 54:16 55:13 56:12 57:17 58:21 59:24 60:16 61:11 62:22 63:18] 2023-10-21T04:34:06.007+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:27 1:22 2:24 3:23 4:23 5:28 6:22 7:23 8:18 9:23 10:35 11:19 12:23 13:22 14:12 15:21 16:15 17:13 18:14 19:28 20:20 21:27 22:24 23:21 24:21 25:21 26:22 27:20 28:17 29:23 30:19 31:28 32:25 33:23 34:25 35:23 36:17 37:19 38:19 39:28 40:27 41:25 42:18 43:26 44:20 45:28 46:23 47:26 48:22 49:19 50:26 51:23 52:22 53:26 54:23 55:20 56:19 57:24 58:28 59:31 60:23 61:18 62:29 63:25] 2023-10-21T04:34:06.007+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:34:06.007+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:33864: use of closed network connection 2023-10-21T04:34:06.007+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:33864" exit: read tcp4 127.0.0.1:9121->127.0.0.1:33864: use of closed network connection 2023-10-21T04:34:06.007+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:33864" closed ! 2023-10-21T04:34:06.007+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:34:06.007+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:06.007+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:06.007+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:06.008+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:06.008+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:34:06.008+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:06.008+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 28627 2023-10-21T04:34:06.008+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:34:06.022+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.651461ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:06.037+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:34:06.037+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:34:06.037+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:34:06.037+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28627 2023-10-21T04:34:06.037+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28627 2023-10-21T04:34:06.037+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:34:06.067+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:06.067+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:06.081+05:30 [Info] switched currmeta from 73 -> 74 force true 2023-10-21T04:34:06.118+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:06.118+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:06.135+05:30 [Info] switched currmeta from 74 -> 75 force true 2023-10-21T04:34:06.487+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6978970835863814, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:34:06.570+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15165282789964834431:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 2023-10-21T04:34:06.874+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3010072456373199569:[0]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:34:06.874+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3010072456373199569:[0]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:34:06.874+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3010072456373199569:[0]]. New rollbacktime: 1697842901234041110, Old rollbacktime: 0 2023-10-21T04:34:09.314+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:09.319+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:09.319+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:09.321+05:30 [Info] switched currmeta from 75 -> 75 force true 2023-10-21T04:34:09.567+05:30 [Info] janitor: running cleanup. 2023-10-21T04:34:09.567+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[15945117906390072636 18046371353728341033 8504686614261377811 4860549750398470299 7525085113233084253 2048006213723315482 8378417662949994532 8857269605176042330 11365633301282701856 12338036531800130039 1252359852432998131 17656144409930763141 573521267164877540 14292583894719865678 17797267771846799851 18323460706027642575 4535197604682045916 14741949230702299196 16162012140118277065 249887442666643345 13268712595133993155 15431181238515168303 4138047918192025493 4532813151040661705 6985049432225870291 12027054384338714804 12833670934079048728 13856704970496601958 14889401051926000176 7392796230534034390 11619165464560280276 12824340686309573428 18026660730857541767 3674078063353481907 8669649359784409226 4790214984019693894 12140458060295645238 12195715103909237684 15968368593314043432 5465835793930010953]} 2023-10-21T04:34:09.567+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:34:09.604+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:34:09.605+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17434304260397056806 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:6 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.029µs SnapOpenDur 134.169µs 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:34:09.605+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 7 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:131 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.949µs SnapOpenDur 72.795µs 2023-10-21T04:34:09.605+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:34:09.605+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 8 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.637µs SnapOpenDur 55.727µs 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:34:09.605+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 1 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.573µs SnapOpenDur 40.577µs 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:34:09.605+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 2 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.013µs SnapOpenDur 46.367µs 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:34:09.605+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 3 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.387µs SnapOpenDur 45.081µs 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Creating New Snapshot SnapshotInfo: count:118 committed:true 2023-10-21T04:34:09.605+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 4 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:118 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.907µs SnapOpenDur 41.556µs 2023-10-21T04:34:09.605+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:34:09.606+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 5 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.179µs SnapOpenDur 43.733µs 2023-10-21T04:34:09.606+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:34:09.606+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 6 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.085µs SnapOpenDur 58.436µs 2023-10-21T04:34:09.606+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15165282789964834431 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:34:09.606+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15165282789964834431 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.356µs SnapOpenDur 61.584µs 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15165282789964834431, PartitionId 0 SnapshotId 17 Creating recovery point ... 2023-10-21T04:34:09.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:09.624+05:30 [Info] memstats {"Alloc":133076664, "TotalAlloc":167514720, "Sys":169774376, "Lookups":0, "Mallocs":544746,"Frees":380671, "HeapAlloc":133076664, "HeapSys":150831104, "HeapIdle":9347072, "HeapInuse":141484032,"HeapReleased":884736, "HeapObjects":164075,"MSpanInuse":609280, "MSpanSys": 750720, "StackInuse": 4358144,"GCSys":11528496, "LastGC":1697842990337591468,"PauseTotalNs":13277827, "PauseNs":[12228962, 425364, 46114, 170650, 187858, 218879], "NumGC":6} 2023-10-21T04:34:09.629+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 4 Created recovery point (took 23.821217ms) 2023-10-21T04:34:09.639+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 4. 2023-10-21T04:34:09.640+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:34:09.751+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15165282789964834431, PartitionId 0 SnapshotId 17 Created recovery point (took 144.622177ms) 2023-10-21T04:34:09.777+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 2 Created recovery point (took 171.050151ms) 2023-10-21T04:34:09.778+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:34:09.779+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:34:09.786+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 2 Created recovery point (took 180.273064ms) 2023-10-21T04:34:09.788+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:34:09.789+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:34:09.799+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 2 Created recovery point (took 192.998503ms) 2023-10-21T04:34:09.800+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:34:09.801+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:34:09.810+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 2 Created recovery point (took 204.062825ms) 2023-10-21T04:34:09.811+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:34:09.812+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:34:09.828+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 2 Created recovery point (took 222.148758ms) 2023-10-21T04:34:09.832+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:34:09.832+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:34:09.842+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 2 Created recovery point (took 235.860831ms) 2023-10-21T04:34:09.843+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:34:09.844+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:34:09.863+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 2 Created recovery point (took 256.899075ms) 2023-10-21T04:34:09.865+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:34:09.865+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:34:09.876+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 2 Created recovery point (took 269.71448ms) 2023-10-21T04:34:09.877+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:34:09.878+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:34:10.216+05:30 [Info] connected with 4 indexers 2023-10-21T04:34:10.216+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:34:10.286+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:34:10.306+05:30 [Info] num concurrent scans {0} 2023-10-21T04:34:10.306+05:30 [Info] average scan response {0 ms} 2023-10-21T04:34:10.324+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:34:10.324+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:34:10.329+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:34:12.084+05:30 [Info] Indexer::ReadMemstats Time Taken 191.332µs 2023-10-21T04:34:12.668+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. err: operation_canceled 2023-10-21T04:34:12.668+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:34:12.668+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:34:12.670+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:34:12.680+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:34:12.680+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[7f731cd927c72e28ff24e69497b93571 964f96068adef144590fc7ab85b2311c 18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:34:12.681+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:34:16.779+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:34:18.585+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:34:18.764+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:18.765+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:18.767+05:30 [Info] switched currmeta from 75 -> 75 force true 2023-10-21T04:34:18.811+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:18.823+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:18.823+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:18.841+05:30 [Info] switched currmeta from 75 -> 75 force true 2023-10-21T04:34:18.885+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to true 2023-10-21T04:34:18.885+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 216] updating rebalancing status to true 2023-10-21T04:34:18.909+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:18.941+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:18.941+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:18.948+05:30 [Info] switched currmeta from 75 -> 75 force true 2023-10-21T04:34:19.083+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:34:19.130+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:19.130+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:19.133+05:30 [Info] switched currmeta from 75 -> 75 force true 2023-10-21T04:34:20.428+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {a4a273dac27463ee6d67902ad7340fd2 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:34:20.435+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 8.121µs 2023-10-21T04:34:20.435+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:34:20.435+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:34:20.435+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:34:20.439+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:34:20.439+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:20.439+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 4.206084ms 2023-10-21T04:34:20.439+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:34:20.439+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:34:20.439+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:34:20.439+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.978µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:20.439+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a4a273dac27463ee6d67902ad7340fd2 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a4a273dac27463ee6d67902ad7340fd2]}]} 2023-10-21T04:34:20.444+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:34:20.444+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:34:21.519+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {8806f5706747af4e635b8da5fcd8aeee a4a273dac27463ee6d67902ad7340fd2 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:34:21.521+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {8806f5706747af4e635b8da5fcd8aeee a4a273dac27463ee6d67902ad7340fd2 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:34:21.521+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"a4a273dac27463ee6d67902ad7340fd2","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:34:21.523+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{8806f5706747af4e635b8da5fcd8aeee a4a273dac27463ee6d67902ad7340fd2 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:34:21.523+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{8806f5706747af4e635b8da5fcd8aeee a4a273dac27463ee6d67902ad7340fd2 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:34:21.523+05:30 [Info] ShardRebalancer::observeRebalance &{8806f5706747af4e635b8da5fcd8aeee a4a273dac27463ee6d67902ad7340fd2 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:34:21.527+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"a4a273dac27463ee6d67902ad7340fd2","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:34:21.527+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:34:21.752+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45056 2023-10-21T04:34:21.759+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45056 2023-10-21T04:34:21.802+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:45056 2023-10-21T04:34:21.919+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:21.958+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:21.958+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:34:21.958+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:34:21.958+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:34:21.959+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.669µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:21.985+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.047+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.096+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.145+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.200+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"a4a273dac27463ee6d67902ad7340fd2","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:34:22.200+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:34:22.200+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:34:22.200+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:34:22.200+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:34:22.200+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:34:22.200+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 14.383µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:22.203+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.273+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.444+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.445+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{8806f5706747af4e635b8da5fcd8aeee a4a273dac27463ee6d67902ad7340fd2 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:34:22.497+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.522+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.522+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:34:22.522+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:34:22.522+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:34:22.522+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.163µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:22.522+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[2925063509317026816:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/2925063509317026816/shards/shard2925063509317026816 16787445029868804812:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/16787445029868804812/shards/shard16787445029868804812] a4a273dac27463ee6d67902ad7340fd2 ShardToken75:68:3d:12:64:22:c1:3 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc005a519e0 0xc005a51a40 map[] 0xc0067c1bc0 0xc000129b60 false} 2023-10-21T04:34:22.523+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/2925063509317026816/shards/shard2925063509317026816 2023-10-21T04:34:22.523+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/16787445029868804812/shards/shard16787445029868804812 2023-10-21T04:34:22.528+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/16787445029868804812/shards/shard16787445029868804812/data: super block snapshot head offset :0 tail offset:1302528 sbgen :204 2023-10-21T04:34:22.528+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/16787445029868804812/shards/shard16787445029868804812/data/recovery: super block snapshot head offset :0 tail offset:1224704 sbgen :204 2023-10-21T04:34:22.535+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/2925063509317026816/shards/shard2925063509317026816/data: super block snapshot head offset :0 tail offset:1200128 sbgen :188 2023-10-21T04:34:22.536+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/2925063509317026816/shards/shard2925063509317026816/data/recovery: super block snapshot head offset :0 tail offset:1089536 sbgen :188 2023-10-21T04:34:22.539+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/16787445029868804812/shards/shard16787445029868804812: metadata saved successfully 2023-10-21T04:34:22.540+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) added 2023-10-21T04:34:22.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:34:22.540+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 149793588 totalBytes: 2546491 numShards: 2 duration: 17.353464ms 2023-10-21T04:34:22.540+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/16787445029868804812/shards/shard16787445029868804812, newshardId 16787445029868804812, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, shardsize 2546491, duration 16.62459ms 2023-10-21T04:34:22.540+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 16787445029868804812, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812, err: , elapsed(sec): 0.01758035 2023-10-21T04:34:22.540+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 16787445029868804812 bytesWritten: 2546491, totalBytes: 2546491, transferRate: 1.591556875e+08 2023-10-21T04:34:22.546+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/2925063509317026816/shards/shard2925063509317026816: metadata saved successfully 2023-10-21T04:34:22.546+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) added 2023-10-21T04:34:22.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:34:22.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a4a273dac27463ee6d67902ad7340fd2_ShardToken75_68_3d_12_64_22_c1_3/2925063509317026816/shards/shard2925063509317026816, newshardId 2925063509317026816, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, shardsize 2308764, duration 25.234509ms 2023-10-21T04:34:22.548+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 2925063509317026816, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816, err: , elapsed(sec): 0.025556136 2023-10-21T04:34:22.549+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: a4a273dac27463ee6d67902ad7340fd2, transferId: ShardToken75:68:3d:12:64:22:c1:3 2023-10-21T04:34:22.549+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: a4a273dac27463ee6d67902ad7340fd2, transferId: ShardToken75:68:3d:12:64:22:c1:3, elapsed(sec): 0.000556475 2023-10-21T04:34:22.549+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[2925063509317026816: 16787445029868804812:], shardPaths: map[2925063509317026816:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 16787445029868804812:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.026728847 2023-10-21T04:34:22.550+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 2925063509317026816 bytesWritten: 2308764, totalBytes: 2308764, transferRate: 9.235056e+07 2023-10-21T04:34:22.550+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [16787445029868804812 2925063509317026816], ttid: ShardToken75:68:3d:12:64:22:c1:3, elapsed(sec): 0.027536769 2023-10-21T04:34:22.593+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:22.628+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:22.628+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken75:68:3d:12:64:22:c1:3 2023-10-21T04:34:22.628+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2925063509317026816:true 16787445029868804812:true] 2023-10-21T04:34:22.628+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.547µs 2023-10-21T04:34:22.628+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [16787445029868804812 2925063509317026816], isLockedForRecovery: true 2023-10-21T04:34:22.628+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [16787445029868804812 2925063509317026816], errMap: map[2925063509317026816: 16787445029868804812:], elapsed: 10.03µs 2023-10-21T04:34:22.651+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:34:22.651+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:34:22.688+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:22.688+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:22.700+05:30 [Info] switched currmeta from 75 -> 76 force true 2023-10-21T04:34:22.739+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15864358084358521096,realInstId: 0, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:34:22.740+05:30 [Info] Indexer::handleRecoverIndex InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:34:22.740+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 448.709µs 2023-10-21T04:34:22.740+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 499.66µs 2023-10-21T04:34:22.740+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:22.741+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 8 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:22.741+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:34:22.743+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15864358084358521096, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:34:22.743+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 92.045771ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:22.745+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner initialized 2023-10-21T04:34:22.745+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:34:22.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:22.745+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCleaner initialized for recovery 2023-10-21T04:34:22.745+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:34:22.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery 2023-10-21T04:34:22.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:22.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true] 2023-10-21T04:34:22.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1224704] Data log Head [0] Tail [1302528] 2023-10-21T04:34:22.746+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1224704] 2023-10-21T04:34:22.748+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.748+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:34:22.748+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:34:22.748+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.748+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:34:22.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:34:22.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:34:22.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:34:22.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.749+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:34:22.749+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.750+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:34:22.750+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.750+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:34:22.751+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.751+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner initialized 2023-10-21T04:34:22.753+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:34:22.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:22.752+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:34:22.755+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCleaner initialized for recovery 2023-10-21T04:34:22.756+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:34:22.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery 2023-10-21T04:34:22.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:22.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true] 2023-10-21T04:34:22.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1089536] Data log Head [0] Tail [1200128] 2023-10-21T04:34:22.757+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1089536] 2023-10-21T04:34:22.760+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.762+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:22.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.762+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:22.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.763+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:22.763+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.763+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:22.763+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.763+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:22.763+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.763+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:22.763+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.763+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:22.764+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.764+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:22.765+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.765+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:34:22.765+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.765+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:34:22.765+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.765+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:34:22.766+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.766+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:34:22.766+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.766+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:34:22.767+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.767+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:34:22.768+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.768+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:34:22.768+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.768+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:34:22.770+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.770+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:22.770+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.771+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:22.771+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.771+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:22.771+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.771+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:22.771+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.771+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:22.771+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.771+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:22.772+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.772+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:22.772+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.772+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:22.773+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:22.775+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:22.782+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex 2023-10-21T04:34:22.793+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : replayOffset: 1298432 (RP:1282048) (CP:1298432) 2023-10-21T04:34:22.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1298432] took [47.498066ms] 2023-10-21T04:34:22.793+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057426, resident 1, memSz 10752, memSzIndex 0, numRecords 106 2023-10-21T04:34:22.794+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.794+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057276, resident 1, memSz 12809, memSzIndex 0, numRecords 127 2023-10-21T04:34:22.794+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.794+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.794+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.794+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.794+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1055176, resident 1, memSz 562, memSzIndex 0, numRecords 8 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057276, resident 1, memSz 12321, memSzIndex 0, numRecords 122 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.795+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1059676, resident 1, memSz 74966, memSzIndex 264, numRecords 1000 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057276, resident 1, memSz 13698, memSzIndex 0, numRecords 136 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057426, resident 1, memSz 12433, memSzIndex 0, numRecords 123 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057276, resident 1, memSz 13717, memSzIndex 0, numRecords 136 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057426, resident 1, memSz 12643, memSzIndex 0, numRecords 125 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: request quota after header replay 1057426, resident 1, memSz 12641, memSzIndex 0, numRecords 125 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44752081 controller mem 2700 2023-10-21T04:34:22.796+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1302528] replayOffset [1298432] 2023-10-21T04:34:22.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [50.495889ms] 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.796+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data], Shared [true]. Built [18] plasmas, took [51.201398ms] 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 106 metaSz: 2512 rp [1] sn: 360007 count: 106 metaSz: 2512 2023-10-21T04:34:22.797+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 84970 recovery UsedSpace 107794 2023-10-21T04:34:22.798+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex 2023-10-21T04:34:22.798+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : replayOffset: 1196032 (RP:1163264) (CP:1196032) 2023-10-21T04:34:22.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1196032] took [41.101577ms] 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053288, resident 1, memSz 10500, memSzIndex 77, numRecords 136 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.798+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9501, memSzIndex 76, numRecords 127 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053364, resident 1, memSz 9269, memSzIndex 92, numRecords 125 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1053288, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: request quota after header replay 1056118, resident 1, memSz 66990, memSzIndex 936, numRecords 1000 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 44757827 controller mem 4820 2023-10-21T04:34:22.799+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1200128] replayOffset [1196032] 2023-10-21T04:34:22.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [42.131363ms] 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.799+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:34:22.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data], Shared [true]. Built [17] plasmas, took [42.969356ms] 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 106 metaSz: 2512 rp [1] sn: 360007 count: 106 metaSz: 2512 2023-10-21T04:34:22.800+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 80316 recovery UsedSpace 111890 2023-10-21T04:34:22.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:34:22.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:22.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:22.821+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:34:22.821+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:34:22.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex started 2023-10-21T04:34:22.821+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:22.821+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:34:22.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:22.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:22.826+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:34:22.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex started 2023-10-21T04:34:22.826+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:34:22.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:22.842+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:22.842+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:22.848+05:30 [Info] switched currmeta from 76 -> 77 force true 2023-10-21T04:34:22.856+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Warmup took 115.070491ms 2023-10-21T04:34:22.856+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 8 WriterThreads 1 2023-10-21T04:34:22.857+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Stopped Billing WUs 2023-10-21T04:34:22.857+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index Index: 15864358084358521096 Partition: 8 2023-10-21T04:34:22.857+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:22.858+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 1 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:22.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:22.859+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:34:22.859+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 127 metaSz: 2513 rp [1] sn: 360007 count: 127 metaSz: 2513 2023-10-21T04:34:22.859+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 87258 recovery UsedSpace 107794 2023-10-21T04:34:22.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:22.859+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:34:22.859+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 127 metaSz: 2513 rp [1] sn: 360007 count: 127 metaSz: 2513 2023-10-21T04:34:22.859+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 84277 recovery UsedSpace 112244 2023-10-21T04:34:22.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:34:22.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:22.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:22.867+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:34:22.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex started 2023-10-21T04:34:22.867+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:22.868+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:34:22.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:22.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:22.872+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:34:22.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex started 2023-10-21T04:34:22.872+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:34:22.872+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:22.872+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.906+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Warmup took 47.738805ms 2023-10-21T04:34:22.906+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 1 WriterThreads 1 2023-10-21T04:34:22.906+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Stopped Billing WUs 2023-10-21T04:34:22.906+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index Index: 15864358084358521096 Partition: 1 2023-10-21T04:34:22.906+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:22.906+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 2 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:22.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:22.906+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 258 2023-10-21T04:34:22.906+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 136 metaSz: 2513 rp [1] sn: 360007 count: 136 metaSz: 2513 2023-10-21T04:34:22.907+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 91076 recovery UsedSpace 112344 2023-10-21T04:34:22.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:22.907+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:34:22.907+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 136 metaSz: 2513 rp [1] sn: 360007 count: 136 metaSz: 2513 2023-10-21T04:34:22.907+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 88205 recovery UsedSpace 107794 2023-10-21T04:34:22.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:34:22.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:22.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:22.919+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:34:22.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex started 2023-10-21T04:34:22.919+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:22.919+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:34:22.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:22.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:22.920+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:34:22.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex started 2023-10-21T04:34:22.920+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:34:22.921+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.921+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:22.941+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Warmup took 35.207311ms 2023-10-21T04:34:22.942+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 2 WriterThreads 1 2023-10-21T04:34:22.942+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Stopped Billing WUs 2023-10-21T04:34:22.942+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index Index: 15864358084358521096 Partition: 2 2023-10-21T04:34:22.942+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:22.942+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 3 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:22.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:22.943+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:34:22.943+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 125 metaSz: 2513 rp [1] sn: 360007 count: 125 metaSz: 2513 2023-10-21T04:34:22.943+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 87264 recovery UsedSpace 107794 2023-10-21T04:34:22.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:22.944+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:34:22.944+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 125 metaSz: 2513 rp [1] sn: 360007 count: 125 metaSz: 2513 2023-10-21T04:34:22.945+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 88841 recovery UsedSpace 112154 2023-10-21T04:34:22.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:34:22.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:22.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:22.955+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:34:22.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex started 2023-10-21T04:34:22.955+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:22.955+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:34:22.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:22.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:22.957+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:34:22.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex started 2023-10-21T04:34:22.957+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:34:22.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:22.957+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.002+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Warmup took 58.830325ms 2023-10-21T04:34:23.002+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 3 WriterThreads 1 2023-10-21T04:34:23.002+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Stopped Billing WUs 2023-10-21T04:34:23.002+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index Index: 15864358084358521096 Partition: 3 2023-10-21T04:34:23.002+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:23.002+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 4 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:23.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:23.002+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 240 2023-10-21T04:34:23.002+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 125 metaSz: 2513 rp [1] sn: 360007 count: 125 metaSz: 2513 2023-10-21T04:34:23.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:23.003+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:34:23.003+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 125 metaSz: 2513 rp [1] sn: 360007 count: 125 metaSz: 2513 2023-10-21T04:34:23.003+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 88376 recovery UsedSpace 112150 2023-10-21T04:34:23.003+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 92618 recovery UsedSpace 107794 2023-10-21T04:34:23.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:34:23.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:23.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:23.026+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:34:23.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex started 2023-10-21T04:34:23.026+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.026+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:34:23.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:23.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:23.027+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:34:23.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex started 2023-10-21T04:34:23.027+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:34:23.027+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.027+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.051+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Warmup took 48.827928ms 2023-10-21T04:34:23.051+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 4 WriterThreads 1 2023-10-21T04:34:23.051+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Stopped Billing WUs 2023-10-21T04:34:23.051+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index Index: 15864358084358521096 Partition: 4 2023-10-21T04:34:23.051+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:23.052+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 5 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:23.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:23.052+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:34:23.052+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 123 metaSz: 2513 rp [1] sn: 360007 count: 123 metaSz: 2513 2023-10-21T04:34:23.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:23.052+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:34:23.052+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 123 metaSz: 2513 rp [1] sn: 360007 count: 123 metaSz: 2513 2023-10-21T04:34:23.052+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 87205 recovery UsedSpace 111890 2023-10-21T04:34:23.052+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 92418 recovery UsedSpace 107794 2023-10-21T04:34:23.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:34:23.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:23.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:23.058+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:34:23.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex started 2023-10-21T04:34:23.058+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.058+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:34:23.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:23.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:23.060+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:34:23.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex started 2023-10-21T04:34:23.060+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:34:23.060+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.060+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.098+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Warmup took 45.922674ms 2023-10-21T04:34:23.098+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 5 WriterThreads 1 2023-10-21T04:34:23.098+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Stopped Billing WUs 2023-10-21T04:34:23.098+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index Index: 15864358084358521096 Partition: 5 2023-10-21T04:34:23.098+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:23.098+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 6 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:23.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:23.098+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:34:23.098+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 122 metaSz: 2513 rp [1] sn: 360007 count: 122 metaSz: 2513 2023-10-21T04:34:23.099+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 92212 recovery UsedSpace 111890 2023-10-21T04:34:23.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:23.100+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:34:23.100+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 122 metaSz: 2513 rp [1] sn: 360007 count: 122 metaSz: 2513 2023-10-21T04:34:23.100+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 92225 recovery UsedSpace 107794 2023-10-21T04:34:23.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:34:23.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:23.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:23.118+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:34:23.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex started 2023-10-21T04:34:23.118+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.118+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:34:23.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:23.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:23.119+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:34:23.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex started 2023-10-21T04:34:23.119+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:34:23.120+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.120+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.146+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Warmup took 47.766832ms 2023-10-21T04:34:23.146+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 6 WriterThreads 1 2023-10-21T04:34:23.146+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Stopped Billing WUs 2023-10-21T04:34:23.146+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index Index: 15864358084358521096 Partition: 6 2023-10-21T04:34:23.146+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15864358084358521096 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:23.146+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15864358084358521096 PartitionId: 7 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:23.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:23.147+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:34:23.147+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 136 metaSz: 2513 rp [1] sn: 360007 count: 136 metaSz: 2513 2023-10-21T04:34:23.147+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 96518 recovery UsedSpace 112338 2023-10-21T04:34:23.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:23.147+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:34:23.147+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360006 count: 136 metaSz: 2513 rp [1] sn: 360007 count: 136 metaSz: 2513 2023-10-21T04:34:23.148+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 93787 recovery UsedSpace 107794 2023-10-21T04:34:23.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:34:23.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:23.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:23.157+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:34:23.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex started 2023-10-21T04:34:23.157+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.157+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:34:23.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:23.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:23.158+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:34:23.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex started 2023-10-21T04:34:23.158+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.158+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.158+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:34:23.177+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Warmup took 30.959414ms 2023-10-21T04:34:23.178+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15864358084358521096 partitionId 7 WriterThreads 1 2023-10-21T04:34:23.178+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Stopped Billing WUs 2023-10-21T04:34:23.178+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15864358084358521096 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index Index: 15864358084358521096 Partition: 7 2023-10-21T04:34:23.178+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15864358084358521096, shardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:34:23.178+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15864358084358521096, partnShardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:34:23.178+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:23.178+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15864358084358521096 2023-10-21T04:34:23.178+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:23.178+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:34:23.179+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:23.179+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:23.179+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:23.179+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15864358084358521096 Partitions [5 6 7 8 1 2 3 4] 2023-10-21T04:34:23.180+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:1 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:34:23.181+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:2 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:34:23.182+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:3 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:34:23.183+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:4 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:34:23.184+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:5 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:34:23.185+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:6 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:34:23.186+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:7 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:34:23.186+05:30 [Info] StorageMgr::openSnapshot IndexInst:15864358084358521096 Partition:8 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:34:23.197+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.484997ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:23.197+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 122.614µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:34:23.208+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:23.208+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:23.211+05:30 [Info] switched currmeta from 77 -> 78 force true 2023-10-21T04:34:23.260+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:23.260+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:23.263+05:30 [Info] switched currmeta from 78 -> 79 force true 2023-10-21T04:34:23.553+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:23.553+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:23.565+05:30 [Info] switched currmeta from 79 -> 80 force true 2023-10-21T04:34:23.817+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[15864358084358521096:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:34:23.818+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 7147407900589480727, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:34:23.818+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 7147407900589480727, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:34:23.840+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 9570541924043341912,realInstId: 0, indexDefn DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:34:23.840+05:30 [Info] Indexer::handleRecoverIndex InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:34:23.840+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 266.652µs 2023-10-21T04:34:23.840+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 311.981µs 2023-10-21T04:34:23.840+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 9570541924043341912 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:23.841+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 9570541924043341912 PartitionId: 0 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:23.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:23.841+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1056118 memSz 2120 2023-10-21T04:34:23.841+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 14 count: 1000 metaSz: 2508 2023-10-21T04:34:23.841+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 142625 recovery UsedSpace 16481 2023-10-21T04:34:23.841+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 9570541924043341912, indexDefn DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:34:23.841+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 23.607445ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:23.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:23.842+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1059676 memSz 696 2023-10-21T04:34:23.842+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 14 count: 1000 metaSz: 2508 2023-10-21T04:34:23.842+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 147120 recovery UsedSpace 13748 2023-10-21T04:34:23.848+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:23.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex 2023-10-21T04:34:23.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:23.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:23.867+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex 2023-10-21T04:34:23.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex started 2023-10-21T04:34:23.867+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.867+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex 2023-10-21T04:34:23.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:23.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:23.868+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex 2023-10-21T04:34:23.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex started 2023-10-21T04:34:23.868+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9570541924043341912 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:34:23.869+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:23.869+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:23.896+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:23.898+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:23.904+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 9570541924043341912 PartitionId 0 Warmup took 63.589714ms 2023-10-21T04:34:23.905+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 9570541924043341912 partitionId 0 WriterThreads 4 2023-10-21T04:34:23.905+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 9570541924043341912 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index Index: 9570541924043341912 Partition: 0 2023-10-21T04:34:23.905+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 9570541924043341912, shardIdMap: map[0:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:34:23.905+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 9570541924043341912, partnShardIdMap: map[0:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:34:23.905+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:23.905+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 9570541924043341912 2023-10-21T04:34:23.905+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:34:23.906+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:34:23.906+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:23.906+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:34:23.906+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:23.906+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 9570541924043341912 Partitions [0] 2023-10-21T04:34:23.907+05:30 [Info] StorageMgr::openSnapshot IndexInst:9570541924043341912 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:34:23.907+05:30 [Info] switched currmeta from 80 -> 81 force true 2023-10-21T04:34:23.917+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.443789ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:23.917+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 86.858µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:34:23.936+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:23.936+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:23.942+05:30 [Info] switched currmeta from 81 -> 82 force true 2023-10-21T04:34:24.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:24.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:24.077+05:30 [Info] switched currmeta from 82 -> 83 force true 2023-10-21T04:34:24.372+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:24.383+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:24.383+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:24.385+05:30 [Info] switched currmeta from 83 -> 83 force true 2023-10-21T04:34:24.587+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:24.587+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:24.591+05:30 [Info] switched currmeta from 83 -> 84 force true 2023-10-21T04:34:24.616+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:24.618+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:24.626+05:30 [Info] switched currmeta from 84 -> 85 force true 2023-10-21T04:34:24.677+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:24.678+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:24.681+05:30 [Info] switched currmeta from 85 -> 86 force true 2023-10-21T04:34:24.852+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[9570541924043341912:INDEX_STATE_RECOVERED 15864358084358521096:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:34:24.853+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:34:24.853+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:34:24.869+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10032435514095102923,realInstId: 0, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:34:24.869+05:30 [Info] Indexer::handleRecoverIndex InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:34:24.869+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 398.778µs 2023-10-21T04:34:24.869+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 445.543µs 2023-10-21T04:34:24.869+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:24.870+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 1 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:24.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:24.870+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:24.870+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:34:24.876+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10032435514095102923, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:34:24.880+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 27.733694ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:24.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:24.881+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:24.881+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:24.890+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:24.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:24.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:24.893+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:24.894+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:24.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:34:24.896+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:24.896+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:24.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:24.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:24.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:24.898+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:24.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:34:24.899+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:34:24.899+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:24.899+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:24.899+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:24.899+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:24.906+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:24.902+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:24.907+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:24.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:34:24.907+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:24.907+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:34:24.907+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:34:24.907+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:24.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:34:24.907+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:24.907+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:24.907+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:24.907+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:24.908+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:24.909+05:30 [Info] switched currmeta from 86 -> 87 force true 2023-10-21T04:34:24.917+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:24.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:34:24.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:24.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:34:24.918+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:24.918+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:34:24.918+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:34:24.918+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:24.918+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:24.918+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:24.923+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:24.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:34:24.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:34:24.923+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:24.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:34:24.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:34:24.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:24.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:34:24.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:34:24.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:34:24.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:24.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed 2023-10-21T04:34:24.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:24.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:24.937+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:24.937+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:24.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:24.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:24.940+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:24.940+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:24.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:24.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:24.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:24.945+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:24.945+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:34:24.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex started 2023-10-21T04:34:24.945+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:24.945+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:24.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:24.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:24.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:24.949+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:24.949+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:34:24.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex started 2023-10-21T04:34:24.950+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Warmup took 79.831583ms 2023-10-21T04:34:24.950+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:24.950+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:24.950+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 1 WriterThreads 1 2023-10-21T04:34:24.950+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 1 Stopped Billing WUs 2023-10-21T04:34:24.950+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index Index: 10032435514095102923 Partition: 1 2023-10-21T04:34:24.950+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:24.950+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 2 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:24.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:24.950+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:24.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:24.951+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:24.951+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:34:24.951+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:24.964+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:34:24.964+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:24.964+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:24.964+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:24.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:34:24.964+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:34:24.964+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:24.964+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:24.964+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:24.965+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:24.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:34:24.965+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:24.965+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:34:24.965+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:34:24.965+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:24.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:34:24.965+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:24.965+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:24.965+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:24.965+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:24.965+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:24.965+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:24.973+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:24.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:34:24.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:34:24.973+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:24.973+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:24.973+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:24.973+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:24.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:34:24.973+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:24.973+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:34:24.973+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:34:24.974+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:24.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:34:24.974+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:24.974+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:24.974+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:24.974+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:24.984+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:24.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:34:24.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:34:24.984+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:24.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:34:24.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:34:24.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:24.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:34:24.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:34:24.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:34:24.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:24.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:34:25.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.000+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.000+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.004+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.004+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:25.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.007+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:25.007+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:34:25.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex started 2023-10-21T04:34:25.007+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.007+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:25.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.011+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:25.011+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:34:25.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex started 2023-10-21T04:34:25.011+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.011+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Warmup took 60.889244ms 2023-10-21T04:34:25.012+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 2 WriterThreads 1 2023-10-21T04:34:25.012+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 2 Stopped Billing WUs 2023-10-21T04:34:25.011+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.012+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index Index: 10032435514095102923 Partition: 2 2023-10-21T04:34:25.012+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:25.012+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 3 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:25.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.013+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.013+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.014+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.014+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:34:25.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:25.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.022+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:25.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:34:25.022+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.022+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:25.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.023+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:25.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.024+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.024+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:34:25.024+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:25.024+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:34:25.024+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:34:25.024+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:25.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.024+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.031+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:34:25.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:34:25.031+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:25.031+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.031+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.031+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:34:25.031+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:25.031+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:34:25.031+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:34:25.031+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:25.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:34:25.031+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.032+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.032+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.032+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.036+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:34:25.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:34:25.037+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:25.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:34:25.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:34:25.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:34:25.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:34:25.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:34:25.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:34:25.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.046+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.046+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.049+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.049+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.049+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:25.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.052+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:25.052+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:34:25.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex started 2023-10-21T04:34:25.052+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.052+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:25.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.055+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:25.055+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:34:25.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex started 2023-10-21T04:34:25.055+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Warmup took 42.141188ms 2023-10-21T04:34:25.055+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.055+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 3 WriterThreads 1 2023-10-21T04:34:25.055+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.055+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 3 Stopped Billing WUs 2023-10-21T04:34:25.055+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index Index: 10032435514095102923 Partition: 3 2023-10-21T04:34:25.055+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:25.055+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 4 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:25.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.055+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.056+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.056+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.056+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.069+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:34:25.069+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.069+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:25.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.074+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:25.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:34:25.074+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:34:25.074+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:34:25.074+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:25.074+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:34:25.074+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:34:25.074+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:25.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.074+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.075+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.087+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:34:25.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:34:25.087+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:25.087+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.088+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.088+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:34:25.088+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:25.088+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:34:25.088+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:34:25.088+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:25.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:34:25.088+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.088+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.088+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.088+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.103+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:34:25.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:34:25.104+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:25.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:34:25.104+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:34:25.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:34:25.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:34:25.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:34:25.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:34:25.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.122+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.122+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.126+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:25.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.130+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:25.130+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:34:25.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex started 2023-10-21T04:34:25.130+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.130+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:25.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.133+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:25.133+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:34:25.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex started 2023-10-21T04:34:25.133+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Warmup took 78.238748ms 2023-10-21T04:34:25.134+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 4 WriterThreads 1 2023-10-21T04:34:25.134+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 4 Stopped Billing WUs 2023-10-21T04:34:25.134+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index Index: 10032435514095102923 Partition: 4 2023-10-21T04:34:25.134+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:25.134+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 5 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:25.134+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.134+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.134+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.135+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.135+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 19 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.143+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:25.143+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:34:25.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:34:25.143+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.143+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.143+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:34:25.143+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.144+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:25.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:34:25.144+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:25.144+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:34:25.144+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:34:25.143+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.144+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:25.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:34:25.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.144+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:34:25.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:34:25.150+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:25.150+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:34:25.150+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:34:25.150+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:25.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.150+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.156+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:34:25.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:34:25.156+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:25.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:34:25.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:34:25.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:34:25.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:34:25.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 19 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:34:25.162+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.162+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:34:25.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.165+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.165+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.168+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.169+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:25.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.172+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:25.172+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:34:25.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex started 2023-10-21T04:34:25.172+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.172+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:25.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.175+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:25.175+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:34:25.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex started 2023-10-21T04:34:25.175+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Warmup took 40.714301ms 2023-10-21T04:34:25.175+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 5 WriterThreads 1 2023-10-21T04:34:25.175+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 5 Stopped Billing WUs 2023-10-21T04:34:25.175+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index Index: 10032435514095102923 Partition: 5 2023-10-21T04:34:25.175+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:25.175+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 6 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:25.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.175+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.176+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.176+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.176+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.176+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.176+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:25.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.188+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:25.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:34:25.188+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.188+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:25.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:25.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:34:25.193+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:34:25.193+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:34:25.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:25.193+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:34:25.193+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:34:25.193+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:25.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.193+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:34:25.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:34:25.202+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:25.202+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:34:25.202+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:34:25.202+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:25.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.202+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.217+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:34:25.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:34:25.218+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:25.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:34:25.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:34:25.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:34:25.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:34:25.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:34:25.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:34:25.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.235+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.235+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.243+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:25.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:25.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:34:25.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex started 2023-10-21T04:34:25.249+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.249+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:25.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.254+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:25.254+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:34:25.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex started 2023-10-21T04:34:25.254+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Warmup took 78.544362ms 2023-10-21T04:34:25.254+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.254+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.254+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 6 WriterThreads 1 2023-10-21T04:34:25.254+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 6 Stopped Billing WUs 2023-10-21T04:34:25.254+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index Index: 10032435514095102923 Partition: 6 2023-10-21T04:34:25.254+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:25.254+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 7 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:25.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.254+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.255+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.255+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.255+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.265+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:34:25.265+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.265+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.265+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:34:25.265+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:34:25.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.265+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:25.265+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.265+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.265+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.265+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:34:25.266+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:25.266+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:34:25.266+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:34:25.266+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:25.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:34:25.266+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.266+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.266+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.266+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:34:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:34:25.272+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:25.272+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:34:25.272+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:34:25.272+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:25.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.272+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.279+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:34:25.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:34:25.279+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:25.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:34:25.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:34:25.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:34:25.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:34:25.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:34:25.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:34:25.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.288+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.288+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.292+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.292+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.292+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:25.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.297+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:25.297+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:34:25.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex started 2023-10-21T04:34:25.297+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.297+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:25.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.300+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:25.300+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:34:25.300+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex started 2023-10-21T04:34:25.300+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Warmup took 45.879119ms 2023-10-21T04:34:25.300+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.300+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.300+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 7 WriterThreads 1 2023-10-21T04:34:25.300+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 7 Stopped Billing WUs 2023-10-21T04:34:25.300+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index Index: 10032435514095102923 Partition: 7 2023-10-21T04:34:25.300+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10032435514095102923 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:25.301+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10032435514095102923 PartitionId: 8 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:25.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.301+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.301+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.301+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:34:25.302+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:34:25.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:25.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.308+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:25.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:34:25.308+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.308+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.309+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:34:25.309+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:34:25.309+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:34:25.309+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.309+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.309+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:34:25.309+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:25.309+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:34:25.309+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:34:25.309+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:34:25.309+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.309+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.310+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.310+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.310+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.310+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:34:25.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:25.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:34:25.315+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:25.315+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:34:25.315+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:34:25.315+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:25.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:25.315+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:25.328+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:25.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:34:25.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:34:25.328+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:25.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:34:25.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:34:25.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:34:25.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:34:25.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:34:25.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:34:25.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:25.346+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.346+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:25.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:34:25.362+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:34:25.362+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:34:25.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:25.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:25.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:25.371+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:25.371+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:34:25.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex started 2023-10-21T04:34:25.371+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.371+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:25.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:34:25.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:34:25.377+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:25.377+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:34:25.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex started 2023-10-21T04:34:25.377+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:25.377+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Warmup took 76.79588ms 2023-10-21T04:34:25.378+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:25.378+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10032435514095102923 partitionId 8 WriterThreads 1 2023-10-21T04:34:25.378+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 10032435514095102923 PartitionId 8 Stopped Billing WUs 2023-10-21T04:34:25.378+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10032435514095102923 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index Index: 10032435514095102923 Partition: 8 2023-10-21T04:34:25.378+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 10032435514095102923, shardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]] err: 2023-10-21T04:34:25.378+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 10032435514095102923, partnShardIdMap: map[1:[16787445029868804812 2925063509317026816] 2:[16787445029868804812 2925063509317026816] 3:[16787445029868804812 2925063509317026816] 4:[16787445029868804812 2925063509317026816] 5:[16787445029868804812 2925063509317026816] 6:[16787445029868804812 2925063509317026816] 7:[16787445029868804812 2925063509317026816] 8:[16787445029868804812 2925063509317026816]], err: 2023-10-21T04:34:25.378+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:25.378+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10032435514095102923 2023-10-21T04:34:25.378+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:25.379+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:34:25.379+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:25.379+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10032435514095102923 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:25.379+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:25.379+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Partitions [5 6 7 8 1 2 3 4] 2023-10-21T04:34:25.379+05:30 [Info] StorageMgr::openSnapshot IndexInst:10032435514095102923 Partition:1 No Snapshot Found. 2023-10-21T04:34:25.379+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10032435514095102923 Adding Nil Snapshot. 2023-10-21T04:34:25.393+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.287668ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:25.394+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 99.784µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:34:25.420+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.420+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.422+05:30 [Info] switched currmeta from 87 -> 88 force true 2023-10-21T04:34:25.650+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.650+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.654+05:30 [Info] switched currmeta from 88 -> 89 force true 2023-10-21T04:34:25.670+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.671+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.680+05:30 [Info] switched currmeta from 89 -> 90 force true 2023-10-21T04:34:25.715+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.715+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.719+05:30 [Info] switched currmeta from 90 -> 91 force true 2023-10-21T04:34:25.735+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.735+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.739+05:30 [Info] switched currmeta from 91 -> 92 force true 2023-10-21T04:34:25.893+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[10032435514095102923:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:34:25.893+05:30 [Info] Indexer::handleMergePartition Source 6853580018063392983 Target 10032435514095102923 2023-10-21T04:34:25.893+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0083e6780 2023-10-21T04:34:25.893+05:30 [Info] MergePartition: sent async request to update index instance 10032435514095102923 rstate moved to ACTIVE 2023-10-21T04:34:25.901+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.367407ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:25.901+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 6853580018063392983, realInst: 10032435514095102923 2023-10-21T04:34:25.901+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [16961575949607441659 7147407900589480727] belonging to collectionId: c, ttid: ShardToken75:68:3d:12:64:22:c1:3. Initiating build 2023-10-21T04:34:25.910+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.910+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.916+05:30 [Info] switched currmeta from 92 -> 93 force true 2023-10-21T04:34:25.939+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.939+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.945+05:30 [Info] switched currmeta from 93 -> 94 force true 2023-10-21T04:34:25.956+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [15864358084358521096 9570541924043341912] &{2} 2023-10-21T04:34:25.956+05:30 [Info] Indexer::handleBuildRecoveredIndexes [15864358084358521096 9570541924043341912] 2023-10-21T04:34:25.956+05:30 [Info] bulkUpdateRState: Index instance 15864358084358521096 rstate moved to PENDING 2023-10-21T04:34:25.956+05:30 [Info] bulkUpdateRState: Index instance 9570541924043341912 rstate moved to PENDING 2023-10-21T04:34:25.956+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [15864358084358521096 9570541924043341912] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:34:25.956+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:34:25.956+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:25.956+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:34:25.974+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:25.974+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:25.976+05:30 [Info] switched currmeta from 94 -> 95 force true 2023-10-21T04:34:26.001+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:34:26.010+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.011+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.011+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:26.011+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:34:26.011+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:34:26.011+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:26.011+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:26.011+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:34:26.012+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.013+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.013+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:34:26.015+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:34:26.016+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId 66597 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:34:26.017+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.018+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.018+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:34:26.018+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:34:26.018+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15864358084358521096 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:34:26.018+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4] INIT_STREAM:map[bucket_1:_default:c1:2]] 2023-10-21T04:34:26.019+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 9570541924043341912 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:34:26.019+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:34:26.019+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=21,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 66597 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:34:26.021+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:34:26.021+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:34:26.025+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:26.030+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 73.955099ms 2023-10-21T04:34:26.029+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:34:26.029+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:34:26.030+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [15864358084358521096 9570541924043341912] 2023-10-21T04:34:26.032+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 130.254971ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:26.030+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 74.182643ms 2023-10-21T04:34:26.034+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[9570541924043341912:true 15864358084358521096:true] 2023-10-21T04:34:26.049+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.545153ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:26.060+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:34476" doAuth() ... 2023-10-21T04:34:26.060+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:34476" auth successful 2023-10-21T04:34:26.064+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.071398ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:26.069+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:34476" +1 2023-10-21T04:34:26.108+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:26.110+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:26.111+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:26.112+05:30 [Info] switched currmeta from 95 -> 97 force true 2023-10-21T04:34:26.134+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:26.134+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:26.139+05:30 [Info] switched currmeta from 97 -> 97 force true 2023-10-21T04:34:26.342+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:26.342+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:26.346+05:30 [Info] switched currmeta from 97 -> 98 force true 2023-10-21T04:34:26.379+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:26.379+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:26.384+05:30 [Info] switched currmeta from 98 -> 99 force true 2023-10-21T04:34:26.597+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 1949031200982 23 66597 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:26.597+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 245782287380001 25 66597 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:34:26.600+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 93079113162334 18 66597 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:26.600+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 52522903947059 28 66597 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:34:26.600+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 205201940883916 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 244791485011286 23 66597 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:26.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 167580527121081 23 66597 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:26.607+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 177815184127237 17 66597 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:26.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 54236768606397 21 66597 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:26.611+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 160381752747972 18 66597 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:26.612+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 134409829206048 26 66597 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:26.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 58522600746999 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.620+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 85302001459467 21 66597 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:26.620+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 142387448675642 30 66597 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:34:26.620+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 90115353725987 21 66597 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:26.620+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 161230548567164 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 123957315902602 28 66597 172.31.5.112:9000. HWT [27-28,28]. 2023-10-21T04:34:26.624+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 213108110408291 23 66597 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:26.626+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 150690956328758 26 66597 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:26.627+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 163460438181870 34 66597 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:34:26.631+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 147946970349859 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.631+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 151273673850546 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.633+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 258671969827141 19 66597 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:26.635+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 53829314402679 32 66597 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:34:26.644+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 114823206239184 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.644+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 19989312661772 15 66597 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:34:26.650+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 68847799765163 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.651+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 232028101514735 19 66597 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:26.652+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 8305763409275 21 66597 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:34:26.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 25510109770064 28 66597 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:34:26.656+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 124811039198764 18 66597 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:26.657+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 33310272161722 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.657+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 207489375637709 12 66597 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:34:26.660+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 107407811917992 20 66597 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:26.662+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 100746255343609 19 66597 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:26.664+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 228857796432725 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.664+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 117298435979365 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.670+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 203503670520010 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.670+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 47156288300713 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.670+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 158357249947806 21 66597 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:26.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 236904976023311 22 66597 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:34:26.675+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 104136033125373 29 66597 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:34:26.677+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 124310544527292 21 66597 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:26.679+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 196269910725961 26 66597 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:26.685+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 276463747117997 23 66597 172.31.5.112:9000. HWT [22-23,23]. 2023-10-21T04:34:26.685+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 246959094981553 34 66597 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:34:26.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 65885584526743 20 66597 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:26.691+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 79077905592013 17 66597 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:26.691+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 172365063201867 21 66597 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:26.693+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 246582249071123 20 66597 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:26.693+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 212884822297038 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 213334533059126 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 254944616021293 24 66597 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:26.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 142324128597367 22 66597 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:34:26.701+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 55145461136105 16 66597 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:34:26.705+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 219528837936411 25 66597 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:26.708+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 80145536651832 17 66597 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:26.708+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 194568213794435 29 66597 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:34:26.713+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 106620692996001 23 66597 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:26.713+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 25511316571199 16 66597 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:34:26.714+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 279733370822238 23 66597 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:26.716+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 223881084081370 17 66597 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:26.718+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 25657368013413 17 66597 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:26.718+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 36300629322478 32 66597 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:34:26.721+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 InstanceIds [15864358084358521096 9570541924043341912] 2023-10-21T04:34:26.722+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 66597 2023-10-21T04:34:26.722+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66597 2023-10-21T04:34:26.722+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:34:26.722+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:34:26.722+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 15864358084358521096 INDEX_STATE_CATCHUP 2023-10-21T04:34:26.722+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 9570541924043341912 INDEX_STATE_CATCHUP 2023-10-21T04:34:26.722+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 15864358084358521096 true 2023-10-21T04:34:26.722+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 9570541924043341912 true 2023-10-21T04:34:26.723+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 15864358084358521096 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 66597 BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=21,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] 2023-10-21T04:34:26.723+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} , fetchKVSeq false 2023-10-21T04:34:26.723+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:34:26.723+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:34:26.723+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66597 STREAM_ACTIVE 2023-10-21T04:34:26.723+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:34:26.723+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:34:26.723+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:26.723+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 66597 2023-10-21T04:34:26.724+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1:_default:c1 66597 2023-10-21T04:34:26.724+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:34:26.724+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:26.724+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:34:26.725+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:26.728+05:30 [Info] send progress stats to clients 2023-10-21T04:34:26.728+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66597 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:26.728+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15864358084358521096 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:34:26.728+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 9570541924043341912 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:34:26.728+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:34:26.728+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:34:26.728+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:26.729+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66597 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:26.729+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:34:26.729+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:34:26.730+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:34:26.730+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:34:26.730+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:34476: use of closed network connection 2023-10-21T04:34:26.730+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:34476" exit: read tcp4 127.0.0.1:9121->127.0.0.1:34476: use of closed network connection 2023-10-21T04:34:26.730+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:34476" closed ! 2023-10-21T04:34:26.730+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:34:26.730+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:34476" already gone 2023-10-21T04:34:26.730+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.730+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.730+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.730+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.731+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:34:26.776+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 48.602748ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:26.819+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 38.911635ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:26.842+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:34:26.845+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:34:26.846+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:34:26.848+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66597 2023-10-21T04:34:26.861+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1:_default:c1 2023-10-21T04:34:26.861+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:34:26.861+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.862+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.862+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:26.863+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:34:26.863+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:34:26.863+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:26.864+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61684 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:26.864+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0046acf00 }] 2023-10-21T04:34:26.876+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.875+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:26.880+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:26.879+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.881+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:34:26.881+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:34:26.881+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 61685 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:34:26.881+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:26.881+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:26.882+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:34:26.882+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:34:26.882+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:34:26.882+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:34:26.882+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 61685 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:34:26.883+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0046acf00 } bucket_1:{0xc00036b740 }] 2023-10-21T04:34:26.886+05:30 [Info] switched currmeta from 99 -> 104 force true 2023-10-21T04:34:26.889+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:34:26.896+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:26.896+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:26.898+05:30 [Info] switched currmeta from 104 -> 104 force true 2023-10-21T04:34:26.925+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:26.926+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:26.933+05:30 [Info] switched currmeta from 104 -> 105 force true 2023-10-21T04:34:27.113+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:27.113+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0074be060 2023-10-21T04:34:27.114+05:30 [Info] handleUpdateIndexRState: Index instance 9570541924043341912 rstate moved to ACTIVE 2023-10-21T04:34:27.114+05:30 [Info] Indexer::handleMergePartition Source 11088380021860709797 Target 15864358084358521096 2023-10-21T04:34:27.114+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00d292480 2023-10-21T04:34:27.114+05:30 [Info] MergePartition: sent async request to update index instance 15864358084358521096 rstate moved to ACTIVE 2023-10-21T04:34:27.121+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.943542ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:27.123+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 3464125101225762400. Retrying... 2023-10-21T04:34:27.123+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[9570541924043341912:true 15864358084358521096:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:34:27.137+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:27.137+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:27.144+05:30 [Info] switched currmeta from 105 -> 106 force true 2023-10-21T04:34:27.146+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 24.883123ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:27.146+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 11088380021860709797, realInst: 15864358084358521096 2023-10-21T04:34:27.155+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:34:27.155+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:34:27.166+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:27.166+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:27.174+05:30 [Info] switched currmeta from 106 -> 107 force true 2023-10-21T04:34:27.207+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:27.207+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:27.211+05:30 [Info] switched currmeta from 107 -> 108 force true 2023-10-21T04:34:27.191+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3464125101225762400,realInstId: 0, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:34:27.215+05:30 [Info] Indexer::handleRecoverIndex InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:34:27.215+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3464125101225762400, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:34:27.215+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 60.553175ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:27.215+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3464125101225762400 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[16787445029868804812 2925063509317026816]] 2023-10-21T04:34:27.216+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3464125101225762400 PartitionId: 0 using shardIds: [16787445029868804812 2925063509317026816], rebalance running: true 2023-10-21T04:34:27.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:34:27.216+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:34:27.216+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 360007 count: 5 metaSz: 2482 rp [1] sn: 360009 count: 6 metaSz: 2482 2023-10-21T04:34:27.216+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: doInit: data UsedSpace 89014 recovery UsedSpace 108026 2023-10-21T04:34:27.218+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 3.247048ms 2023-10-21T04:34:27.218+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 3.290095ms 2023-10-21T04:34:27.221+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:27.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:34:27.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:34:27.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:34:27.223+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:34:27.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex started 2023-10-21T04:34:27.223+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:34:27.225+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:34:27.225+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:34:27.233+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Warmup took 16.939205ms 2023-10-21T04:34:27.233+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3464125101225762400 partitionId 0 WriterThreads 4 2023-10-21T04:34:27.233+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3464125101225762400 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index Index: 3464125101225762400 Partition: 0 2023-10-21T04:34:27.233+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3464125101225762400, shardIdMap: map[0:[16787445029868804812]] err: 2023-10-21T04:34:27.233+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3464125101225762400, partnShardIdMap: map[0:[16787445029868804812]], err: 2023-10-21T04:34:27.233+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:27.233+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3464125101225762400 2023-10-21T04:34:27.233+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:27.233+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:34:27.234+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:27.234+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:27.234+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:27.234+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3464125101225762400 Partitions [0] 2023-10-21T04:34:27.235+05:30 [Info] StorageMgr::openSnapshot IndexInst:3464125101225762400 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:34:27.247+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.459663ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:27.247+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 83.537µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:34:27.265+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:27.265+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:27.277+05:30 [Info] switched currmeta from 108 -> 109 force true 2023-10-21T04:34:27.377+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:27.377+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1] 11645751077768222772:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:27.377+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1] 11645751077768222772:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:27.377+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1] 11645751077768222772:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:27.377+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3345094951175719474:[2 3 4 5 6 7 8 1] 11645751077768222772:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:27.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 245782287380001 25 61685 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:34:27.532+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 1949031200982 23 61685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:27.537+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 205201940883916 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.537+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 93079113162334 18 61685 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:27.537+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 52522903947059 28 61685 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:34:27.539+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 167580527121081 23 61685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:27.539+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 244791485011286 23 61685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:27.544+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 177815184127237 17 61685 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:27.545+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 54236768606397 21 61685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:27.547+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 134409829206048 26 61685 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:27.549+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 160381752747972 18 61685 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:27.551+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 58522600746999 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.553+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 142387448675642 30 61685 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:34:27.557+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 90115353725987 21 61685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:27.559+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 161230548567164 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.574+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 123957315902602 28 61685 172.31.5.112:9000. HWT [27-28,28]. 2023-10-21T04:34:27.574+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 85302001459467 21 61685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:27.574+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 213108110408291 23 61685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:27.578+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 150690956328758 26 61685 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:27.578+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 163460438181870 34 61685 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:34:27.581+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 151273673850546 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.586+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 147946970349859 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.588+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 258671969827141 19 61685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:27.590+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 53829314402679 32 61685 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:34:27.592+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 19989312661772 15 61685 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:34:27.597+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 68847799765163 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.597+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 232028101514735 19 61685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:27.597+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 114823206239184 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.598+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 25510109770064 28 61685 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:34:27.603+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 124811039198764 18 61685 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:27.603+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 8305763409275 21 61685 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:34:27.603+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 33310272161722 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.604+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 207489375637709 12 61685 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:34:27.606+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 107407811917992 20 61685 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:27.608+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 100746255343609 19 61685 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:27.610+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 117298435979365 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.611+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 228857796432725 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.613+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 203503670520010 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.615+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 47156288300713 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.617+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 158357249947806 21 61685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:27.620+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 236904976023311 22 61685 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:34:27.620+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 104136033125373 29 61685 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:34:27.623+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 196269910725961 26 61685 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:27.623+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 124310544527292 21 61685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:27.625+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 246959094981553 34 61685 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:34:27.627+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 65885584526743 20 61685 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:27.627+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 276463747117997 23 61685 172.31.5.112:9000. HWT [22-23,23]. 2023-10-21T04:34:27.631+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 172365063201867 21 61685 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:27.633+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 79077905592013 17 61685 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:27.635+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 212884822297038 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.642+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 246582249071123 20 61685 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:27.643+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 213334533059126 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.645+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 142324128597367 22 61685 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:34:27.646+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 254944616021293 24 61685 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:27.649+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 55145461136105 16 61685 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:34:27.651+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 219528837936411 25 61685 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:27.653+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 194568213794435 29 61685 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:34:27.655+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 80145536651832 17 61685 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:27.658+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 25511316571199 16 61685 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:34:27.663+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 223881084081370 17 61685 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:27.663+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 106620692996001 23 61685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:27.669+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 25657368013413 17 61685 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:27.669+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 279733370822238 23 61685 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:27.669+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 36300629322478 32 61685 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:34:27.669+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 InstanceIds [17434304260397056806 8965923595543532942 15165282789964834431 15864358084358521096 9570541924043341912] 2023-10-21T04:34:27.669+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 61685 2023-10-21T04:34:27.669+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 61685 2023-10-21T04:34:27.669+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:34:27.669+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:34:27.669+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:34:27.669+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:34:27.669+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 61685 2023-10-21T04:34:27.669+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 61685 STREAM_ACTIVE 2023-10-21T04:34:27.669+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:34:27.669+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:34:27.724+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:34:27.925+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:27.927+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:27.943+05:30 [Info] switched currmeta from 109 -> 111 force true 2023-10-21T04:34:27.955+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:27.955+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:27.957+05:30 [Info] switched currmeta from 111 -> 111 force true 2023-10-21T04:34:28.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15864358084358521096:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:34:28.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9570541924043341912:[0] 15864358084358521096:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:34:28.070+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9570541924043341912:[0] 15864358084358521096:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:34:28.225+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3464125101225762400:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:34:28.225+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [11276338709617092948] belonging to collectionId: 9, ttid: ShardToken75:68:3d:12:64:22:c1:3. Initiating build 2023-10-21T04:34:28.232+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [3464125101225762400] &{2} 2023-10-21T04:34:28.233+05:30 [Info] Indexer::handleBuildRecoveredIndexes [3464125101225762400] 2023-10-21T04:34:28.233+05:30 [Info] bulkUpdateRState: Index instance 3464125101225762400 rstate moved to PENDING 2023-10-21T04:34:28.233+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [3464125101225762400] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:34:28.233+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:28.233+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:28.233+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:28.234+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:34:28.234+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:28.234+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:28.234+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:28.234+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:34:28.234+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:34:28.234+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:28.234+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:28.234+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:34:28.234+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:28.234+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:28.234+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:34:28.234+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:34:28.234+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 7300 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:34:28.235+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:28.235+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:28.235+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:34:28.235+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:2] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:34:28.235+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3464125101225762400 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:34:28.235+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:34:28.235+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=21,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 7300 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 25 24 25} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 21 0 21} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 28 27 28} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 23 22 23} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:34:28.235+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:34:28.235+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:34:28.236+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:28.236+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 3.476191ms 2023-10-21T04:34:28.236+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 3.518076ms 2023-10-21T04:34:28.236+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [3464125101225762400] 2023-10-21T04:34:28.238+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 12.294808ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:28.247+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[3464125101225762400:true] 2023-10-21T04:34:28.248+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:34:28.252+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.043302ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:28.258+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:28.273+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:28.273+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:28.279+05:30 [Info] switched currmeta from 111 -> 113 force true 2023-10-21T04:34:28.292+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:28.292+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:28.294+05:30 [Info] switched currmeta from 113 -> 113 force true 2023-10-21T04:34:28.671+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:34:28.671+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:34:29.242+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:34:29.264+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:29.273+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:34654" doAuth() ... 2023-10-21T04:34:29.274+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:34654" auth successful 2023-10-21T04:34:29.274+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:34654" +1 2023-10-21T04:34:29.471+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:29.471+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:29.474+05:30 [Info] switched currmeta from 113 -> 114 force true 2023-10-21T04:34:29.491+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:29.497+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:29.497+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:29.501+05:30 [Info] switched currmeta from 114 -> 114 force true 2023-10-21T04:34:29.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 0 1949031200982 23 7300 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:29.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 2 52522903947059 28 7300 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:34:29.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 1 245782287380001 25 7300 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:34:29.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 3 205201940883916 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.676+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 4 93079113162334 18 7300 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:29.678+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 5 244791485011286 23 7300 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:29.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 7 177815184127237 17 7300 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:29.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 6 167580527121081 23 7300 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:29.684+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 9 134409829206048 26 7300 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:29.684+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 8 54236768606397 21 7300 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:29.685+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_system:_query setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:34:29.686+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Creating New Snapshot SnapshotInfo: count:5 committed:true 2023-10-21T04:34:29.686+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3464125101225762400 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:5 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.389µs SnapOpenDur 130.343µs 2023-10-21T04:34:29.686+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:34:29.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 10 160381752747972 18 7300 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:29.689+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 11 58522600746999 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.692+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 13 90115353725987 21 7300 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:29.694+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 12 142387448675642 30 7300 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:34:29.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 15 85302001459467 21 7300 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:29.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 14 161230548567164 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 16 123957315902602 28 7300 172.31.5.112:9000. HWT [27-28,28]. 2023-10-21T04:34:29.700+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 17 213108110408291 23 7300 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:29.702+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 19 150690956328758 26 7300 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:29.702+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 18 163460438181870 34 7300 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:34:29.705+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 21 147946970349859 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.705+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 20 151273673850546 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.707+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 SnapshotId 2 Created recovery point (took 21.020959ms) 2023-10-21T04:34:29.708+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 22 258671969827141 19 7300 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:29.709+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:5 committed:false. num RPs 3. 2023-10-21T04:34:29.711+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 23 53829314402679 32 7300 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:34:29.715+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 25 114823206239184 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.715+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 24 19989312661772 15 7300 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:34:29.723+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 26 68847799765163 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.724+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 27 232028101514735 19 7300 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:29.728+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 29 8305763409275 21 7300 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:34:29.728+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 28 25510109770064 28 7300 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:34:29.731+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 30 124811039198764 18 7300 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:34:29.733+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 31 207489375637709 12 7300 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:34:29.736+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 33 107407811917992 20 7300 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:29.736+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 32 33310272161722 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.739+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 34 100746255343609 19 7300 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:34:29.742+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 35 117298435979365 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.744+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 37 203503670520010 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.744+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 36 228857796432725 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.748+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 38 47156288300713 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.759+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 40 104136033125373 29 7300 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:34:29.759+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 39 158357249947806 21 7300 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:29.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 46 65885584526743 20 7300 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:29.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 41 236904976023311 22 7300 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:34:29.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 42 124310544527292 21 7300 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:29.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 43 196269910725961 26 7300 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:34:29.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 44 246959094981553 34 7300 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:34:29.772+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 45 276463747117997 23 7300 172.31.5.112:9000. HWT [22-23,23]. 2023-10-21T04:34:29.778+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 47 172365063201867 21 7300 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:34:29.778+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 48 79077905592013 17 7300 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:29.778+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 49 212884822297038 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.778+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 50 246582249071123 20 7300 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:34:29.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 52 142324128597367 22 7300 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:34:29.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 55 219528837936411 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 53 254944616021293 24 7300 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:34:29.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 56 194568213794435 29 7300 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:34:29.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 54 55145461136105 16 7300 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:34:29.790+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 51 213334533059126 25 7300 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:34:29.792+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 58 25511316571199 16 7300 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:34:29.792+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 57 80145536651832 17 7300 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:29.796+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 59 106620692996001 23 7300 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:29.799+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 60 279733370822238 23 7300 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:34:29.802+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 61 223881084081370 17 7300 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:29.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 62 36300629322478 32 7300 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:34:29.810+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query InstanceIds [3464125101225762400] 2023-10-21T04:34:29.810+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 7300 2023-10-21T04:34:29.810+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7300 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 3464125101225762400 INDEX_STATE_CATCHUP 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 3464125101225762400 true 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 3464125101225762400 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query Session: 7300 BuildTS: [0=23,1=25,2=28,3=24,4=18,5=23,6=23,7=17,8=21,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=28,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=23,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_system:_query, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 26 24 26} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 29 27 29} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 24 22 24} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 32 0 32} { 63 1755d24e6a65 2 17 0 17} , fetchKVSeq false 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:34:29.810+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:34:29.810+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7300 STREAM_ACTIVE 2023-10-21T04:34:29.810+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 7300 2023-10-21T04:34:29.811+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_system:_query 7300 2023-10-21T04:34:29.811+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:29.811+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:29.811+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:29.811+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:29.811+05:30 [Info] send progress stats to clients 2023-10-21T04:34:29.811+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:34:29.811+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:34:29.811+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:3] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:34:29.811+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 7300 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:34:29.812+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:34:29.814+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=4,1=26,2=4,3=4,4=18,5=4,6=4,7=4,8=22,9=4,10=4,11=4,12=4,13=4,14=4,15=4,16=29,17=4,18=4,19=4,20=4,21=4,22=4,23=4,24=4,25=4,26=4,27=4,28=4,29=21,30=4,31=4,32=4,33=4,34=4,35=4,36=4,37=4,38=4,39=4,40=4,41=4,42=4,43=26,44=4,45=24,46=4,47=4,48=4,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=4,57=4,58=4,59=4,60=4,61=4,62=33,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:29.814+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:34:29.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 63 25657368013413 17 7300 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:34:29.819+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 2023-10-21T04:34:29.820+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7300 2023-10-21T04:34:29.820+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7300 2023-10-21T04:34:29.820+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:34:29.820+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:34:29.820+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 3464125101225762400 false 2023-10-21T04:34:29.820+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_system:_query 3464125101225762400 2023-10-21T04:34:29.820+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_system:_query. 2023-10-21T04:34:29.820+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 7300 2023-10-21T04:34:29.822+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.585217ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:29.833+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:29.833+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:29.836+05:30 [Info] switched currmeta from 114 -> 115 force true 2023-10-21T04:34:29.858+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33906 2023-10-21T04:34:29.873+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 11.025962ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:29.902+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33906. Reason = EOF. Cleaning Pipe. 2023-10-21T04:34:29.903+05:30 [Info] messageListener.start(): message channel closed. Remove peer 1:96:10:27:96:4:46:82 as follower. 2023-10-21T04:34:30.106+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is enabled 2023-10-21T04:34:30.269+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:30.375+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:34:30.456+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:30.458+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:30.461+05:30 [Info] switched currmeta from 115 -> 116 force true 2023-10-21T04:34:30.813+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:34:30.813+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_system:_query 2023-10-21T04:34:30.813+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 3464125101225762400 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId: 7300, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 26 24 26} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 29 27 29} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 24 22 24} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 33 0 33} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:34:30.813+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 3464125101225762400 INDEX_STATE_ACTIVE 2023-10-21T04:34:30.813+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_system:_query State Changed to INACTIVE 2023-10-21T04:34:30.813+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:34:30.813+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:34:30.813+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 7300 2023-10-21T04:34:30.813+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:30.814+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:30.814+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:30.814+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:34:30.814+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 7300 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:30.814+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3464125101225762400 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:34:30.814+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:34:30.814+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:34:30.814+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:34:30.814+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 7300 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:30.814+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:34:30.815+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query total received key-versions: map[0:0 1:1 2:0 3:0 4:0 5:0 6:0 7:0 8:1 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:1 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:1 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:1 63:0] 2023-10-21T04:34:30.815+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query latest sequence numbers: map[0:0 1:26 2:0 3:0 4:0 5:0 6:0 7:0 8:22 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:29 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:24 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:33 63:0] 2023-10-21T04:34:30.815+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:34:30.815+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:34654: use of closed network connection 2023-10-21T04:34:30.815+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:34654" exit: read tcp4 127.0.0.1:9121->127.0.0.1:34654: use of closed network connection 2023-10-21T04:34:30.815+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:34654" closed ! 2023-10-21T04:34:30.815+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:34:30.815+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:30.815+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:30.815+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:34:30.815+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:34:30.815+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:34:30.815+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:34:30.815+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId 7300 2023-10-21T04:34:30.816+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:34:30.831+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query]. Err genServer.closed 2023-10-21T04:34:30.831+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:34:30.832+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Treating genServer.closed As Success 2023-10-21T04:34:30.832+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7300 2023-10-21T04:34:30.832+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7300 2023-10-21T04:34:30.831+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.044382ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:30.833+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:34:30.877+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:30.877+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:30.881+05:30 [Info] switched currmeta from 116 -> 117 force true 2023-10-21T04:34:30.937+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:31.000+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:31.000+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:31.006+05:30 [Info] switched currmeta from 117 -> 118 force true 2023-10-21T04:34:31.274+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:34:31.274+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3464125101225762400 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007495440 2023-10-21T04:34:31.274+05:30 [Info] handleUpdateIndexRState: Index instance 3464125101225762400 rstate moved to ACTIVE 2023-10-21T04:34:31.282+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.747396ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:31.282+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[3464125101225762400:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:34:31.282+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken75:68:3d:12:64:22:c1:3, elapsed(sec): 8.653801339 2023-10-21T04:34:31.295+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:31.295+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken75:68:3d:12:64:22:c1:3 2023-10-21T04:34:31.383+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:31.383+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:31.393+05:30 [Info] switched currmeta from 118 -> 119 force true 2023-10-21T04:34:31.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:31.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:31.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:31.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:31.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:31.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697842901255695677, Old rollbacktime: 0 2023-10-21T04:34:31.403+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken60:98:e:25:85:32:29:6a MasterId: SourceId: DestId: RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken75:68:3d:12:64:22:c1:3 SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:31.812+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:34:32.071+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:34:32.071+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:34:32.071+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:34:32.436+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.436+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.458+05:30 [Info] switched currmeta from 119 -> 121 force true 2023-10-21T04:34:32.502+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.502+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.521+05:30 [Info] switched currmeta from 121 -> 124 force true 2023-10-21T04:34:32.536+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.539+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.546+05:30 [Info] switched currmeta from 124 -> 124 force true 2023-10-21T04:34:32.556+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.556+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.560+05:30 [Info] switched currmeta from 124 -> 125 force true 2023-10-21T04:34:32.609+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.609+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.612+05:30 [Info] switched currmeta from 125 -> 126 force true 2023-10-21T04:34:32.639+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.639+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.650+05:30 [Info] switched currmeta from 126 -> 126 force true 2023-10-21T04:34:32.668+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.668+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.705+05:30 [Info] switched currmeta from 126 -> 128 force true 2023-10-21T04:34:32.712+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.712+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.715+05:30 [Info] switched currmeta from 128 -> 128 force true 2023-10-21T04:34:32.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.753+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.765+05:30 [Info] switched currmeta from 128 -> 128 force true 2023-10-21T04:34:32.831+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.831+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.838+05:30 [Info] switched currmeta from 128 -> 129 force true 2023-10-21T04:34:32.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.861+05:30 [Info] switched currmeta from 129 -> 130 force true 2023-10-21T04:34:32.905+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.905+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.913+05:30 [Info] switched currmeta from 130 -> 131 force true 2023-10-21T04:34:32.930+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.930+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:32.939+05:30 [Info] switched currmeta from 131 -> 131 force true 2023-10-21T04:34:32.987+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:32.988+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:33.029+05:30 [Info] switched currmeta from 131 -> 132 force true 2023-10-21T04:34:33.210+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:33.210+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:33.246+05:30 [Info] switched currmeta from 132 -> 134 force true 2023-10-21T04:34:33.498+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:33.499+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:33.520+05:30 [Info] switched currmeta from 134 -> 135 force true 2023-10-21T04:34:33.611+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:33.611+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:33.675+05:30 [Info] switched currmeta from 135 -> 135 force true 2023-10-21T04:34:33.887+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:33.887+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [16787445029868804812 2925063509317026816] 2023-10-21T04:34:33.887+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : all deamons started 2023-10-21T04:34:33.887+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : all deamons started 2023-10-21T04:34:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:34:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:34:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : RestoreShardDone completed 2023-10-21T04:34:33.887+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : all deamons started 2023-10-21T04:34:33.887+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : all deamons started 2023-10-21T04:34:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:34:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:34:33.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : RestoreShardDone completed 2023-10-21T04:34:33.887+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [16787445029868804812 2925063509317026816], elapsed: 294.071µs 2023-10-21T04:34:33.887+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[2925063509317026816:true 16787445029868804812:true] 2023-10-21T04:34:33.887+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 9570541924043341912, partnId: 0 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 2 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 3 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 4 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 5 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 6 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 7 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 8 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 10032435514095102923, partnId: 1 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 3464125101225762400, partnId: 0 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 2 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 3 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 4 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 5 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 6 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 7 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 8 2023-10-21T04:34:33.888+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 15864358084358521096, partnId: 1 2023-10-21T04:34:33.888+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.458µs 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:34:33.888+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [16787445029868804812 2925063509317026816] 2023-10-21T04:34:33.888+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [16787445029868804812 2925063509317026816], errMap: map[2925063509317026816: 16787445029868804812:], elapsed: 8.251µs 2023-10-21T04:34:33.889+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:34:33.889+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_DONE]} 2023-10-21T04:34:33.889+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_DONE] 2023-10-21T04:34:33.889+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.868µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:33.915+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken75:68:3d:12:64:22:c1:3 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 964f96068adef144590fc7ab85b2311c (127.0.0.1:9002) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken78:15:9:44:9a:2d:c7:38 Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11088380021860709797 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9570541924043341912 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6853580018063392983 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:33.993+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken75:68:3d:12:64:22:c1:3 2023-10-21T04:34:34.131+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken78:15:9:44:9a:2d:c7:38 MasterId: 8806f5706747af4e635b8da5fcd8aeee SourceId: 7f731cd927c72e28ff24e69497b93571 (127.0.0.1:9001) DestId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) RebalId: a4a273dac27463ee6d67902ad7340fd2 ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken75:68:3d:12:64:22:c1:3 Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12264640536674801671 RealInstId: 3345094951175719474 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11821615801380882181 RealInstId: 6377153279641237139 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11645751077768222772 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10601152754683910907 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:34:34.274+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken78:15:9:44:9a:2d:c7:38 2023-10-21T04:34:34.487+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996437463924477, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:34:34.585+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = EOF. Cleaning Pipe. 2023-10-21T04:34:34.585+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly. 2023-10-21T04:34:34.590+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:59992. Error = read tcp4 127.0.0.1:9118->127.0.0.1:59992: read: connection reset by peer. Kill Pipe. 2023-10-21T04:34:34.590+05:30 [Info] messageListener.start(): message channel closed. Remove peer 69:f6:b5:13:a6:83:5c:2 as follower. 2023-10-21T04:34:34.693+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:34:34.693+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a4a273dac27463ee6d67902ad7340fd2 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a4a273dac27463ee6d67902ad7340fd2]}]} 2023-10-21T04:34:34.798+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:34:34.798+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:34:34.825+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/a4a273dac27463ee6d67902ad7340fd2, rev: [] 2023-10-21T04:34:34.825+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/a4a273dac27463ee6d67902ad7340fd2, rev: [] 2023-10-21T04:34:34.825+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:34:34.825+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:34:34.825+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:34:34.825+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:34:34.825+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:34:34.876+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:34:34.876+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:34:34.876+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.664µs 2023-10-21T04:34:34.876+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:34:34.876+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:34:34.876+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:34:34.921+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:34:34.922+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:34:34.922+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:34:34.922+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:34:34.922+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 30.949µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:34.929+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:34:34.929+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:34:34.929+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:34:34.929+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:34:34.930+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:34:34.931+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:34:34.932+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:34:34.932+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 35.993µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:34.940+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:34:34.942+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:34:34.942+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/a4a273dac27463ee6d67902ad7340fd2, rev: [] call. err: 2023-10-21T04:34:34.942+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:34:34.942+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:34:34.954+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:34:34.954+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:34:34.954+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:34:34.986+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:34:34.986+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:34:35.005+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:34:35.008+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:34:35.384+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:35.384+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:35.395+05:30 [Info] switched currmeta from 135 -> 135 force true 2023-10-21T04:34:35.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9995368407262137, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:34:35.737+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:35.738+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to false 2023-10-21T04:34:35.794+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:35.794+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:35.834+05:30 [Info] switched currmeta from 135 -> 135 force true 2023-10-21T04:34:36.487+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9881546546051152 2023-10-21T04:34:37.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8510749612584902, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:34:37.588+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:34:38.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.5984642864819122, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:34:38.827+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:34:38.895+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44912 2023-10-21T04:34:38.964+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:34308 2023-10-21T04:34:39.025+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 40.140742ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:39.026+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 107.71µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:39.105+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45904" doAuth() ... 2023-10-21T04:34:39.106+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:45904" doReceive() ... 2023-10-21T04:34:39.527+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:34:39.745+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:39.752+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:39.752+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:39.757+05:30 [Info] switched currmeta from 135 -> 139 force true 2023-10-21T04:34:42.824+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 1697842901234041110 2023-10-21T04:34:42.825+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 1697842901234041110 2023-10-21T04:34:42.826+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 1697842901255695677 {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 1697842901234041110 2023-10-21T04:34:42.826+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 1697842901234041110 {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 1697842901255695677 2023-10-21T04:34:42.826+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 1697842901234041110 {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 11645751077768222772:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 1697842901255695677 2023-10-21T04:34:42.826+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2535881715450691825:[0] 3010072456373199569:[0] 6043349427976659583:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 1697842901234041110 {Index inst : [partitions]}: map[3345094951175719474:[1 2 3 4 5 6 7 8] 10601152754683910907:[0] 11645751077768222772:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 1697842901255695677 2023-10-21T04:34:46.783+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:34:56.951+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 9570541924043341912 &{0} 2023-10-21T04:34:56.951+05:30 [Info] Indexer::handleDropIndex - IndexInstId 9570541924043341912 2023-10-21T04:34:56.954+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:34:56.954+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:56.954+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 9570541924043341912 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:34:56.954+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [9570541924043341912] 2023-10-21T04:34:56.954+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [9570541924043341912] 2023-10-21T04:34:56.954+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:56.954+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [9570541924043341912] 2023-10-21T04:34:56.954+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:56.955+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 9570541924043341912 Defn: DefnId: 7147407900589480727 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:56.955+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.016024ms 2023-10-21T04:34:56.955+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 9570541924043341912, PartitionId 0, IndexDefnId 7147407900589480727. 2023-10-21T04:34:56.955+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:56.955+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:56.956+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:56.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex] 2023-10-21T04:34:56.956+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex 2023-10-21T04:34:56.956+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex] 2023-10-21T04:34:56.956+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex] 2023-10-21T04:34:56.956+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex 2023-10-21T04:34:56.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex stopped 2023-10-21T04:34:56.956+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:56.956+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:56.956+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:56.956+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:56.956+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [9570541924043341912] 2023-10-21T04:34:56.956+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 9570541924043341912 2023-10-21T04:34:56.960+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:34:56.960+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 61685 2023-10-21T04:34:56.960+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.505048ms 2023-10-21T04:34:56.960+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 8.59046ms 2023-10-21T04:34:56.971+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 27.607716ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:56.972+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:56.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex] 2023-10-21T04:34:56.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex closed 2023-10-21T04:34:56.974+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:56.974+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:56.974+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:56.974+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:56.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex] 2023-10-21T04:34:56.974+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex 2023-10-21T04:34:56.974+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex] 2023-10-21T04:34:56.975+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex] 2023-10-21T04:34:56.975+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex 2023-10-21T04:34:56.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex stopped 2023-10-21T04:34:56.976+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:56.976+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:56.976+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:56.976+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:57.008+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:57.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex] 2023-10-21T04:34:57.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex closed 2023-10-21T04:34:57.009+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:57.010+05:30 [Info] Indexer::cleanupIndexData IndexInst 9570541924043341912 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:57.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex ... 2023-10-21T04:34:57.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex ... 2023-10-21T04:34:57.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:34:57.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/mainIndex sucessfully destroyed 2023-10-21T04:34:57.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex ... 2023-10-21T04:34:57.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex ... 2023-10-21T04:34:57.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:34:57.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_9570541924043341912_0.index destroyed 2023-10-21T04:34:57.026+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 9570541924043341912, PartitionId 0, IndexDefnId 7147407900589480727. 2023-10-21T04:34:57.026+05:30 [Info] Indexer::cleanupIndexData IndexInst 9570541924043341912 Partition 0 Destroy Done 2023-10-21T04:34:57.095+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:57.095+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:57.110+05:30 [Info] switched currmeta from 139 -> 143 force true 2023-10-21T04:34:57.143+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15165282789964834431 &{0} 2023-10-21T04:34:57.143+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15165282789964834431 2023-10-21T04:34:57.144+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:57.144+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:57.144+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15165282789964834431 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:34:57.145+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15165282789964834431] 2023-10-21T04:34:57.145+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15165282789964834431] 2023-10-21T04:34:57.145+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:34:57.145+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15165282789964834431] 2023-10-21T04:34:57.145+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 62731 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:57.145+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15165282789964834431 Defn: DefnId: 12599498416578472779 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 62731 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:34:57.146+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.266268ms 2023-10-21T04:34:57.146+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15165282789964834431, PartitionId 0, IndexDefnId 12599498416578472779. 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:57.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex] 2023-10-21T04:34:57.146+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex] 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex] 2023-10-21T04:34:57.146+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex 2023-10-21T04:34:57.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex stopped 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:57.146+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:57.146+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15165282789964834431 2023-10-21T04:34:57.147+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [15165282789964834431] 2023-10-21T04:34:57.153+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:34:57.153+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 62731 2023-10-21T04:34:57.153+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 7.214924ms 2023-10-21T04:34:57.153+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 9.541375ms 2023-10-21T04:34:57.155+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:57.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex] 2023-10-21T04:34:57.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex closed 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Mainstore#15165282789964834431:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:34:57.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex] 2023-10-21T04:34:57.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex] 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex] 2023-10-21T04:34:57.157+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex 2023-10-21T04:34:57.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex stopped 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:34:57.157+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:34:57.167+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:34:57.167+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex] 2023-10-21T04:34:57.167+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex closed 2023-10-21T04:34:57.168+05:30 [Info] bucket_%2/idx_secondary/Backstore#15165282789964834431:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:34:57.168+05:30 [Info] Indexer::cleanupIndexData IndexInst 15165282789964834431 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:34:57.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex ... 2023-10-21T04:34:57.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex ... 2023-10-21T04:34:57.169+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 37.274592ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:34:57.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:34:57.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/docIndex sucessfully destroyed 2023-10-21T04:34:57.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex ... 2023-10-21T04:34:57.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex ... 2023-10-21T04:34:57.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:34:57.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15165282789964834431_0.index destroyed 2023-10-21T04:34:57.209+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15165282789964834431, PartitionId 0, IndexDefnId 12599498416578472779. 2023-10-21T04:34:57.209+05:30 [Info] Indexer::cleanupIndexData IndexInst 15165282789964834431 Partition 0 Destroy Done 2023-10-21T04:34:57.307+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:57.307+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:57.337+05:30 [Info] switched currmeta from 143 -> 147 force true 2023-10-21T04:34:57.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:57.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:57.363+05:30 [Info] switched currmeta from 147 -> 147 force true 2023-10-21T04:34:59.796+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:34:59.802+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:34:59.802+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:34:59.804+05:30 [Info] switched currmeta from 147 -> 147 force true 2023-10-21T04:35:05.006+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:35:05.006+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:35:05.006+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:35:05.007+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:35:05.008+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:35:05.008+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:35:05.010+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:35:09.626+05:30 [Info] memstats {"Alloc":147188800, "TotalAlloc":500255680, "Sys":261708088, "Lookups":0, "Mallocs":4587929,"Frees":4259095, "HeapAlloc":147188800, "HeapSys":238485504, "HeapIdle":82010112, "HeapInuse":156475392,"HeapReleased":2621440, "HeapObjects":328834,"MSpanInuse":830400, "MSpanSys": 1909440, "StackInuse": 4784128,"GCSys":14059728, "LastGC":1697843045331149618,"PauseTotalNs":19213768, "PauseNs":[288816, 5422899, 224226], "NumGC":9} 2023-10-21T04:35:10.215+05:30 [Info] connected with 4 indexers 2023-10-21T04:35:10.215+05:30 [Info] client stats current counts: current: 12, not current: 0 2023-10-21T04:35:10.306+05:30 [Info] num concurrent scans {0} 2023-10-21T04:35:10.306+05:30 [Info] average scan response {0 ms} 2023-10-21T04:35:10.339+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:35:10.369+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:35:10.372+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:35:10.407+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:35:10.586+05:30 [Info] janitor: running cleanup. 2023-10-21T04:35:10.586+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[7147407900589480727 7392796230534034390 8504686614261377811 15945117906390072636 15968368593314043432 18046371353728341033 11619165464560280276 14292583894719865678 15431181238515168303 4535197604682045916 13268712595133993155 4860549750398470299 573521267164877540 1252359852432998131 12140458060295645238 3674078063353481907 12195715103909237684 2048006213723315482 14741949230702299196 12599498416578472779 8857269605176042330 12338036531800130039 13856704970496601958 5465835793930010953 16162012140118277065 249887442666643345 18323460706027642575 12824340686309573428 8669649359784409226 18026660730857541767 4138047918192025493 12027054384338714804 17797267771846799851 12833670934079048728 4790214984019693894 6985049432225870291 7525085113233084253 4532813151040661705 11365633301282701856 8378417662949994532 17656144409930763141 14889401051926000176]} 2023-10-21T04:35:12.092+05:30 [Info] Indexer::ReadMemstats Time Taken 209.541µs 2023-10-21T04:35:16.785+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:35:17.690+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 12582561312368048736 2023-10-21T04:35:17.690+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 169.286µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:17.747+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48272 2023-10-21T04:35:17.757+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48272 2023-10-21T04:35:17.779+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48272 2023-10-21T04:35:17.830+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12582561312368048736, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:35:17.831+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12582561312368048736, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:35:17.848+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 15055366427687156590, indexDefn DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:35:17.848+05:30 [Info] Indexer::handleCreateIndex InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:35:17.848+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15055366427687156590 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:35:17.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:35:17.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery) 2023-10-21T04:35:17.852+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:35:17.853+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:35:17.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:35:17.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery) 2023-10-21T04:35:17.856+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:35:17.856+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:35:17.864+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:17.864+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:17.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex 2023-10-21T04:35:17.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 2023-10-21T04:35:17.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data 2023-10-21T04:35:17.865+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex 2023-10-21T04:35:17.865+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex 2023-10-21T04:35:17.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex started 2023-10-21T04:35:17.865+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:35:17.865+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:35:17.872+05:30 [Info] switched currmeta from 147 -> 150 force true 2023-10-21T04:35:17.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex 2023-10-21T04:35:17.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 2023-10-21T04:35:17.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data 2023-10-21T04:35:17.876+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex 2023-10-21T04:35:17.876+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex 2023-10-21T04:35:17.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex started 2023-10-21T04:35:17.876+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15055366427687156590 partitionId 0 WriterThreads 4 2023-10-21T04:35:17.876+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15055366427687156590 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index Index: 15055366427687156590 Partition: 0 2023-10-21T04:35:17.876+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15055366427687156590 2023-10-21T04:35:17.876+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:17.876+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:35:17.876+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:17.877+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:35:17.876+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:17.877+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:35:17.877+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 28.607999ms 2023-10-21T04:35:17.877+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 28.657893ms 2023-10-21T04:35:17.877+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 15055366427687156590, partnShardIdMap: map[0:[16787445029868804812 2925063509317026816]], indexDefn DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:35:17.891+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:17.891+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:17.893+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [15055366427687156590] &{0} 2023-10-21T04:35:17.893+05:30 [Info] Indexer::handleBuildIndex [15055366427687156590] 2023-10-21T04:35:17.894+05:30 [Info] bulkUpdateRState: Index instance 15055366427687156590 rstate moved to ACTIVE 2023-10-21T04:35:17.894+05:30 [Info] Indexer::handleBuildIndex Added Index: [15055366427687156590] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:35:17.895+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:17.895+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:17.895+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:17.896+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:35:17.897+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:17.897+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:17.897+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66597 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:17.897+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:35:17.897+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:35:17.898+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:35:17.898+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66597 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:17.898+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:35:17.898+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:17.898+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:17.898+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:35:17.899+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:35:17.899+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:17.899+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:17.899+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:35:17.899+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:3] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:35:17.900+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15055366427687156590 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:35:17.900+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:35:17.900+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 66598 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:35:17.901+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:35:17.902+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:35:17.903+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:35:17.903+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 9.458437ms 2023-10-21T04:35:17.903+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 9.496282ms 2023-10-21T04:35:17.903+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [15055366427687156590] 2023-10-21T04:35:17.919+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 12582561312368048736 2023-10-21T04:35:17.919+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 89.18847ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:17.920+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:35:17.926+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:35:17.927+05:30 [Info] switched currmeta from 150 -> 154 force true 2023-10-21T04:35:17.947+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 28.587395ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:18.013+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:18.013+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:18.026+05:30 [Info] switched currmeta from 154 -> 156 force true 2023-10-21T04:35:18.033+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:18.033+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:18.035+05:30 [Info] switched currmeta from 156 -> 156 force true 2023-10-21T04:35:18.047+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35958" doAuth() ... 2023-10-21T04:35:18.047+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35958" auth successful 2023-10-21T04:35:18.047+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:35958" +1 2023-10-21T04:35:18.135+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35202 2023-10-21T04:35:18.165+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 18.978986ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:18.207+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 36.217µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:18.227+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35202. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:18.227+05:30 [Info] messageListener.start(): message channel closed. Remove peer be:ee:63:20:bf:3d:a2:76 as follower. 2023-10-21T04:35:18.516+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 1949031200982 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.516+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 245782287380001 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.520+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 205201940883916 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 244791485011286 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 93079113162334 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 52522903947059 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.527+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 167580527121081 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.527+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 177815184127237 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.529+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 54236768606397 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.534+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 160381752747972 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.535+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 58522600746999 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.536+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 142387448675642 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.541+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 134409829206048 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.541+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 90115353725987 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.541+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 161230548567164 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.543+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 85302001459467 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.549+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 213108110408291 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.549+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 123957315902602 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 163460438181870 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 150690956328758 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 151273673850546 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 147946970349859 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.554+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 258671969827141 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.560+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 19989312661772 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.560+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 53829314402679 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 114823206239184 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 68847799765163 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.568+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 232028101514735 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.568+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 25510109770064 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 107407811917992 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 8305763409275 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 124811039198764 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 207489375637709 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.578+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 33310272161722 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 203503670520010 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 228857796432725 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 100746255343609 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 117298435979365 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.586+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 47156288300713 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.591+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 104136033125373 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.592+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 236904976023311 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.593+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 158357249947806 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.602+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 172365063201867 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.603+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 276463747117997 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.604+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 196269910725961 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.605+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 246959094981553 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.606+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 124310544527292 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.607+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 65885584526743 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 79077905592013 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 212884822297038 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 246582249071123 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 213334533059126 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 254944616021293 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.614+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 55145461136105 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.618+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 80145536651832 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.619+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 25511316571199 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.624+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 223881084081370 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.625+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 142324128597367 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 36300629322478 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.629+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 InstanceIds [15055366427687156590] 2023-10-21T04:35:18.629+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 66598 2023-10-21T04:35:18.629+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:35:18.629+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:35:18.629+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_1:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:35:18.629+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 56 Crc64: 5435225025284402718 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 26 0 26} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 0 0 0} { 4 54a7ab79de5e 2 0 0 0} { 5 dea2f5821556 2 0 0 23} { 6 9869e1ca52b9 2 0 0 0} { 7 a1b8d2e2b505 0 0 0 0} { 8 3153fb5398bd 0 0 0 0} { 9 7a3eba1fc820 0 0 0 0} { 10 91ddc966a7c4 0 0 0 0} { 11 3539dad42ff7 0 0 0 0} { 12 8180294c7d3a 0 0 0 0} { 13 51f59da5cc23 0 0 0 0} { 14 92a36993047c 0 0 0 0} { 15 4d94eb7ee10b 0 0 0 0} { 16 70bd0fc81c8a 0 0 0 0} { 17 c1d219383a63 0 0 0 0} { 18 94aa995a47ee 0 0 0 0} { 19 890d78ec3f36 0 0 0 0} { 20 89952597feb2 0 0 0 0} { 21 868e969d0123 0 0 0 0} { 22 eb42c2dff945 0 0 0 0} { 23 30f51d29d977 0 0 0 0} { 24 122e1fe1690c 0 0 0 0} { 25 686e5c6b7bd0 0 0 0 0} { 26 3e9de0a608ab 0 0 0 0} { 27 d307409195ef 0 0 0 0} { 28 17338907f150 0 0 0 0} { 29 78dd613dd7b 0 0 0 0} { 30 7183d5a7ce2c 0 0 0 0} { 31 bcb5e28790cd 0 0 0 0} { 32 1e4ba6f087ba 0 0 0 0} { 33 61afd479b8a8 0 0 0 0} { 34 5ba0d0c31ff9 0 0 0 0} { 35 6aaeab9bf865 0 0 0 0} { 36 d0251ba91355 0 0 0 0} { 37 b915e3fde8ca 0 0 0 0} { 38 2ae36dd026a9 0 0 0 0} { 39 90066ba1789e 0 0 0 0} { 40 5eb60f4343fd 0 0 0 0} { 41 d776bcf0ff0f 0 0 0 0} { 42 710f4dd92fbc 0 0 0 0} { 43 b281a64f1549 0 0 0 0} { 44 e09ba52163b1 0 0 0 0} { 45 fb713b70b3ad 0 0 0 0} { 46 3bec2edc8d97 0 0 0 0} { 47 9cc3de5d244b 0 0 0 0} { 48 47ebc278eecd 0 0 0 0} { 49 c19e1c3609ce 0 0 0 0} { 50 e043e75d2213 0 0 0 0} { 51 c206d10fb236 0 0 0 0} { 53 e7deeb467d2d 0 0 0 0} { 54 32278d9f3ae9 0 0 0 0} { 57 48e4563e1238 0 0 0 0} { 58 1733d0f6443f 0 0 0 0} HWT bucket: bucket_1, scopeId: , collectionId: , vbuckets: 59 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 26 0 26} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 21 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 7 0 23} { 6 9869e1ca52b9 2 8 0 23} { 7 a1b8d2e2b505 0 0 0 0} { 8 3153fb5398bd 0 0 0 0} { 9 7a3eba1fc820 0 0 0 0} { 10 91ddc966a7c4 0 0 0 0} { 11 3539dad42ff7 0 0 0 0} { 12 8180294c7d3a 0 0 0 0} { 13 51f59da5cc23 0 0 0 0} { 14 92a36993047c 0 0 0 0} { 15 4d94eb7ee10b 0 0 0 0} { 16 70bd0fc81c8a 0 0 0 0} { 17 c1d219383a63 0 0 0 0} { 18 94aa995a47ee 0 0 0 0} { 19 890d78ec3f36 0 0 0 0} { 20 89952597feb2 0 0 0 0} { 21 868e969d0123 0 0 0 0} { 22 eb42c2dff945 0 0 0 0} { 23 30f51d29d977 0 0 0 0} { 24 122e1fe1690c 0 0 0 0} { 25 686e5c6b7bd0 0 0 0 0} { 26 3e9de0a608ab 0 0 0 0} { 27 d307409195ef 0 0 0 0} { 28 17338907f150 0 0 0 0} { 29 78dd613dd7b 0 0 0 0} { 30 7183d5a7ce2c 0 0 0 0} { 31 bcb5e28790cd 0 0 0 0} { 32 1e4ba6f087ba 0 0 0 0} { 33 61afd479b8a8 0 0 0 0} { 34 5ba0d0c31ff9 0 0 0 0} { 35 6aaeab9bf865 0 0 0 0} { 36 d0251ba91355 0 0 0 0} { 37 b915e3fde8ca 0 0 0 0} { 38 2ae36dd026a9 0 0 0 0} { 39 90066ba1789e 0 0 0 0} { 40 5eb60f4343fd 0 0 0 0} { 41 d776bcf0ff0f 0 0 0 0} { 42 710f4dd92fbc 0 0 0 0} { 43 b281a64f1549 0 0 0 0} { 44 e09ba52163b1 0 0 0 0} { 45 fb713b70b3ad 0 0 0 0} { 46 3bec2edc8d97 0 0 0 0} { 47 9cc3de5d244b 0 0 0 0} { 48 47ebc278eecd 0 0 0 0} { 49 c19e1c3609ce 0 0 0 0} { 50 e043e75d2213 0 0 0 0} { 51 c206d10fb236 0 0 0 0} { 52 81716b20c577 0 0 0 0} { 53 e7deeb467d2d 0 0 0 0} { 54 32278d9f3ae9 0 0 0 0} { 57 48e4563e1238 0 0 0 0} { 58 1733d0f6443f 0 0 0 0} { 61 cb9e607d14da 0 0 0 0} { 62 2103e61daaee 0 0 0 0} 2023-10-21T04:35:18.629+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:35:18.629+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:35:18.630+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:35:18.631+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:18.636+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 194568213794435 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.638+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 106620692996001 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.638+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 279733370822238 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.638+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 219528837936411 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.638+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 25657368013413 0 66598 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:18.667+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:18.687+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:35:18.687+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:35:18.699+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:35:18.746+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35236 2023-10-21T04:35:18.771+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 19.090012ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:18.789+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:35:18.789+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:35:18.789+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:35:18.804+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:35:18.839+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:35:18.839+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:35:18.839+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:35:18.849+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:35:18.893+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:35:18.894+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:35:18.894+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:35:18.902+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35254 2023-10-21T04:35:18.902+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:35:18.908+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35236. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:18.908+05:30 [Info] messageListener.start(): message channel closed. Remove peer e3:3d:43:78:3e:c5:39:59 as follower. 2023-10-21T04:35:18.925+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 17.253596ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:18.926+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:35:18.926+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:35:18.926+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:35:18.926+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:35:18.926+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:35:18.926+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:35:18.926+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:35:18.926+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:35:18.926+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:35:18.926+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:35:18.926+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:54728->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:18.926+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:55192->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:18.927+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:59838->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:18.927+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:35254->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:18.929+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35254. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:18.929+05:30 [Info] messageListener.start(): message channel closed. Remove peer 63:c7:56:47:76:99:26:65 as follower. 2023-10-21T04:35:18.962+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35258 2023-10-21T04:35:18.984+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 14.586412ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:19.056+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35258. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:19.056+05:30 [Info] messageListener.start(): message channel closed. Remove peer d3:e:56:65:6b:63:ca:30 as follower. 2023-10-21T04:35:19.078+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:19.078+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:19.081+05:30 [Info] switched currmeta from 156 -> 157 force true 2023-10-21T04:35:19.121+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:35:19.125+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15055366427687156590 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:35:19.125+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15055366427687156590 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.148µs SnapOpenDur 137.865µs 2023-10-21T04:35:19.125+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 15055366427687156590 INDEX_STATE_CATCHUP 2023-10-21T04:35:19.125+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 15055366427687156590 true 2023-10-21T04:35:19.125+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 15055366427687156590 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 66598 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] 2023-10-21T04:35:19.126+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15055366427687156590, PartitionId 0 SnapshotId 23 Creating recovery point ... 2023-10-21T04:35:19.126+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 66598 2023-10-21T04:35:19.126+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 66598 2023-10-21T04:35:19.126+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:19.126+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:19.126+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:19.127+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:35:19.127+05:30 [Info] send progress stats to clients 2023-10-21T04:35:19.127+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:35:19.127+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:35:19.127+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:4] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:35:19.127+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 66598 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:35:19.130+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:35:19.135+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 2023-10-21T04:35:19.135+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66598 2023-10-21T04:35:19.135+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66598 2023-10-21T04:35:19.135+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:35:19.136+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:35:19.136+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 15055366427687156590 false 2023-10-21T04:35:19.136+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 15055366427687156590 2023-10-21T04:35:19.136+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:35:19.136+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 66598 2023-10-21T04:35:19.137+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 9.739078ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:19.154+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:19.154+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:19.159+05:30 [Info] switched currmeta from 157 -> 158 force true 2023-10-21T04:35:19.177+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15055366427687156590, PartitionId 0 SnapshotId 23 Created recovery point (took 51.263871ms) 2023-10-21T04:35:19.272+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:19.427+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:19.485+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:19.650+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:35:19.650+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:35:19.650+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 15055366427687156590 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 66598, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 26 0 26} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 29 0 29} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 0 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 24 0 24} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 24 0 24} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 33 0 33} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:35:19.650+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 15055366427687156590 INDEX_STATE_ACTIVE 2023-10-21T04:35:19.650+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:35:19.650+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:35:19.650+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:35:19.650+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 29 CurrMaint 0 New 0 2023-10-21T04:35:19.650+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 66598 2023-10-21T04:35:19.651+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:19.651+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:19.651+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:19.651+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66598 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:19.651+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15055366427687156590 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:35:19.651+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:35:19.651+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:35:19.651+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:35:19.651+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66598 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:19.651+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:35:19.651+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:16 1:14 2:21 3:17 4:10 5:16 6:16 7:10 8:13 9:19 10:11 11:17 12:23 13:14 14:18 15:14 16:16 17:16 18:27 19:19 20:18 21:17 22:12 23:25 24:8 25:18 26:18 27:12 28:21 29:12 30:11 31:5 32:17 33:13 34:12 35:17 36:18 37:17 38:17 39:14 40:22 41:15 42:14 43:18 44:27 45:12 46:13 47:14 48:10 49:18 50:13 51:18 52:15 53:17 54:9 55:18 56:22 57:10 58:9 59:16 60:16 61:10 62:25 63:10] 2023-10-21T04:35:19.652+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:23 1:21 2:28 3:24 4:17 5:23 6:23 7:17 8:20 9:26 10:18 11:24 12:30 13:21 14:25 15:21 16:23 17:23 18:34 19:26 20:25 21:24 22:19 23:32 24:15 25:25 26:25 27:19 28:28 29:19 30:18 31:12 32:24 33:20 34:19 35:24 36:25 37:24 38:24 39:21 40:29 41:22 42:21 43:25 44:34 45:19 46:20 47:21 48:17 49:25 50:20 51:25 52:22 53:24 54:16 55:25 56:29 57:17 58:16 59:23 60:23 61:17 62:32 63:17] 2023-10-21T04:35:19.652+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:35:19.652+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:35958: use of closed network connection 2023-10-21T04:35:19.652+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:35958" exit: read tcp4 127.0.0.1:9121->127.0.0.1:35958: use of closed network connection 2023-10-21T04:35:19.652+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35958" closed ! 2023-10-21T04:35:19.652+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:35:19.653+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:19.653+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:19.653+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:19.653+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:19.653+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:35:19.653+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:35:19.654+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 66598 2023-10-21T04:35:19.656+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:35:19.661+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.710914ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:19.689+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:19.689+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:19.692+05:30 [Info] switched currmeta from 158 -> 160 force true 2023-10-21T04:35:19.727+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:19.727+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:19.735+05:30 [Info] switched currmeta from 160 -> 160 force true 2023-10-21T04:35:19.752+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:35:19.752+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:35:19.752+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:35:19.752+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66598 2023-10-21T04:35:19.752+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66598 2023-10-21T04:35:19.752+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:35:19.847+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:19.853+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:19.853+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:19.854+05:30 [Info] switched currmeta from 160 -> 160 force true 2023-10-21T04:35:20.631+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:35:20.820+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1361754384643230279:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 2023-10-21T04:35:21.071+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15055366427687156590:[0]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:35:23.989+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 14549775068266281786 2023-10-21T04:35:23.989+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 117.708µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:24.047+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48272 2023-10-21T04:35:24.050+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48272 2023-10-21T04:35:24.078+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48272 2023-10-21T04:35:24.131+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 14549775068266281786, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:35:24.132+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 14549775068266281786, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:35:24.166+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 3127033699511712809, indexDefn DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:35:24.171+05:30 [Info] Indexer::handleCreateIndex InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:35:24.173+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3127033699511712809 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:35:24.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:35:24.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:35:24.181+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:35:24.181+05:30 [Info] bucket_%2/idx_secondary/Mainstore#3127033699511712809:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:35:24.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:35:24.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:35:24.185+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:35:24.185+05:30 [Info] bucket_%2/idx_secondary/Backstore#3127033699511712809:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:35:24.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:35:24.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:35:24.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:35:24.197+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:35:24.197+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:35:24.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex started 2023-10-21T04:35:24.198+05:30 [Info] bucket_%2/idx_secondary/Backstore#3127033699511712809:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:35:24.198+05:30 [Info] bucket_%2/idx_secondary/Backstore#3127033699511712809:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:35:24.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:35:24.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:35:24.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:35:24.199+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:35:24.200+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:35:24.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex started 2023-10-21T04:35:24.200+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3127033699511712809 partitionId 0 WriterThreads 4 2023-10-21T04:35:24.200+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3127033699511712809 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index Index: 3127033699511712809 Partition: 0 2023-10-21T04:35:24.200+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3127033699511712809 2023-10-21T04:35:24.200+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:24.200+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:35:24.200+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:24.200+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:24.201+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 29.375057ms 2023-10-21T04:35:24.201+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 29.419001ms 2023-10-21T04:35:24.201+05:30 [Info] bucket_%2/idx_secondary/Mainstore#3127033699511712809:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:35:24.201+05:30 [Info] bucket_%2/idx_secondary/Mainstore#3127033699511712809:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:35:24.201+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 3127033699511712809, partnShardIdMap: map[0:[17406544375239214575 10170029854712240877]], indexDefn DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:35:24.208+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:24.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:24.214+05:30 [Info] switched currmeta from 160 -> 163 force true 2023-10-21T04:35:24.220+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:24.220+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:24.227+05:30 [Info] switched currmeta from 163 -> 165 force true 2023-10-21T04:35:24.242+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [3127033699511712809] &{0} 2023-10-21T04:35:24.242+05:30 [Info] Indexer::handleBuildIndex [3127033699511712809] 2023-10-21T04:35:24.243+05:30 [Info] bulkUpdateRState: Index instance 3127033699511712809 rstate moved to ACTIVE 2023-10-21T04:35:24.243+05:30 [Info] Indexer::handleBuildIndex Added Index: [3127033699511712809] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:35:24.243+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:24.244+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:24.244+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:24.244+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:35:24.244+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:24.244+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:24.244+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 28627 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:24.244+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:35:24.244+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:35:24.244+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:35:24.244+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 28627 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:24.244+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:35:24.244+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:24.244+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:24.244+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:35:24.244+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:35:24.244+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:24.244+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:24.244+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:35:24.244+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:4] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:35:24.245+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3127033699511712809 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:35:24.245+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:35:24.245+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [ InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 28628 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:35:24.247+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:35:24.247+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:35:24.248+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:35:24.248+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 6.187059ms 2023-10-21T04:35:24.248+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 6.2272ms 2023-10-21T04:35:24.249+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [3127033699511712809] 2023-10-21T04:35:24.255+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:35:24.261+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:35:24.264+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:24.264+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:24.281+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 14549775068266281786 2023-10-21T04:35:24.281+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 150.211233ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:24.302+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 20.593937ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:24.328+05:30 [Info] switched currmeta from 165 -> 169 force true 2023-10-21T04:35:24.429+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:36156" doAuth() ... 2023-10-21T04:35:24.429+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:36156" auth successful 2023-10-21T04:35:24.429+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:36156" +1 2023-10-21T04:35:24.481+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:24.481+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:24.501+05:30 [Info] switched currmeta from 169 -> 169 force true 2023-10-21T04:35:24.681+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:35:24.681+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:35:24.698+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35402 2023-10-21T04:35:24.698+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:35:24.725+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 22.917179ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:24.726+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:35:24.726+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:35:24.726+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:35:24.745+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:35:24.783+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:35:24.783+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:35:24.783+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:35:24.824+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:35:24.844+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35426 2023-10-21T04:35:24.849+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:35:24.849+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:35:24.849+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:35:24.854+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:35:24.886+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:35:24.886+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:35:24.886+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:35:24.886+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:35:24.886+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:35:24.886+05:30 [Info] DDLServiceMgr: processing create index token /indexing/ddl/commandToken/create/14549775068266281786/0 2023-10-21T04:35:24.901+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 53.216778ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:24.907+05:30 [Info] DDLServiceMgr: Update Replica Count. Index Defn 14549775068266281786 replica Count {true 1 0 0} 2023-10-21T04:35:24.907+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 30.187µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:24.954+05:30 [Info] DDLServiceMgr: Remove create index token 14549775068266281786. 2023-10-21T04:35:24.954+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:35:24.954+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:35:24.954+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:35:24.954+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:35:24.954+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:35:24.954+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:35402->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:24.954+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:60014->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:24.955+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:54906->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:24.955+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:55370->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:24.955+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35402. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:24.955+05:30 [Info] messageListener.start(): message channel closed. Remove peer a:2b:44:be:6c:be:b5:9 as follower. 2023-10-21T04:35:25.073+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35440 2023-10-21T04:35:25.098+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 34.391µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:25.121+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35426. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:25.121+05:30 [Info] messageListener.start(): message channel closed. Remove peer a8:ff:17:86:62:b3:21:d as follower. 2023-10-21T04:35:25.139+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:25.174+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:25.175+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:25.199+05:30 [Info] switched currmeta from 169 -> 169 force true 2023-10-21T04:35:25.227+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:25.230+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 149.610735ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:25.341+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35440. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:25.341+05:30 [Info] messageListener.start(): message channel closed. Remove peer 67:0:29:f:54:56:86:5d as follower. 2023-10-21T04:35:25.379+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35456 2023-10-21T04:35:25.402+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:25.450+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 65.758487ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:25.471+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 274143649547516 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 206082810619393 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.478+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 120349894144442 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.478+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 143687065777286 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.478+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 162049908746825 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.481+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 116493903186721 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.481+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 212967951483349 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.483+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 78446412189949 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.488+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 263088038522959 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.488+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 136568163328749 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.489+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 127746572208200 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.490+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 261951617336488 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.492+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 58733680850130 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.492+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 188740876924377 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.495+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 277338544220217 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.495+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 173457131919400 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.498+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 206088069380492 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.498+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 245247859472006 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.505+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35456. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:25.505+05:30 [Info] messageListener.start(): message channel closed. Remove peer 9f:18:1f:33:b9:97:ad:7f as follower. 2023-10-21T04:35:25.506+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 250426459207023 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.509+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 125696667341112 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.509+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 274158831830257 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.511+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 35716737784090 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.514+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 106231614376481 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.514+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 34510046292135 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.514+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 191500187207978 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.518+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 137256376714252 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.518+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5906787184229 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.518+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 118920785117980 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.520+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 146375746634523 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.523+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 41101832875819 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.523+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 112482982607415 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.524+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 241391494503904 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.526+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 121450709505223 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.528+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 216238759475962 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.528+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 168446351781198 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.531+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 127702263378100 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.533+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 100008922352935 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.534+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 169268048765249 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.535+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 149033089785397 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.536+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 245900669851618 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.538+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 106297365959267 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.541+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 88718752743571 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.541+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 39045953044628 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.543+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 160399698726237 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.544+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 217237947818243 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.544+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 110286285845649 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.548+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 162870112426804 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.550+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 132470209691538 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.552+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 264001907139144 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.552+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 63737850123332 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.555+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 32274016651656 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.555+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 266604622117081 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 225662991941187 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.558+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 84581802233088 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.559+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 241607113601772 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 272523968790649 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 77598347829810 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.565+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 197957220074784 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.570+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 97677619676504 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.570+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 136038310030103 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.573+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 72072509155168 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.574+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 105415986286631 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.582+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 161870047388121 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.583+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 27479155597282 0 28628 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:35:25.584+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 InstanceIds [3127033699511712809] 2023-10-21T04:35:25.585+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 28628 2023-10-21T04:35:25.585+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:35:25.585+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:35:25.585+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:35:25.585+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:35:25.585+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:35:25.585+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:35:25.585+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:25.731+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:25.797+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:35:25.804+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3127033699511712809 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:35:25.804+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3127033699511712809 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.309µs SnapOpenDur 233.871µs 2023-10-21T04:35:25.804+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 3127033699511712809 INDEX_STATE_CATCHUP 2023-10-21T04:35:25.806+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 3127033699511712809 true 2023-10-21T04:35:25.806+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 3127033699511712809 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 28628 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] 2023-10-21T04:35:25.804+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 SnapshotId 11 Creating recovery point ... 2023-10-21T04:35:25.818+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 28628 2023-10-21T04:35:25.832+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 28628 2023-10-21T04:35:25.832+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:25.833+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:25.834+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:25.834+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:35:25.835+05:30 [Info] send progress stats to clients 2023-10-21T04:35:25.835+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:35:25.838+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:25.838+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:25.841+05:30 [Info] switched currmeta from 169 -> 170 force true 2023-10-21T04:35:25.836+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:35:25.842+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:4] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:35:25.842+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 28628 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:35:25.845+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:35:25.851+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 2023-10-21T04:35:25.851+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28628 2023-10-21T04:35:25.851+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28628 2023-10-21T04:35:25.851+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:35:25.851+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:35:25.851+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 3127033699511712809 false 2023-10-21T04:35:25.851+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 3127033699511712809 2023-10-21T04:35:25.851+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:35:25.851+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 28628 2023-10-21T04:35:25.863+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.871562ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:25.879+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:25.880+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:25.887+05:30 [Info] switched currmeta from 170 -> 171 force true 2023-10-21T04:35:25.890+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 SnapshotId 11 Created recovery point (took 82.687662ms) 2023-10-21T04:35:25.950+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:26.194+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:26.194+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:26.196+05:30 [Info] switched currmeta from 171 -> 172 force true 2023-10-21T04:35:26.587+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:35:26.587+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:35:26.587+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 3127033699511712809 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 28628, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 0 29} { 1 93622f3f2a49 2 27 0 27} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 19 0 19} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 19 0 19} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 27 0 27} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 33 0 33} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 21 0 21} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:35:26.587+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 3127033699511712809 INDEX_STATE_ACTIVE 2023-10-21T04:35:26.587+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:35:26.587+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:35:26.587+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:35:26.587+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 5 CurrMaint 0 New 0 2023-10-21T04:35:26.587+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 28628 2023-10-21T04:35:26.588+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:26.588+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:35:26.588+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:35:26.588+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 28628 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:26.588+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3127033699511712809 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:35:26.588+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:35:26.588+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:35:26.588+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:35:26.588+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 28628 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:35:26.589+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:35:26.589+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:20 1:15 2:17 3:16 4:16 5:21 6:15 7:16 8:11 9:16 10:28 11:12 12:16 13:15 14:5 15:14 16:8 17:6 18:7 19:21 20:13 21:20 22:17 23:14 24:14 25:14 26:15 27:13 28:10 29:16 30:12 31:21 32:18 33:16 34:18 35:16 36:10 37:12 38:12 39:21 40:20 41:18 42:11 43:19 44:13 45:21 46:16 47:19 48:15 49:12 50:19 51:16 52:15 53:19 54:16 55:13 56:12 57:17 58:21 59:24 60:16 61:11 62:22 63:18] 2023-10-21T04:35:26.589+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:27 1:22 2:24 3:23 4:23 5:28 6:22 7:23 8:18 9:23 10:35 11:19 12:23 13:22 14:12 15:21 16:15 17:13 18:14 19:28 20:20 21:27 22:24 23:21 24:21 25:21 26:22 27:20 28:17 29:23 30:19 31:28 32:25 33:23 34:25 35:23 36:17 37:19 38:19 39:28 40:27 41:25 42:18 43:26 44:20 45:28 46:23 47:26 48:22 49:19 50:26 51:23 52:22 53:26 54:23 55:20 56:19 57:24 58:28 59:31 60:23 61:18 62:29 63:25] 2023-10-21T04:35:26.589+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:35:26.589+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:36156: use of closed network connection 2023-10-21T04:35:26.589+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:36156" exit: read tcp4 127.0.0.1:9121->127.0.0.1:36156: use of closed network connection 2023-10-21T04:35:26.589+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:36156" closed ! 2023-10-21T04:35:26.589+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:35:26.589+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:26.589+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:26.590+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:35:26.590+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:35:26.590+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:35:26.590+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:35:26.590+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 28628 2023-10-21T04:35:26.590+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:35:26.602+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:35:26.602+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:35:26.602+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:35:26.602+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28628 2023-10-21T04:35:26.602+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 28628 2023-10-21T04:35:26.602+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:35:26.628+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 38.28566ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:26.670+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:26.670+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:26.673+05:30 [Info] switched currmeta from 172 -> 173 force true 2023-10-21T04:35:26.883+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_1 Forcing Overdue Commit 2023-10-21T04:35:26.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 4 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:35:26.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 4 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.232µs SnapOpenDur 80.771µs 2023-10-21T04:35:26.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 5 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:35:26.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3464125101225762400 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:35:26.883+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3464125101225762400 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:6 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.699µs SnapOpenDur 77.179µs 2023-10-21T04:35:26.883+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15055366427687156590 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:35:26.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15055366427687156590 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.194µs SnapOpenDur 54.936µs 2023-10-21T04:35:26.883+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 SnapshotId 5 Creating recovery point ... 2023-10-21T04:35:26.883+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 5 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.679µs SnapOpenDur 31.038µs 2023-10-21T04:35:26.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 6 Creating New Snapshot SnapshotInfo: count:122 committed:true 2023-10-21T04:35:26.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 6 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:122 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.988µs SnapOpenDur 43.287µs 2023-10-21T04:35:26.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 7 Creating New Snapshot SnapshotInfo: count:136 committed:true 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15055366427687156590, PartitionId 0 SnapshotId 24 Creating recovery point ... 2023-10-21T04:35:26.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 7 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:136 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.866µs SnapOpenDur 40.556µs 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 8 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:35:26.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 8 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:106 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.166µs SnapOpenDur 37.788µs 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 1 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 1 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.325µs SnapOpenDur 70.991µs 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 2 Creating New Snapshot SnapshotInfo: count:136 committed:true 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 2 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:136 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.952µs SnapOpenDur 46.257µs 2023-10-21T04:35:26.884+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 15864358084358521096 PartitionId 3 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:35:26.884+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 15864358084358521096 PartitionId: 3 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.884µs SnapOpenDur 39.156µs 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.884+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:35:26.900+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 SnapshotId 5 Created recovery point (took 16.141759ms) 2023-10-21T04:35:26.901+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:35:26.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1 SnapshotId 2 Created recovery point (took 106.971073ms) 2023-10-21T04:35:26.992+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:35:26.993+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:35:27.033+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5 SnapshotId 2 Created recovery point (took 149.731415ms) 2023-10-21T04:35:27.034+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:35:27.035+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:35:27.043+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15055366427687156590, PartitionId 0 SnapshotId 24 Created recovery point (took 159.724651ms) 2023-10-21T04:35:27.049+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6 SnapshotId 2 Created recovery point (took 165.337736ms) 2023-10-21T04:35:27.050+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:35:27.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:35:27.061+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7 SnapshotId 2 Created recovery point (took 176.933974ms) 2023-10-21T04:35:27.062+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:35:27.062+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:35:27.076+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4 SnapshotId 2 Created recovery point (took 192.052586ms) 2023-10-21T04:35:27.077+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:35:27.078+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:35:27.101+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8 SnapshotId 2 Created recovery point (took 216.477439ms) 2023-10-21T04:35:27.102+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:35:27.102+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:35:27.117+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3 SnapshotId 2 Created recovery point (took 232.476308ms) 2023-10-21T04:35:27.119+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:35:27.119+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:35:27.130+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2 SnapshotId 2 Created recovery point (took 246.031737ms) 2023-10-21T04:35:27.131+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:35:27.132+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:35:27.321+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:35:27.322+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:35:27.322+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:35:27.322+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:35:27.322+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:35:27.322+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:35:27.573+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 2023-10-21T04:35:27.573+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 2023-10-21T04:35:27.573+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 2023-10-21T04:35:27.573+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0]]. New rollbacktime: 1697842989598078587, Old rollbacktime: 0 2023-10-21T04:35:28.050+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3127033699511712809 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:35:28.050+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3127033699511712809 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 3.099µs SnapOpenDur 93.85µs 2023-10-21T04:35:28.050+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:35:28.050+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17434304260397056806 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:6 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 5.091µs SnapOpenDur 59.663µs 2023-10-21T04:35:28.050+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 SnapshotId 12 Creating recovery point ... 2023-10-21T04:35:28.050+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 5 Creating recovery point ... 2023-10-21T04:35:28.050+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:35:28.050+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 3 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.368µs SnapOpenDur 82.024µs 2023-10-21T04:35:28.050+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Creating New Snapshot SnapshotInfo: count:118 committed:true 2023-10-21T04:35:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 4 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:118 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.732µs SnapOpenDur 39.899µs 2023-10-21T04:35:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:35:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 5 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:138 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.306µs SnapOpenDur 39.521µs 2023-10-21T04:35:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:35:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 6 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:127 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.41µs SnapOpenDur 40.377µs 2023-10-21T04:35:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:35:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 7 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:131 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.158µs SnapOpenDur 34.085µs 2023-10-21T04:35:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:35:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 8 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:112 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.299µs SnapOpenDur 28.367µs 2023-10-21T04:35:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:35:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 1 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:128 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.889µs SnapOpenDur 27.441µs 2023-10-21T04:35:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:35:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 2 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 1.883µs SnapOpenDur 37.41µs 2023-10-21T04:35:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.052+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.052+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:35:28.075+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 5 Created recovery point (took 24.429845ms) 2023-10-21T04:35:28.076+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:35:28.121+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 3 Created recovery point (took 70.102072ms) 2023-10-21T04:35:28.122+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:35:28.123+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:35:28.193+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 SnapshotId 12 Created recovery point (took 142.694395ms) 2023-10-21T04:35:28.207+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 3 Created recovery point (took 156.488497ms) 2023-10-21T04:35:28.208+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:35:28.209+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:35:28.226+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 3 Created recovery point (took 175.404751ms) 2023-10-21T04:35:28.227+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:35:28.228+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:35:28.237+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 3 Created recovery point (took 185.345206ms) 2023-10-21T04:35:28.238+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:35:28.238+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:35:28.250+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 3 Created recovery point (took 198.601257ms) 2023-10-21T04:35:28.251+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:35:28.251+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:35:28.259+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 3 Created recovery point (took 207.703294ms) 2023-10-21T04:35:28.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:35:28.261+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:35:28.274+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 3 Created recovery point (took 222.523509ms) 2023-10-21T04:35:28.277+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:35:28.277+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:35:28.281+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 3 Created recovery point (took 229.237959ms) 2023-10-21T04:35:28.282+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:35:28.283+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:35:29.894+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:29.898+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:29.899+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:29.901+05:30 [Info] switched currmeta from 173 -> 173 force true 2023-10-21T04:35:35.008+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. err: operation_canceled 2023-10-21T04:35:35.008+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:35:35.008+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[]} 2023-10-21T04:35:35.009+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:35:35.010+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:35:35.010+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:35:35.012+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:35:38.619+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47314" doAuth() ... 2023-10-21T04:35:38.620+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47314" doReceive() ... 2023-10-21T04:35:40.431+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:35:40.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:40.629+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:40.636+05:30 [Info] switched currmeta from 173 -> 173 force true 2023-10-21T04:35:40.905+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:35:40.946+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to true 2023-10-21T04:35:40.946+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 216] updating rebalancing status to true 2023-10-21T04:35:40.985+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:41.143+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:41.143+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:41.172+05:30 [Info] switched currmeta from 173 -> 173 force true 2023-10-21T04:35:41.247+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:41.249+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:41.265+05:30 [Info] switched currmeta from 173 -> 173 force true 2023-10-21T04:35:42.541+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {c90b9c83d7ffff2117bc8bbb59b77784 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{7f731cd927c72e28ff24e69497b93571 7060000 } {964f96068adef144590fc7ab85b2311c 7060000 }]} 2023-10-21T04:35:42.543+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 13.549µs 2023-10-21T04:35:42.543+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:35:42.543+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:35:42.543+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:35:42.545+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:35:42.545+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:35:42.545+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 1.700745ms 2023-10-21T04:35:42.545+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:35:42.545+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:35:42.545+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:35:42.545+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 9.465µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:42.545+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/c90b9c83d7ffff2117bc8bbb59b77784 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:c90b9c83d7ffff2117bc8bbb59b77784]}]} 2023-10-21T04:35:42.556+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:35:42.556+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:35:43.641+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {8806f5706747af4e635b8da5fcd8aeee c90b9c83d7ffff2117bc8bbb59b77784 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:35:43.643+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {8806f5706747af4e635b8da5fcd8aeee c90b9c83d7ffff2117bc8bbb59b77784 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:35:43.644+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"c90b9c83d7ffff2117bc8bbb59b77784","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:35:43.646+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{8806f5706747af4e635b8da5fcd8aeee c90b9c83d7ffff2117bc8bbb59b77784 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:35:43.646+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{8806f5706747af4e635b8da5fcd8aeee c90b9c83d7ffff2117bc8bbb59b77784 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:35:43.646+05:30 [Info] ShardRebalancer::observeRebalance &{8806f5706747af4e635b8da5fcd8aeee c90b9c83d7ffff2117bc8bbb59b77784 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:35:43.653+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"8806f5706747af4e635b8da5fcd8aeee","RebalId":"c90b9c83d7ffff2117bc8bbb59b77784","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:35:43.653+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:35:43.780+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47390 2023-10-21T04:35:43.785+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47390 2023-10-21T04:35:43.812+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:47390 2023-10-21T04:35:43.891+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken 2023-10-21T04:35:43.891+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token Deleted. Mark Done. 2023-10-21T04:35:43.891+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:35:43.891+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:35:43.891+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:35:43.891+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:35:43.899+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:35:43.899+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:35:43.899+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.528µs 2023-10-21T04:35:43.900+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:35:43.900+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:35:43.900+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:35:43.902+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:35:43.902+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:35:43.902+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:35:43.909+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:35:43.909+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 55.402µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:43.911+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:35:43.911+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:35:43.911+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:35:43.911+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:35:43.923+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:35:43.923+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/c90b9c83d7ffff2117bc8bbb59b77784 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:c90b9c83d7ffff2117bc8bbb59b77784]}]} 2023-10-21T04:35:43.924+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:35:43.924+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:35:43.926+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/c90b9c83d7ffff2117bc8bbb59b77784, rev: [] 2023-10-21T04:35:43.926+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/c90b9c83d7ffff2117bc8bbb59b77784, rev: [] 2023-10-21T04:35:43.926+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:35:43.926+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:35:43.927+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:35:43.927+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:35:43.927+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 59.564µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:43.929+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:35:43.929+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:35:43.929+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/c90b9c83d7ffff2117bc8bbb59b77784, rev: [] call. err: 2023-10-21T04:35:43.930+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:35:43.930+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:35:43.940+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:35:43.940+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:35:43.942+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:35:43.942+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:35:43.944+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:35:43.944+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:35:44.059+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:35:44.191+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47484" doAuth() ... 2023-10-21T04:35:44.192+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47484" doReceive() ... 2023-10-21T04:35:44.209+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47488" doAuth() ... 2023-10-21T04:35:44.209+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47488" doReceive() ... 2023-10-21T04:35:44.214+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:44.214+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:44.214+05:30 [Info] UnwatchMetadata(): indexer 964f96068adef144590fc7ab85b2311c 2023-10-21T04:35:44.214+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:35:44.214+05:30 [Info] UnwatchMetadata(): finish for indexer 964f96068adef144590fc7ab85b2311c 2023-10-21T04:35:44.214+05:30 [Info] UnwatchMetadata(): indexer 7f731cd927c72e28ff24e69497b93571 2023-10-21T04:35:44.214+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:35:44.214+05:30 [Info] UnwatchMetadata(): finish for indexer 7f731cd927c72e28ff24e69497b93571 2023-10-21T04:35:44.215+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:51784->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:44.215+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:51322->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:35:44.222+05:30 [Info] switched currmeta from 173 -> 175 force true 2023-10-21T04:35:44.222+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:35:44.222+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] ... stopped 2023-10-21T04:35:44.222+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:35:44.222+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] ... stopped 2023-10-21T04:35:44.223+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47490" doAuth() ... 2023-10-21T04:35:44.223+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47490" doReceive() ... 2023-10-21T04:35:44.224+05:30 [Error] transport error between 127.0.0.1:42752->127.0.0.1:9125: write tcp 127.0.0.1:42752->127.0.0.1:9125: write: broken pipe 2023-10-21T04:35:44.224+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:35:44.227+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] closing unhealthy connection "127.0.0.1:42752" 2023-10-21T04:35:44.237+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35916 2023-10-21T04:35:44.268+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:44.293+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47506" doAuth() ... 2023-10-21T04:35:44.296+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:47506" doReceive() ... 2023-10-21T04:35:44.332+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 79.610359ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:44.363+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Stopping releaseConnsRoutine 2023-10-21T04:35:44.392+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Stopping releaseConnsRoutine 2023-10-21T04:35:44.475+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:35:44.475+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:44.475+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:44.497+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:44.653+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9002. 2023-10-21T04:35:44.653+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 5 != len(c.nodesvs): 4 2023-10-21T04:35:44.653+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:35:44.682+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:60268. Error = read tcp4 127.0.0.1:9118->127.0.0.1:60268: read: connection reset by peer. Kill Pipe. 2023-10-21T04:35:44.682+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2:dc:4a:ad:34:cb:1a:28 as follower. 2023-10-21T04:35:44.690+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:47490 closed 2023-10-21T04:35:44.833+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:35:45.090+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to false 2023-10-21T04:35:45.174+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:35:45.174+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:35:45.174+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:35:45.186+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35916. Reason = EOF. Cleaning Pipe. 2023-10-21T04:35:45.187+05:30 [Info] messageListener.start(): message channel closed. Remove peer b1:3e:e:6c:7a:37:9a:7f as follower. 2023-10-21T04:35:45.245+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:45.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:45.248+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:45.291+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:45.291+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:45.304+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:45.378+05:30 [Warn] ClusterInfoCache Initialization: Unable to identify server group for node 127.0.0.1:9001. 2023-10-21T04:35:45.378+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:45.536+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.536+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(1) 2023-10-21T04:35:45.539+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.540+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(1) 2023-10-21T04:35:45.553+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.553+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(2) 2023-10-21T04:35:45.554+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.554+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(2) 2023-10-21T04:35:45.586+05:30 [Error] PeerPipe.doRecieve() : ecounter error when received mesasage from Peer 127.0.0.1:60266. Error = read tcp4 127.0.0.1:9118->127.0.0.1:60266: read: connection reset by peer. Kill Pipe. 2023-10-21T04:35:45.586+05:30 [Info] messageListener.start(): message channel closed. Remove peer ce:4c:6e:40:74:a1:36:f4 as follower. 2023-10-21T04:35:45.601+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.601+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(3) 2023-10-21T04:35:45.607+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.609+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(3) 2023-10-21T04:35:45.614+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.615+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(4) 2023-10-21T04:35:45.618+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:47506 closed 2023-10-21T04:35:45.628+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.628+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(4) 2023-10-21T04:35:45.640+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.641+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(5) 2023-10-21T04:35:45.645+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.646+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(5) 2023-10-21T04:35:45.689+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.689+05:30 [Info] Validation Failed for cluster info.. Retrying(1) 2023-10-21T04:35:45.701+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.702+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(6) 2023-10-21T04:35:45.702+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.724+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(6) 2023-10-21T04:35:45.825+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.825+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(7) 2023-10-21T04:35:45.850+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.850+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(8) 2023-10-21T04:35:45.851+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.851+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(7) 2023-10-21T04:35:45.870+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.871+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(9) 2023-10-21T04:35:45.883+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:45.883+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(8) 2023-10-21T04:35:46.088+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:46.088+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(10) 2023-10-21T04:35:46.092+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:46.092+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(9) 2023-10-21T04:35:46.105+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:46.105+05:30 [Error] Validation Failed while updating nodes and nodesvs.. &{{{1 0} 0 0 {{} -1073741824} {{} 0}} http://%40index:e5edc8b277786f52ec7dd5a38f845b83@127.0.0.1:9003 default updateIndexerList 300 2000000000 1 false map[] {0xc007d08fc0 0xc006d68060 {map[ale:0.0.0 asn1:5.0.21 chronicle:0.0.1 crypto:5.1.4 esaml:4.4.0 inets:8.3.1 kernel:8.5.4 lhttpc:1.3.0 ns_server:7.6.0-1559-rel-EE-enterprise os_mon:2.8.2 public_key:1.13.3 sasl:4.2 ssl:10.9.1 stdlib:4.3.1.1 xmerl:1.3.31.1] 7.6.0-1559-rel-EE-enterprise true 73d3202e218e8e8b3a6d8b83ef2f6375 [{default /poolsStreaming/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375 /pools/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375}] false} updateIndexerList} {map[] [{458758 active 172.31.5.112:9000 map[cmd_get:0 couch_docs_actual_disk_size:4.821376e+06 couch_docs_data_size:1.452915e+06 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:2012 curr_items_tot:2012 ep_bg_fetched:0 get_hits:0 mem_used:8.542544e+06 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1430449152e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12000 distTCP:21403 distTLS:21453 httpsMgmt:19000] healthy 2083 7.6.0-1559-rel-EE-enterprise false [kv n1ql] e3997f8ca787b6ec4f8b06cf33f6aa3e inet 20013814 Group 1} {458758 active 127.0.0.1:9003 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.100656e+06 index_disk_size:1.2759916e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1424432128e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12006 distTCP:21403 distTLS:21453 httpsMgmt:19003] healthy 2077 7.6.0-1559-rel-EE-enterprise true [index] 18194302e3f10df59bd076e1c4142d0b inet 75644330 Group 2} {458758 active 127.0.0.1:9004 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.046887e+06 index_disk_size:1.0147496e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1480416256e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12008 distTCP:21403 distTLS:21453 httpsMgmt:19004] healthy 2074 7.6.0-1559-rel-EE-enterprise false [index] 8806f5706747af4e635b8da5fcd8aeee inet 54230642 Group 1}] map[] map[terseBucketsBase:/pools/default/b/ terseStreamingBucketsBase:/pools/default/bs/ uri:/pools/default/buckets?v=128083561&uuid=73d3202e218e8e8b3a6d8b83ef2f6375] /pools/default/nodeServices?v=8773313 /pools/default/serverGroups?v=76905227 [{bucket_%2 b9538b29d7e025c819e734b646db039c} {bucket_1 1fc3689e709c62b9ee184f2ec13e6f43}] {0xc007d08fc0 0xc006d68060 {map[ale:0.0.0 asn1:5.0.21 chronicle:0.0.1 crypto:5.1.4 esaml:4.4.0 inets:8.3.1 kernel:8.5.4 lhttpc:1.3.0 ns_server:7.6.0-1559-rel-EE-enterprise os_mon:2.8.2 public_key:1.13.3 sasl:4.2 ssl:10.9.1 stdlib:4.3.1.1 xmerl:1.3.31.1] 7.6.0-1559-rel-EE-enterprise true 73d3202e218e8e8b3a6d8b83ef2f6375 [{default /poolsStreaming/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375 /pools/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375}] false} updateIndexerList}} [{458758 active 172.31.5.112:9000 map[cmd_get:0 couch_docs_actual_disk_size:4.821376e+06 couch_docs_data_size:1.452915e+06 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:2012 curr_items_tot:2012 ep_bg_fetched:0 get_hits:0 mem_used:8.542544e+06 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1430449152e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12000 distTCP:21403 distTLS:21453 httpsMgmt:19000] healthy 2083 7.6.0-1559-rel-EE-enterprise false [kv n1ql] e3997f8ca787b6ec4f8b06cf33f6aa3e inet 20013814 Group 1} {458758 active 127.0.0.1:9003 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.100656e+06 index_disk_size:1.2759916e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1424432128e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12006 distTCP:21403 distTLS:21453 httpsMgmt:19003] healthy 2077 7.6.0-1559-rel-EE-enterprise true [index] 18194302e3f10df59bd076e1c4142d0b inet 75644330 Group 2} {458758 active 127.0.0.1:9004 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.046887e+06 index_disk_size:1.0147496e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1480416256e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12008 distTCP:21403 distTLS:21453 httpsMgmt:19004] healthy 2074 7.6.0-1559-rel-EE-enterprise false [index] 8806f5706747af4e635b8da5fcd8aeee inet 54230642 Group 1}] [{map[kv:12000 kvSSL:11998 mgmt:9000 mgmtSSL:19000 n1ql:9499 n1qlSSL:19499 projector:10000] 172.31.5.112 false} {map[mgmt:9001 mgmtSSL:19001] 127.0.0.1 false} {map[indexAdmin:9118 indexHttp:9120 indexHttps:19120 indexScan:9119 indexStreamCatchup:9122 indexStreamInit:9121 indexStreamMaint:9123 mgmt:9003 mgmtSSL:19003] 127.0.0.1 true} {map[indexAdmin:9124 indexHttp:9126 indexHttps:19126 indexScan:9125 indexStreamCatchup:9128 indexStreamInit:9127 indexStreamMaint:9129 mgmt:9004 mgmtSSL:19004] 127.0.0.1 false}] map[] [] [] 7 6 map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999]} 2023-10-21T04:35:46.109+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:46.109+05:30 [Info] Validation Failed while updating nodes and nodesvs.. Retrying(10) 2023-10-21T04:35:46.148+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:46.150+05:30 [Info] Validation Failed for cluster info.. Retrying(2) 2023-10-21T04:35:46.155+05:30 [Warn] ClusterInfoCache:validateCache - Failed as len(c.nodes): 3 != len(c.nodesvs): 4 2023-10-21T04:35:46.155+05:30 [Error] Validation Failed while updating nodes and nodesvs.. &{{{1 0} 0 0 {{} -1073741824} {{} 0}} http://%40index:e5edc8b277786f52ec7dd5a38f845b83@127.0.0.1:9003 default MonitorKVNodes 300 2000000000 1 false map[] {0xc007a45ef0 0xc00803c760 {map[ale:0.0.0 asn1:5.0.21 chronicle:0.0.1 crypto:5.1.4 esaml:4.4.0 inets:8.3.1 kernel:8.5.4 lhttpc:1.3.0 ns_server:7.6.0-1559-rel-EE-enterprise os_mon:2.8.2 public_key:1.13.3 sasl:4.2 ssl:10.9.1 stdlib:4.3.1.1 xmerl:1.3.31.1] 7.6.0-1559-rel-EE-enterprise true 73d3202e218e8e8b3a6d8b83ef2f6375 [{default /poolsStreaming/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375 /pools/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375}] false} MonitorKVNodes} {map[] [{458758 active 172.31.5.112:9000 map[cmd_get:0 couch_docs_actual_disk_size:4.821376e+06 couch_docs_data_size:1.452915e+06 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:2012 curr_items_tot:2012 ep_bg_fetched:0 get_hits:0 mem_used:8.542544e+06 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1430449152e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12000 distTCP:21403 distTLS:21453 httpsMgmt:19000] healthy 2083 7.6.0-1559-rel-EE-enterprise false [kv n1ql] e3997f8ca787b6ec4f8b06cf33f6aa3e inet 20013814 Group 1} {458758 active 127.0.0.1:9003 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.100656e+06 index_disk_size:1.2759916e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1424432128e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12006 distTCP:21403 distTLS:21453 httpsMgmt:19003] healthy 2077 7.6.0-1559-rel-EE-enterprise true [index] 18194302e3f10df59bd076e1c4142d0b inet 75644330 Group 2} {458758 active 127.0.0.1:9004 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.046887e+06 index_disk_size:1.0147496e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1480416256e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12008 distTCP:21403 distTLS:21453 httpsMgmt:19004] healthy 2074 7.6.0-1559-rel-EE-enterprise false [index] 8806f5706747af4e635b8da5fcd8aeee inet 54230642 Group 1}] map[] map[terseBucketsBase:/pools/default/b/ terseStreamingBucketsBase:/pools/default/bs/ uri:/pools/default/buckets?v=128083561&uuid=73d3202e218e8e8b3a6d8b83ef2f6375] /pools/default/nodeServices?v=8773313 /pools/default/serverGroups?v=76905227 [{bucket_%2 b9538b29d7e025c819e734b646db039c} {bucket_1 1fc3689e709c62b9ee184f2ec13e6f43}] {0xc007a45ef0 0xc00803c760 {map[ale:0.0.0 asn1:5.0.21 chronicle:0.0.1 crypto:5.1.4 esaml:4.4.0 inets:8.3.1 kernel:8.5.4 lhttpc:1.3.0 ns_server:7.6.0-1559-rel-EE-enterprise os_mon:2.8.2 public_key:1.13.3 sasl:4.2 ssl:10.9.1 stdlib:4.3.1.1 xmerl:1.3.31.1] 7.6.0-1559-rel-EE-enterprise true 73d3202e218e8e8b3a6d8b83ef2f6375 [{default /poolsStreaming/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375 /pools/default?uuid=73d3202e218e8e8b3a6d8b83ef2f6375}] false} MonitorKVNodes}} [{458758 active 172.31.5.112:9000 map[cmd_get:0 couch_docs_actual_disk_size:4.821376e+06 couch_docs_data_size:1.452915e+06 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:2012 curr_items_tot:2012 ep_bg_fetched:0 get_hits:0 mem_used:8.542544e+06 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1430449152e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12000 distTCP:21403 distTLS:21453 httpsMgmt:19000] healthy 2083 7.6.0-1559-rel-EE-enterprise false [kv n1ql] e3997f8ca787b6ec4f8b06cf33f6aa3e inet 20013814 Group 1} {458758 active 127.0.0.1:9003 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.100656e+06 index_disk_size:1.2759916e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1424432128e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12006 distTCP:21403 distTLS:21453 httpsMgmt:19003] healthy 2077 7.6.0-1559-rel-EE-enterprise true [index] 18194302e3f10df59bd076e1c4142d0b inet 75644330 Group 2} {458758 active 127.0.0.1:9004 map[cmd_get:0 couch_docs_actual_disk_size:0 couch_docs_data_size:0 couch_spatial_data_size:0 couch_spatial_disk_size:0 couch_views_actual_disk_size:0 couch_views_data_size:0 curr_items:0 curr_items_tot:0 ep_bg_fetched:0 get_hits:0 index_data_size:2.046887e+06 index_disk_size:1.0147496e+07 mem_used:0 ops:0 vb_active_num_non_resident:0 vb_replica_curr_items:0] 11800 11800 1.1480416256e+10 1.5466930176e+10 x86_64-pc-linux-gnu map[direct:12008 distTCP:21403 distTLS:21453 httpsMgmt:19004] healthy 2074 7.6.0-1559-rel-EE-enterprise false [index] 8806f5706747af4e635b8da5fcd8aeee inet 54230642 Group 1}] [{map[kv:12000 kvSSL:11998 mgmt:9000 mgmtSSL:19000 n1ql:9499 n1qlSSL:19499 projector:10000] 172.31.5.112 false} {map[mgmt:9001 mgmtSSL:19001] 127.0.0.1 false} {map[indexAdmin:9118 indexHttp:9120 indexHttps:19120 indexScan:9119 indexStreamCatchup:9122 indexStreamInit:9121 indexStreamMaint:9123 mgmt:9003 mgmtSSL:19003] 127.0.0.1 true} {map[indexAdmin:9124 indexHttp:9126 indexHttps:19126 indexScan:9125 indexStreamCatchup:9128 indexStreamInit:9127 indexStreamMaint:9129 mgmt:9004 mgmtSSL:19004] 127.0.0.1 false}] map[0:Group 1 1:Group 2 2:Group 1 3:Group 2 4:Group 1] [] [] 7 6 map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999]} 2023-10-21T04:35:46.159+05:30 [Info] MapSettings: config before mapping: {"indexer.rebalance.serverless.perNodeTransferBatchSize":1} 2023-10-21T04:35:46.159+05:30 [Info] MapSettings: config after mapping: {"indexer.rebalance.serverless.perNodeTransferBatchSize":1} 2023-10-21T04:35:46.175+05:30 [Info] settingsManager::metakvCallback triggered for path /indexing/settings/config (rev [131 104 2 97 0 108 0 0 0 5 104 2 109 0 0 0 32 49 56 49 57 52 51 48 50 101 51 102 49 48 100 102 53 57 98 100 48 55 54 101 49 99 52 49 52 50 100 48 98 104 2 97 1 110 5 0 202 131 167 222 14 104 2 109 0 0 0 32 50 56 49 98 52 49 53 48 55 98 49 102 57 51 55 97 49 98 54 99 51 55 98 99 52 48 50 55 101 101 102 54 104 2 97 12 110 5 0 118 125 167 222 14 104 2 109 0 0 0 32 53 48 50 48 54 57 51 101 57 97 101 53 99 53 50 49 54 56 100 52 49 52 98 48 50 102 57 50 50 97 100 56 104 2 97 4 110 5 0 121 124 167 222 14 104 2 109 0 0 0 32 55 102 55 51 49 99 100 57 50 55 99 55 50 101 50 56 102 102 50 52 101 54 57 52 57 55 98 57 51 53 55 49 104 2 97 1 110 5 0 233 129 167 222 14 104 2 109 0 0 0 32 101 51 57 57 55 102 56 99 97 55 56 55 98 54 101 99 52 102 56 98 48 54 99 102 51 51 102 54 97 97 51 101 104 2 97 2 110 5 0 31 124 167 222 14 106]) 2023-10-21T04:35:46.175+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:35:46.176+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:35:46.176+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:35:46.177+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:35:46.177+05:30 [Info] Setting log level to Info 2023-10-21T04:35:46.177+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:35:46.178+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:35:46.186+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:35:46.186+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:35:46.186+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:35:46.186+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:35:46.187+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:35:46.187+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:35:46.187+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:35:46.187+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:35:46.187+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:35:46.191+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 2.947894ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:35:46.487+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9971458406282827, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:35:46.789+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:35:47.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9194115117482636, throttleDelayMs (new, old, change): (2, 5, -3) 2023-10-21T04:35:48.105+05:30 [Info] Error occurred during nodes and nodesvs update (ClusterInfo Validation Failed) .. Retrying(1) 2023-10-21T04:35:48.119+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:35:48.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:48.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:48.123+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:48.134+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:48.134+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:48.137+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:48.156+05:30 [Info] Error occurred during nodes and nodesvs update (ClusterInfo Validation Failed) .. Retrying(1) 2023-10-21T04:35:48.160+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:35:48.160+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:48.184+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:48.184+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:48.187+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:48.193+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:48.193+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:48.195+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:48.200+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:48.200+05:30 [Warn] validateNodesAndSvs - Failed as len(nodes): 3 != len(nodesExt): 4 2023-10-21T04:35:48.200+05:30 [Warn] HostNames Nodes: [172.31.5.112:9000 127.0.0.1:9003 127.0.0.1:9004] NodesExt: [172.31.5.112:9000 127.0.0.1:9001 127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:35:48.200+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:35:48.236+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:48.237+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:48.239+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:48.245+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:35:48.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:35:48.246+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:35:48.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8182580297168635, throttleDelayMs (new, old, change): (0, 2, -2) 2023-10-21T04:36:00.981+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:01.053+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 1 2023-10-21T04:36:01.054+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:01.054+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:01.059+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:01.091+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:01.112+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:01.112+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:01.116+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:02.393+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:02.489+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9763755937540743, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:36:02.500+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:02.500+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:02.517+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:02.600+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:02.623+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:02.623+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:02.636+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:03.034+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 238] aggregate storage info: map[bucket_%2:map[kv:2377920 index:13676804] bucket_1:map[kv:2443456 index:13119672]] 2023-10-21T04:36:03.487+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9217156621949744, throttleDelayMs (new, old, change): (0, 3, -3) 2023-10-21T04:36:09.627+05:30 [Info] memstats {"Alloc":205709256, "TotalAlloc":1034449360, "Sys":283515192, "Lookups":0, "Mallocs":11665534,"Frees":10803378, "HeapAlloc":205709256, "HeapSys":258834432, "HeapIdle":45703168, "HeapInuse":213131264,"HeapReleased":3145728, "HeapObjects":862156,"MSpanInuse":1484800, "MSpanSys": 2235840, "StackInuse": 5406720,"GCSys":14447328, "LastGC":1697843099601524435,"PauseTotalNs":20357939, "PauseNs":[665995, 215117, 91559, 171500], "NumGC":13} 2023-10-21T04:36:10.216+05:30 [Info] connected with 2 indexers 2023-10-21T04:36:10.216+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:36:10.306+05:30 [Info] num concurrent scans {0} 2023-10-21T04:36:10.306+05:30 [Info] average scan response {0 ms} 2023-10-21T04:36:10.501+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:36:10.501+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:36:10.587+05:30 [Info] janitor: running cleanup. 2023-10-21T04:36:10.587+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[15945117906390072636 8378417662949994532 12824340686309573428 16162012140118277065 17797267771846799851 7392796230534034390 7147407900589480727 13268712595133993155 4532813151040661705 5465835793930010953 1252359852432998131 15431181238515168303 4535197604682045916 18026660730857541767 6985049432225870291 14889401051926000176 14292583894719865678 8669649359784409226 4138047918192025493 14741949230702299196 12338036531800130039 8857269605176042330 573521267164877540 12195715103909237684 12140458060295645238 17656144409930763141 2048006213723315482 13856704970496601958 11365633301282701856 12599498416578472779 249887442666643345 3674078063353481907 7525085113233084253 12833670934079048728 15968368593314043432 18046371353728341033 4790214984019693894 8504686614261377811 4860549750398470299 12027054384338714804 11619165464560280276 18323460706027642575]} 2023-10-21T04:36:12.627+05:30 [Info] Indexer::ReadMemstats Time Taken 1.007686ms 2023-10-21T04:36:12.688+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:12.689+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:12.749+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:12.749+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:12.753+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:12.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:12.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:12.768+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:12.775+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to true 2023-10-21T04:36:12.820+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:12.865+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:12.865+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:12.878+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:13.115+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:13.143+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:36:13.146+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:13.146+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:13.149+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:13.164+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:13.267+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:13.268+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:13.274+05:30 [Info] switched currmeta from 175 -> 175 force true 2023-10-21T04:36:13.944+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. err: operation_canceled 2023-10-21T04:36:13.944+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:36:13.944+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:36:13.945+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:36:13.945+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:13.951+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:36:13.952+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:36:14.781+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 239] /regulator/report/s/index/7f731cd927c72e28ff24e69497b93571 has 0-length value 2023-10-21T04:36:14.791+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 239] /regulator/report/s/index/964f96068adef144590fc7ab85b2311c has 0-length value 2023-10-21T04:36:15.009+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {2a6a1478070bd9349853af27a661ad2b [] topology-change-rebalance [{{268b83ffbd4df37978153ecb76d33816 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] [{18194302e3f10df59bd076e1c4142d0b 7060000 }]} 2023-10-21T04:36:15.012+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 10.13µs 2023-10-21T04:36:15.013+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:36:15.013+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:36:15.013+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:36:15.014+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:36:15.014+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:15.015+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.0096ms 2023-10-21T04:36:15.015+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:36:15.015+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:36:15.015+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[268b83ffbd4df37978153ecb76d33816 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:15.015+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 7.376µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:15.015+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/2a6a1478070bd9349853af27a661ad2b Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:2a6a1478070bd9349853af27a661ad2b]}]} 2023-10-21T04:36:15.023+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 6] 2023-10-21T04:36:15.023+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 6] 2023-10-21T04:36:15.057+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:15.106+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:15.106+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:15.106+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:36:15.114+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:36:15.146+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:15.282+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:36:15.285+05:30 [Info] switched currmeta from 175 -> 176 force true 2023-10-21T04:36:15.289+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:15.289+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:15.296+05:30 [Info] switched currmeta from 176 -> 176 force true 2023-10-21T04:36:15.312+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:36:15.312+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:36:15.312+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:36:15.313+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:36:15.358+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:37386 2023-10-21T04:36:15.376+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 9.65693ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:15.377+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 43.498µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:15.427+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:48976" doAuth() ... 2023-10-21T04:36:15.428+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:48976" doReceive() ... 2023-10-21T04:36:15.475+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {268b83ffbd4df37978153ecb76d33816 2a6a1478070bd9349853af27a661ad2b Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:36:15.477+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Waiting for Global Rebalance Token In Metakv 2023-10-21T04:36:15.877+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:36:16.481+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {268b83ffbd4df37978153ecb76d33816 2a6a1478070bd9349853af27a661ad2b Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:36:16.481+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"268b83ffbd4df37978153ecb76d33816","RebalId":"2a6a1478070bd9349853af27a661ad2b","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:36:16.483+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{268b83ffbd4df37978153ecb76d33816 2a6a1478070bd9349853af27a661ad2b Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:36:16.483+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{268b83ffbd4df37978153ecb76d33816 2a6a1478070bd9349853af27a661ad2b Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:36:16.483+05:30 [Info] ShardRebalancer::observeRebalance &{268b83ffbd4df37978153ecb76d33816 2a6a1478070bd9349853af27a661ad2b Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:36:16.488+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"268b83ffbd4df37978153ecb76d33816","RebalId":"2a6a1478070bd9349853af27a661ad2b","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:36:16.488+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:36:16.659+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:50436 2023-10-21T04:36:16.667+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:50436 2023-10-21T04:36:16.697+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:50436 2023-10-21T04:36:16.784+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene9:6c:98:3e:fb:d7:77:38 MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12127044935527081292 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3127033699511712809 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14327413444160233932 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:16.784+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:36:16.784+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:36:16.784+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:36:16.784+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 55.235µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:16.795+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:36:16.807+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:16.808+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:36:16.808+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:36:16.808+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:36:16.808+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 30.125µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:16.817+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene9:6c:98:3e:fb:d7:77:38 MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12127044935527081292 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3127033699511712809 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14327413444160233932 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:16.820+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:16.894+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene9:6c:98:3e:fb:d7:77:38 MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12127044935527081292 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3127033699511712809 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14327413444160233932 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:16.928+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:16.938+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"268b83ffbd4df37978153ecb76d33816","RebalId":"2a6a1478070bd9349853af27a661ad2b","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:36:16.938+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:36:16.938+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:36:16.938+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:36:16.938+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:36:16.938+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:36:16.938+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 17.624µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:16.962+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:16.962+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[2925063509317026816:true 16787445029868804812:true] 2023-10-21T04:36:16.963+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.53µs 2023-10-21T04:36:16.963+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [16787445029868804812 2925063509317026816], isLockedForRecovery: false 2023-10-21T04:36:16.963+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [16787445029868804812 2925063509317026816], errMap: map[2925063509317026816: 16787445029868804812:], elapsed: 11.846µs 2023-10-21T04:36:16.963+05:30 [Info] StorageMgr::handleShardTransfer Updated book-keeping for shardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:36:16.963+05:30 [Info] ShardTransferManager::processShardTransferMessage Initiating command: ShardIds: [16787445029868804812 2925063509317026816] RebalanceId: 2a6a1478070bd9349853af27a661ad2b TransferTokenId: ShardTokenf5:22:2e:2d:1a:c4:9b:9b Task Type: Rebalance IsPeerTransfer: false NewAlternateShardIds: [] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:36:16.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : initiating transfer for shardId 2925063509317026816, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816 2023-10-21T04:36:16.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : initiating transfer for shardId 16787445029868804812, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812 2023-10-21T04:36:16.968+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSS Snapshot SBHead :0 SBTail :1466368 2023-10-21T04:36:16.970+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSS Snapshot SBHead :0 SBTail :1699840 2023-10-21T04:36:16.972+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:36:16.972+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:36:17.001+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery/log.00000000000000.data size:1699840 dur:29.111111ms 2023-10-21T04:36:17.015+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :1699840 current tail :1699840 2023-10-21T04:36:17.016+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{268b83ffbd4df37978153ecb76d33816 2a6a1478070bd9349853af27a661ad2b Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:36:17.016+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery/log.00000000000000.data size:1466368 dur:44.702565ms 2023-10-21T04:36:17.016+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :1466368 current tail :1466368 2023-10-21T04:36:17.033+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSS Snapshot SBHead :0 SBTail :1798144 2023-10-21T04:36:17.042+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSS Snapshot SBHead :0 SBTail :1601536 2023-10-21T04:36:17.043+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/log.00000000000000.data holdOff:0 2023-10-21T04:36:17.058+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/log.00000000000000.data size:1798144 dur:15.298941ms 2023-10-21T04:36:17.060+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data: log copy completed, snapshot head :0 current head :0 snapshot tail :1798144 current tail :1798144 2023-10-21T04:36:17.060+05:30 [Info] plasma shard transfer total bandwidth stats (upload), avgBW: 36260258 totalBytes: 3517245 numShards: 2 duration: 97.270802ms 2023-10-21T04:36:17.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : completed shardId 16787445029868804812 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812 took 96.309544ms size :3517245 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data size :1798144 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/log.00000000000000.data size :1798144 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812/data/log.00000000000000.data written :1798144 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery size :1699840 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery/log.00000000000000.data size :1699840 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812/data/recovery/log.00000000000000.data written :1699840 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/meta/shard.json size :2807 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812/meta/shard.json written :2807 retry :0 status :TransferComplete 2023-10-21T04:36:17.060+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 16787445029868804812 bytesWritten: 3517245, totalBytes: 3517245, transferRate: 3.7023631578947365e+07 2023-10-21T04:36:17.061+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 16787445029868804812, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812, err: , elapsed(sec): 0.098672895 2023-10-21T04:36:17.062+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/log.00000000000000.data holdOff:0 2023-10-21T04:36:17.090+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/log.00000000000000.data size:1601536 dur:28.523878ms 2023-10-21T04:36:17.093+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data: log copy completed, snapshot head :0 current head :0 snapshot tail :1601536 current tail :1601536 2023-10-21T04:36:17.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : completed shardId 2925063509317026816 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816 took 130.74121ms size :3087006 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data size :1601536 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/log.00000000000000.data size :1601536 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816/data/log.00000000000000.data written :1601536 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery size :1466368 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery/log.00000000000000.data size :1466368 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816/data/recovery/log.00000000000000.data written :1466368 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/meta/shard.json size :2648 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816/meta/shard.json written :2648 retry :0 status :TransferComplete 2023-10-21T04:36:17.095+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 2925063509317026816, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816, err: , elapsed(sec): 0.132544852 2023-10-21T04:36:17.096+05:30 [Info] ShardTransferManager::processShardTransferMessage All shards processing done. Sending response errMap: map[2925063509317026816: 16787445029868804812:], shardPaths: map[2925063509317026816:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/2925063509317026816/shards/shard2925063509317026816 16787445029868804812:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokenf5_22_2e_2d_1a_c4_9b_9b/16787445029868804812/shards/shard16787445029868804812], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.133008796 2023-10-21T04:36:17.096+05:30 [Info] StorageMgr::ShardTransferResponse Clearing book-keeping for shardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:36:17.096+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 2925063509317026816 bytesWritten: 3087006, totalBytes: 3087006, transferRate: 2.37462e+07 2023-10-21T04:36:17.097+05:30 [Info] ShardRebalancer::startShardTransfer Received response for transfer of shards: [16787445029868804812 2925063509317026816], ttid: ShardTokenf5:22:2e:2d:1a:c4:9b:9b, elapsed(sec): 0.134363165 2023-10-21T04:36:17.100+05:30 [Info] TestCode::TestActionAtTag: currTag: 2, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:17.106+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:17.107+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:36:17.107+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:36:17.107+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:36:17.107+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 30.013µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:17.144+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene9:6c:98:3e:fb:d7:77:38 MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12127044935527081292 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3127033699511712809 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14327413444160233932 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:17.144+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[10170029854712240877:true 17406544375239214575:true] 2023-10-21T04:36:17.144+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.629µs 2023-10-21T04:36:17.145+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [17406544375239214575 10170029854712240877], isLockedForRecovery: false 2023-10-21T04:36:17.145+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [17406544375239214575 10170029854712240877], errMap: map[10170029854712240877: 17406544375239214575:], elapsed: 10.711µs 2023-10-21T04:36:17.145+05:30 [Info] StorageMgr::handleShardTransfer Updated book-keeping for shardIds: [17406544375239214575 10170029854712240877] 2023-10-21T04:36:17.145+05:30 [Info] ShardTransferManager::processShardTransferMessage Initiating command: ShardIds: [17406544375239214575 10170029854712240877] RebalanceId: 2a6a1478070bd9349853af27a661ad2b TransferTokenId: ShardTokene9:6c:98:3e:fb:d7:77:38 Task Type: Rebalance IsPeerTransfer: false NewAlternateShardIds: [] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:36:17.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : initiating transfer for shardId 10170029854712240877, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877 2023-10-21T04:36:17.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : initiating transfer for shardId 17406544375239214575, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575 2023-10-21T04:36:17.149+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSS Snapshot SBHead :0 SBTail :1855488 2023-10-21T04:36:17.149+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSS Snapshot SBHead :0 SBTail :1843200 2023-10-21T04:36:17.151+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:36:17.153+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery/log.00000000000000.data holdOff:0 2023-10-21T04:36:17.188+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery/log.00000000000000.data size:1855488 dur:36.647266ms 2023-10-21T04:36:17.191+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :1855488 current tail :1855488 2023-10-21T04:36:17.189+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery/log.00000000000000.data size:1843200 dur:35.850968ms 2023-10-21T04:36:17.192+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery: log copy completed, snapshot head :0 current head :0 snapshot tail :1843200 current tail :1843200 2023-10-21T04:36:17.195+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:17.197+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSS Snapshot SBHead :0 SBTail :1990656 2023-10-21T04:36:17.199+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSS Snapshot SBHead :0 SBTail :1970176 2023-10-21T04:36:17.202+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/log.00000000000000.data holdOff:0 2023-10-21T04:36:17.219+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data: log segment copy init, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/log.00000000000000.data holdOff:0 2023-10-21T04:36:17.225+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/log.00000000000000.data size:1990656 dur:22.729821ms 2023-10-21T04:36:17.225+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data: log copy completed, snapshot head :0 current head :0 snapshot tail :1990656 current tail :1990656 2023-10-21T04:36:17.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : completed shardId 10170029854712240877 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877 took 80.297692ms size :3865248 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data size :1990656 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/log.00000000000000.data size :1990656 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877/data/log.00000000000000.data written :1990656 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery size :1855488 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery/log.00000000000000.data size :1855488 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877/data/recovery/log.00000000000000.data written :1855488 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/meta/shard.json size :2650 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877/meta/shard.json written :2650 retry :0 status :TransferComplete 2023-10-21T04:36:17.225+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 10170029854712240877 bytesWritten: 3865248, totalBytes: 3865248, transferRate: 4.83156e+07 2023-10-21T04:36:17.226+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 10170029854712240877, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877, err: , elapsed(sec): 0.081098313 2023-10-21T04:36:17.257+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data: log segment copy done, src:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/log.00000000000000.data size:1970176 dur:37.738875ms 2023-10-21T04:36:17.258+05:30 [Info] /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data: log copy completed, snapshot head :0 current head :0 snapshot tail :1970176 current tail :1970176 2023-10-21T04:36:17.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : completed shardId 17406544375239214575 transfer, upload path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575 took 111.418316ms size :3832642 dumping copy stats: src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/config.json size :70 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575/config.json written :70 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data size :1970176 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575/data/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/log.00000000000000.data size :1970176 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575/data/log.00000000000000.data written :1970176 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery size :1843200 snapshot :true dst : written :0 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery/header.data size :8192 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575/data/recovery/header.data written :8192 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery/log.00000000000000.data size :1843200 snapshot :false dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575/data/recovery/log.00000000000000.data written :1843200 retry :0 status :TransferComplete src :/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/meta/shard.json size :2812 snapshot :true dst :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575/meta/shard.json written :2812 retry :0 status :TransferComplete 2023-10-21T04:36:17.258+05:30 [Info] ShardTransferManager::processShardTransferMessage doneCb invoked for shardId: 17406544375239214575, path: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575, err: , elapsed(sec): 0.113596087 2023-10-21T04:36:17.258+05:30 [Info] ShardRebalancer::startShardTranfser ShardId: 17406544375239214575 bytesWritten: 3832642, totalBytes: 3832642, transferRate: 3.452830630630631e+07 2023-10-21T04:36:17.258+05:30 [Info] ShardTransferManager::processShardTransferMessage All shards processing done. Sending response errMap: map[10170029854712240877: 17406544375239214575:], shardPaths: map[10170029854712240877:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/10170029854712240877/shards/shard10170029854712240877 17406544375239214575:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38/17406544375239214575/shards/shard17406544375239214575], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.113710129 2023-10-21T04:36:17.258+05:30 [Info] StorageMgr::ShardTransferResponse Clearing book-keeping for shardIds: [17406544375239214575 10170029854712240877] 2023-10-21T04:36:17.259+05:30 [Info] ShardRebalancer::startShardTransfer Received response for transfer of shards: [17406544375239214575 10170029854712240877], ttid: ShardTokene9:6c:98:3e:fb:d7:77:38, elapsed(sec): 0.114165445 2023-10-21T04:36:17.273+05:30 [Info] TestCode::TestActionAtTag: currTag: 2, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:17.285+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene9:6c:98:3e:fb:d7:77:38 MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12127044935527081292 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3127033699511712809 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14327413444160233932 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:17.285+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:36:17.285+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:36:17.285+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:36:17.285+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 22.121µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:17.302+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:17.302+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:17.305+05:30 [Info] switched currmeta from 176 -> 177 force true 2023-10-21T04:36:17.387+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene9:6c:98:3e:fb:d7:77:38 MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12127044935527081292 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3127033699511712809 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14327413444160233932 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:17.491+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:17.491+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:17.498+05:30 [Info] switched currmeta from 177 -> 178 force true 2023-10-21T04:36:18.320+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:18.320+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:18.331+05:30 [Info] switched currmeta from 178 -> 180 force true 2023-10-21T04:36:18.346+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:18.346+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:18.350+05:30 [Info] switched currmeta from 180 -> 180 force true 2023-10-21T04:36:18.807+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:18.807+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:18.811+05:30 [Info] switched currmeta from 180 -> 181 force true 2023-10-21T04:36:19.363+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:19.363+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:19.367+05:30 [Info] switched currmeta from 181 -> 182 force true 2023-10-21T04:36:19.389+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:19.389+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:19.391+05:30 [Info] switched currmeta from 182 -> 183 force true 2023-10-21T04:36:19.482+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:19.482+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:19.487+05:30 [Info] switched currmeta from 183 -> 184 force true 2023-10-21T04:36:20.063+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:20.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:20.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:20.074+05:30 [Info] switched currmeta from 184 -> 184 force true 2023-10-21T04:36:20.109+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697843177420910984, Old rollbacktime: 0 2023-10-21T04:36:20.453+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:20.453+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:20.457+05:30 [Info] switched currmeta from 184 -> 185 force true 2023-10-21T04:36:20.646+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:20.646+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:20.649+05:30 [Info] switched currmeta from 185 -> 186 force true 2023-10-21T04:36:21.485+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:21.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:21.489+05:30 [Info] switched currmeta from 186 -> 187 force true 2023-10-21T04:36:21.913+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:21.913+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:21.918+05:30 [Info] switched currmeta from 187 -> 188 force true 2023-10-21T04:36:22.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:22.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:22.503+05:30 [Info] switched currmeta from 188 -> 190 force true 2023-10-21T04:36:22.510+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:22.510+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:22.514+05:30 [Info] switched currmeta from 190 -> 190 force true 2023-10-21T04:36:22.521+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:22.521+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:22.524+05:30 [Info] switched currmeta from 190 -> 191 force true 2023-10-21T04:36:22.566+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:22.566+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:22.573+05:30 [Info] switched currmeta from 191 -> 193 force true 2023-10-21T04:36:22.582+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:22.584+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:22.587+05:30 [Info] switched currmeta from 193 -> 193 force true 2023-10-21T04:36:23.074+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:23.074+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:23.082+05:30 [Info] switched currmeta from 193 -> 194 force true 2023-10-21T04:36:23.091+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:23.091+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:23.097+05:30 [Info] switched currmeta from 194 -> 195 force true 2023-10-21T04:36:24.103+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:24.103+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:24.113+05:30 [Info] switched currmeta from 195 -> 197 force true 2023-10-21T04:36:24.123+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:24.123+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:24.131+05:30 [Info] switched currmeta from 197 -> 197 force true 2023-10-21T04:36:24.600+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:24.600+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:24.609+05:30 [Info] switched currmeta from 197 -> 199 force true 2023-10-21T04:36:24.630+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:24.630+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:24.634+05:30 [Info] switched currmeta from 199 -> 199 force true 2023-10-21T04:36:24.672+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:24.709+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = EOF. Cleaning Pipe. 2023-10-21T04:36:24.709+05:30 [Error] WatcherServer.runOnce() : Watcher terminated unexpectedly. 2023-10-21T04:36:24.709+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:37386. Reason = EOF. Cleaning Pipe. 2023-10-21T04:36:24.710+05:30 [Info] messageListener.start(): message channel closed. Remove peer 13:92:d1:e:7a:f3:d1:b9 as follower. 2023-10-21T04:36:24.710+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:48976 closed 2023-10-21T04:36:24.737+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:36:24.737+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/2a6a1478070bd9349853af27a661ad2b Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:2a6a1478070bd9349853af27a661ad2b]}]} 2023-10-21T04:36:24.738+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:36:24.739+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[268b83ffbd4df37978153ecb76d33816 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:24.740+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/2a6a1478070bd9349853af27a661ad2b, rev: [] 2023-10-21T04:36:24.740+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/2a6a1478070bd9349853af27a661ad2b, rev: [] 2023-10-21T04:36:24.740+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:36:24.740+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:36:24.740+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:36:24.740+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:36:24.740+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:36:24.750+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:36:24.750+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenf5:22:2e:2d:1a:c4:9b:9b MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [16787445029868804812 2925063509317026816] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15055366427687156590 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14729093590251115073 RealInstId: 15864358084358521096 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 11648680422243286912 RealInstId: 10032435514095102923 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3464125101225762400 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:24.750+05:30 [Info] RebalanceServiceManager::cleanupShardTokenForSource Cleaning up token: ShardTokenf5:22:2e:2d:1a:c4:9b:9b on source as ShardTokenDropOnSource is posted for this token 2023-10-21T04:36:24.750+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenf5:22:2e:2d:1a:c4:9b:9b, dropIndexes: true 2023-10-21T04:36:24.763+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15055366427687156590 &{1} 2023-10-21T04:36:24.763+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15055366427687156590 2023-10-21T04:36:24.763+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 12582561312368048736, instId: 15055366427687156590 2023-10-21T04:36:24.763+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:36:24.763+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:24.764+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15055366427687156590 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:36:24.764+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15055366427687156590] 2023-10-21T04:36:24.764+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15055366427687156590] 2023-10-21T04:36:24.764+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:24.764+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15055366427687156590] 2023-10-21T04:36:24.764+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:24.765+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15055366427687156590 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:24.765+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.973782ms 2023-10-21T04:36:24.765+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15055366427687156590, PartitionId 0, IndexDefnId 12582561312368048736. 2023-10-21T04:36:24.765+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.765+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.765+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex] 2023-10-21T04:36:24.765+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex 2023-10-21T04:36:24.766+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex] 2023-10-21T04:36:24.766+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex] 2023-10-21T04:36:24.766+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex 2023-10-21T04:36:24.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex stopped 2023-10-21T04:36:24.766+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.766+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.766+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.766+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.768+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [15055366427687156590] 2023-10-21T04:36:24.768+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15055366427687156590 2023-10-21T04:36:24.774+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:36:24.774+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 61685 2023-10-21T04:36:24.774+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 9.098022ms 2023-10-21T04:36:24.774+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 11.142779ms 2023-10-21T04:36:24.788+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:24.788+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:24.798+05:30 [Info] switched currmeta from 199 -> 200 force true 2023-10-21T04:36:24.801+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex] 2023-10-21T04:36:24.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex closed 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Mainstore#15055366427687156590:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex] 2023-10-21T04:36:24.802+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex 2023-10-21T04:36:24.802+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex] 2023-10-21T04:36:24.802+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex] 2023-10-21T04:36:24.802+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex 2023-10-21T04:36:24.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex stopped 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.802+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.810+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 58.978601ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:24.812+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16961575949607441659 instance id 14729093590251115073 real instance id 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:36:24.812+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16961575949607441659 instance 15864358084358521096 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:36:24.812+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16961575949607441659 instance id 15864358084358521096 2023-10-21T04:36:24.812+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16961575949607441659 2023-10-21T04:36:24.822+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex] 2023-10-21T04:36:24.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex closed 2023-10-21T04:36:24.823+05:30 [Info] bucket_1/idx_secondary/Backstore#15055366427687156590:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.823+05:30 [Info] Indexer::cleanupIndexData IndexInst 15055366427687156590 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex ... 2023-10-21T04:36:24.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex ... 2023-10-21T04:36:24.826+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15864358084358521096 &{1} 2023-10-21T04:36:24.826+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15864358084358521096 2023-10-21T04:36:24.826+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16961575949607441659, instId: 15864358084358521096 2023-10-21T04:36:24.826+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:36:24.826+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:24.826+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15864358084358521096 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:36:24.826+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15864358084358521096] 2023-10-21T04:36:24.826+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15864358084358521096] 2023-10-21T04:36:24.826+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:24.827+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15864358084358521096] 2023-10-21T04:36:24.827+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:36:24.827+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:36:24.827+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:36:24.827+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:36:24.827+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.827+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.828+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.828+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:36:24.828+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.828+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.828+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.828+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:36:24.828+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:24.829+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.829+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.829+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.829+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:36:24.829+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.829+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.829+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.829+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:36:24.828+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15864358084358521096 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:24.831+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 5.859322ms 2023-10-21T04:36:24.832+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15864358084358521096 2023-10-21T04:36:24.832+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [15864358084358521096] 2023-10-21T04:36:24.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex ... 2023-10-21T04:36:24.836+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex ... 2023-10-21T04:36:24.838+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:36:24.838+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:36:24.838+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:36:24.838+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex 2023-10-21T04:36:24.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex stopped 2023-10-21T04:36:24.838+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex stopped 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex stopped 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex stopped 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex stopped 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.839+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.839+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.839+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:36:24.840+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex 2023-10-21T04:36:24.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex stopped 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.840+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:36:24.840+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:36:24.840+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:36:24.840+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex 2023-10-21T04:36:24.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex stopped 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.840+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:36:24.840+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:36:24.840+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:36:24.840+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex 2023-10-21T04:36:24.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex stopped 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.840+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.844+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex] 2023-10-21T04:36:24.845+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:36:24.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex closed 2023-10-21T04:36:24.845+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 61685 2023-10-21T04:36:24.845+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 13.164955ms 2023-10-21T04:36:24.845+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 19.090707ms 2023-10-21T04:36:24.845+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.845+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.845+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.845+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:36:24.846+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex] 2023-10-21T04:36:24.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex closed 2023-10-21T04:36:24.846+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.846+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.846+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.846+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:36:24.847+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex] 2023-10-21T04:36:24.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex closed 2023-10-21T04:36:24.848+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.848+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.848+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.848+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:36:24.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_15055366427687156590_0.index destroyed 2023-10-21T04:36:24.848+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15055366427687156590, PartitionId 0, IndexDefnId 12582561312368048736. 2023-10-21T04:36:24.848+05:30 [Info] Indexer::cleanupIndexData IndexInst 15055366427687156590 Partition 0 Destroy Done 2023-10-21T04:36:24.848+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:36:24.848+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:36:24.849+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:36:24.849+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex 2023-10-21T04:36:24.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex stopped 2023-10-21T04:36:24.849+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.849+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.849+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.849+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.850+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:36:24.850+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:36:24.850+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:36:24.850+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex 2023-10-21T04:36:24.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex stopped 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.850+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:36:24.850+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:36:24.850+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:36:24.850+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex 2023-10-21T04:36:24.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex stopped 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.850+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex] 2023-10-21T04:36:24.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex closed 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:36:24.853+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:36:24.853+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:36:24.853+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:36:24.853+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex 2023-10-21T04:36:24.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex stopped 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.853+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.862+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex] 2023-10-21T04:36:24.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex closed 2023-10-21T04:36:24.868+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.868+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex ... 2023-10-21T04:36:24.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex ... 2023-10-21T04:36:24.875+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex] 2023-10-21T04:36:24.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex closed 2023-10-21T04:36:24.877+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.878+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.882+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex] 2023-10-21T04:36:24.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex closed 2023-10-21T04:36:24.882+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.882+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.884+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex] 2023-10-21T04:36:24.885+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex closed 2023-10-21T04:36:24.885+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.885+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex ... 2023-10-21T04:36:24.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex ... 2023-10-21T04:36:24.894+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex ... 2023-10-21T04:36:24.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex ... 2023-10-21T04:36:24.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_4.index destroyed 2023-10-21T04:36:24.897+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.898+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.898+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 4 Destroy Done 2023-10-21T04:36:24.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex] 2023-10-21T04:36:24.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex closed 2023-10-21T04:36:24.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex] 2023-10-21T04:36:24.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex closed 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:36:24.901+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:36:24.901+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:36:24.901+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:36:24.901+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex 2023-10-21T04:36:24.901+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex stopped 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.901+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex ... 2023-10-21T04:36:24.902+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex ... 2023-10-21T04:36:24.903+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.903+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.903+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.903+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:36:24.904+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:36:24.904+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:36:24.904+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:36:24.904+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex 2023-10-21T04:36:24.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex stopped 2023-10-21T04:36:24.904+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.904+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.904+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.904+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.904+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex] 2023-10-21T04:36:24.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex closed 2023-10-21T04:36:24.904+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.904+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex ... 2023-10-21T04:36:24.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex ... 2023-10-21T04:36:24.905+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_5.index destroyed 2023-10-21T04:36:24.908+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.909+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 5 Destroy Done 2023-10-21T04:36:24.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex] 2023-10-21T04:36:24.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex closed 2023-10-21T04:36:24.909+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.909+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex ... 2023-10-21T04:36:24.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex ... 2023-10-21T04:36:24.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex ... 2023-10-21T04:36:24.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex ... 2023-10-21T04:36:24.917+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex ... 2023-10-21T04:36:24.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex ... 2023-10-21T04:36:24.920+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 107.547984ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:24.921+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.931+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_8.index destroyed 2023-10-21T04:36:24.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex ... 2023-10-21T04:36:24.932+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex ... 2023-10-21T04:36:24.922+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16337157693634799600 instance id 11648680422243286912 real instance id 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:36:24.933+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 16337157693634799600 instance 10032435514095102923 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:36:24.933+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 16337157693634799600 instance id 10032435514095102923 2023-10-21T04:36:24.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.932+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.934+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 8 Destroy Done 2023-10-21T04:36:24.934+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 16337157693634799600 2023-10-21T04:36:24.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex] 2023-10-21T04:36:24.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex closed 2023-10-21T04:36:24.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex] 2023-10-21T04:36:24.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex closed 2023-10-21T04:36:24.938+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.938+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.938+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.938+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:36:24.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex ... 2023-10-21T04:36:24.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex ... 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:36:24.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_1.index destroyed 2023-10-21T04:36:24.940+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.940+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 1 Destroy Done 2023-10-21T04:36:24.940+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:36:24.940+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:36:24.940+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:36:24.940+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex 2023-10-21T04:36:24.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex stopped 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex ... 2023-10-21T04:36:24.940+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex ... 2023-10-21T04:36:24.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.942+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:24.942+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:24.944+05:30 [Info] switched currmeta from 200 -> 203 force true 2023-10-21T04:36:24.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index/docIndex sucessfully destroyed 2023-10-21T04:36:24.946+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:36:24.946+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_3.index destroyed 2023-10-21T04:36:24.946+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.946+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 3 Destroy Done 2023-10-21T04:36:24.947+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:36:24.947+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex 2023-10-21T04:36:24.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex stopped 2023-10-21T04:36:24.947+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.947+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.947+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.947+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex ... 2023-10-21T04:36:24.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex ... 2023-10-21T04:36:24.950+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.957+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10032435514095102923 &{1} 2023-10-21T04:36:24.957+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10032435514095102923 2023-10-21T04:36:24.957+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 16337157693634799600, instId: 10032435514095102923 2023-10-21T04:36:24.957+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:36:24.957+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10032435514095102923] 2023-10-21T04:36:24.957+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:24.957+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10032435514095102923] 2023-10-21T04:36:24.958+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.958+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.958+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.958+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:36:24.958+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:36:24.958+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:36:24.958+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:36:24.958+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex 2023-10-21T04:36:24.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex stopped 2023-10-21T04:36:24.958+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.958+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.958+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.958+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.959+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.959+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:24.959+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.959+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.959+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.959+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:36:24.959+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex stopped 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.960+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex stopped 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex stopped 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.960+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:36:24.960+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.960+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:36:24.961+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex 2023-10-21T04:36:24.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex stopped 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.960+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 10032435514095102923 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:36:24.961+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.011587ms 2023-10-21T04:36:24.961+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 4.053231ms 2023-10-21T04:36:24.961+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:36:24.961+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:36:24.961+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:36:24.961+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:36:24.961+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex 2023-10-21T04:36:24.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex stopped 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.961+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.962+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:36:24.962+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:36:24.962+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:36:24.963+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex 2023-10-21T04:36:24.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex stopped 2023-10-21T04:36:24.963+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.963+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.963+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.963+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.963+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:36:24.963+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:24.963+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:24.969+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:24.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:36:24.969+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:36:24.969+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:36:24.969+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:36:24.969+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex 2023-10-21T04:36:24.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex stopped 2023-10-21T04:36:24.969+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:24.969+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:24.969+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:24.969+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:24.963+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10032435514095102923 2023-10-21T04:36:24.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_6.index destroyed 2023-10-21T04:36:24.972+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.976+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 6 Destroy Done 2023-10-21T04:36:24.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex] 2023-10-21T04:36:24.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex closed 2023-10-21T04:36:24.977+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.978+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex ... 2023-10-21T04:36:24.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex ... 2023-10-21T04:36:24.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex] 2023-10-21T04:36:24.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex closed 2023-10-21T04:36:24.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/mainIndex sucessfully destroyed 2023-10-21T04:36:24.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex ... 2023-10-21T04:36:24.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex ... 2023-10-21T04:36:24.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:24.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_2.index destroyed 2023-10-21T04:36:24.999+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:36:24.999+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 2 Destroy Done 2023-10-21T04:36:24.999+05:30 [Info] bucket_1/idx_partitioned/Backstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:24.999+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:24.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex ... 2023-10-21T04:36:24.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex ... 2023-10-21T04:36:25.004+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex ... 2023-10-21T04:36:25.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex ... 2023-10-21T04:36:25.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex] 2023-10-21T04:36:25.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex closed 2023-10-21T04:36:25.013+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex] 2023-10-21T04:36:25.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex closed 2023-10-21T04:36:25.013+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.013+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.013+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.013+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:36:25.013+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.016+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.016+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.016+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:36:25.025+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex] 2023-10-21T04:36:25.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex closed 2023-10-21T04:36:25.025+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.025+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.025+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.025+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:36:25.026+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex] 2023-10-21T04:36:25.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex closed 2023-10-21T04:36:25.027+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.027+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.027+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.027+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:36:25.033+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex] 2023-10-21T04:36:25.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex closed 2023-10-21T04:36:25.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.044+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 122.411479ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:25.050+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.058+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.066+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.066+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.066+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.066+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:36:25.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_15864358084358521096_7.index destroyed 2023-10-21T04:36:25.066+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15864358084358521096, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:36:25.067+05:30 [Info] Indexer::cleanupIndexData IndexInst 15864358084358521096 Partition 7 Destroy Done 2023-10-21T04:36:25.067+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:36:25.067+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:36:25.067+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:36:25.067+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex 2023-10-21T04:36:25.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex stopped 2023-10-21T04:36:25.067+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.067+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.067+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.067+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex] 2023-10-21T04:36:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex closed 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:36:25.069+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:36:25.069+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:36:25.069+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:36:25.069+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex 2023-10-21T04:36:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex stopped 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex] 2023-10-21T04:36:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex closed 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.069+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:36:25.069+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:36:25.070+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:36:25.070+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:36:25.070+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex 2023-10-21T04:36:25.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex stopped 2023-10-21T04:36:25.070+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:36:25.070+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:36:25.070+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:36:25.070+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex 2023-10-21T04:36:25.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex stopped 2023-10-21T04:36:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.071+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.070+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:36:25.073+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:36:25.073+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:36:25.073+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex 2023-10-21T04:36:25.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex stopped 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.070+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex] 2023-10-21T04:36:25.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex closed 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.073+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.074+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.074+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:36:25.074+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:36:25.074+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:36:25.075+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:36:25.075+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex 2023-10-21T04:36:25.075+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex stopped 2023-10-21T04:36:25.075+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.075+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.075+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.075+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.076+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:36:25.076+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:36:25.076+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:36:25.076+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex 2023-10-21T04:36:25.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex stopped 2023-10-21T04:36:25.076+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.077+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.077+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.077+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.078+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:36:25.078+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:36:25.078+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:36:25.078+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex 2023-10-21T04:36:25.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex stopped 2023-10-21T04:36:25.078+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.078+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.079+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.079+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.089+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3464125101225762400 &{1} 2023-10-21T04:36:25.101+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3464125101225762400 2023-10-21T04:36:25.101+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 11276338709617092948, instId: 3464125101225762400 2023-10-21T04:36:25.102+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:36:25.102+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:25.102+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3464125101225762400 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:36:25.104+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3464125101225762400] 2023-10-21T04:36:25.115+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3464125101225762400] 2023-10-21T04:36:25.116+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:25.119+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3464125101225762400] 2023-10-21T04:36:25.120+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:25.122+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:25.120+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:36:25.123+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:25.123+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:25.125+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:25.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:36:25.125+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:36:25.125+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:36:25.124+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:25.128+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0046acf00 }] 2023-10-21T04:36:25.130+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:25.131+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:36:25.132+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:36:25.133+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:36:25.134+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:25.138+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:25.138+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 36.501278ms 2023-10-21T04:36:25.138+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:25.138+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:25.138+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3464125101225762400 2023-10-21T04:36:25.139+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1] 2023-10-21T04:36:25.150+05:30 [Info] switched currmeta from 203 -> 206 force true 2023-10-21T04:36:25.150+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 19 150690956328758 26 2023-10-21T04:36:25.151+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 0 1949031200982 23 2023-10-21T04:36:25.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.152+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 1 245782287380001 27 2023-10-21T04:36:25.152+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.153+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 2 52522903947059 28 2023-10-21T04:36:25.153+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.155+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 4 93079113162334 18 2023-10-21T04:36:25.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.155+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 5 244791485011286 23 2023-10-21T04:36:25.155+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.156+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 6 167580527121081 23 2023-10-21T04:36:25.158+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.161+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 7 177815184127237 17 2023-10-21T04:36:25.161+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.161+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 8 54236768606397 22 2023-10-21T04:36:25.161+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.161+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 9 134409829206048 26 2023-10-21T04:36:25.161+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.161+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 3 205201940883916 24 2023-10-21T04:36:25.162+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.162+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 16 123957315902602 30 2023-10-21T04:36:25.163+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.163+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 17 213108110408291 23 2023-10-21T04:36:25.163+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.163+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1] 2023-10-21T04:36:25.163+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 61685 2023-10-21T04:36:25.163+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 25.606952ms 2023-10-21T04:36:25.163+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 62.249045ms 2023-10-21T04:36:25.166+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 18 163460438181870 34 2023-10-21T04:36:25.166+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.166+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 10 160381752747972 18 2023-10-21T04:36:25.166+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.167+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 11 58522600746999 24 2023-10-21T04:36:25.167+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.167+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 14 161230548567164 25 2023-10-21T04:36:25.167+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.168+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 15 85302001459467 21 2023-10-21T04:36:25.168+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.168+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 12 142387448675642 30 2023-10-21T04:36:25.171+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.171+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 13 90115353725987 21 2023-10-21T04:36:25.171+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.179+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex] 2023-10-21T04:36:25.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex closed 2023-10-21T04:36:25.180+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:36:25.182+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.184+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.185+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex] 2023-10-21T04:36:25.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex closed 2023-10-21T04:36:25.185+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.186+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.187+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex] 2023-10-21T04:36:25.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex closed 2023-10-21T04:36:25.196+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.197+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.199+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:25.200+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:25.202+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex] 2023-10-21T04:36:25.203+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex closed 2023-10-21T04:36:25.203+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.203+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.205+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 51 213334533059126 25 2023-10-21T04:36:25.220+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.220+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 20 151273673850546 25 2023-10-21T04:36:25.220+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.221+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 52 142324128597367 22 2023-10-21T04:36:25.221+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.221+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 25 114823206239184 25 2023-10-21T04:36:25.221+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.221+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 57 80145536651832 17 2023-10-21T04:36:25.221+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.221+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 26 68847799765163 25 2023-10-21T04:36:25.221+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.214+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.218+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.219+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:36:25.224+05:30 [Info] switched currmeta from 206 -> 206 force true 2023-10-21T04:36:25.228+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex] 2023-10-21T04:36:25.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex closed 2023-10-21T04:36:25.238+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.238+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 58 25511316571199 16 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 27 232028101514735 19 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 59 106620692996001 23 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 28 25510109770064 28 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 60 279733370822238 23 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 21 147946970349859 24 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 53 254944616021293 24 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 22 258671969827141 19 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 54 55145461136105 16 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 23 53829314402679 32 2023-10-21T04:36:25.238+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.238+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 55 219528837936411 25 2023-10-21T04:36:25.244+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.244+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 24 19989312661772 15 2023-10-21T04:36:25.244+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.244+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 56 194568213794435 29 2023-10-21T04:36:25.244+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 29 8305763409275 21 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 61 223881084081370 17 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 30 124811039198764 18 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 62 36300629322478 34 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 33 107407811917992 20 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 36 228857796432725 25 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 31 207489375637709 12 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 63 25657368013413 17 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 32 33310272161722 25 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 34 100746255343609 19 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 35 117298435979365 24 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 37 203503670520010 24 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 38 47156288300713 24 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 39 158357249947806 21 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 43 196269910725961 26 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 44 246959094981553 34 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 47 172365063201867 21 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 48 79077905592013 17 2023-10-21T04:36:25.245+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.245+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 49 212884822297038 25 2023-10-21T04:36:25.246+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.246+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 40 104136033125373 29 2023-10-21T04:36:25.246+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.246+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 41 236904976023311 22 2023-10-21T04:36:25.246+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.246+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 42 124310544527292 21 2023-10-21T04:36:25.246+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.246+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 45 276463747117997 25 2023-10-21T04:36:25.246+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.246+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 46 65885584526743 20 2023-10-21T04:36:25.246+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.246+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 50 246582249071123 20 2023-10-21T04:36:25.246+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:36:25.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex] 2023-10-21T04:36:25.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex closed 2023-10-21T04:36:25.246+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.246+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex] 2023-10-21T04:36:25.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex closed 2023-10-21T04:36:25.250+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex 2023-10-21T04:36:25.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex stopped 2023-10-21T04:36:25.251+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:25.251+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:25.251+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:25.251+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:25.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:36:25.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex ... 2023-10-21T04:36:25.257+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 189.080238ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:25.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:36:25.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex ... 2023-10-21T04:36:25.261+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.259+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.261+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Close Done 2023-10-21T04:36:25.259+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [16787445029868804812 2925063509317026816] 2023-10-21T04:36:25.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... 2023-10-21T04:36:25.262+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:36:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex ... 2023-10-21T04:36:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_7.index destroyed 2023-10-21T04:36:25.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:36:25.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex ... 2023-10-21T04:36:25.267+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.267+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 7 Destroy Done 2023-10-21T04:36:25.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:36:25.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex ... 2023-10-21T04:36:25.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_2.index destroyed 2023-10-21T04:36:25.271+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:25.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:36:25.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex ... 2023-10-21T04:36:25.271+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.273+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:36:25.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex ... 2023-10-21T04:36:25.283+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 2 Destroy Done 2023-10-21T04:36:25.284+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to false 2023-10-21T04:36:25.286+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:25.286+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:25.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_6.index destroyed 2023-10-21T04:36:25.287+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.287+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 6 Destroy Done 2023-10-21T04:36:25.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex] 2023-10-21T04:36:25.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex closed 2023-10-21T04:36:25.288+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#10032435514095102923:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.288+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:36:25.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex ... 2023-10-21T04:36:25.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:36:25.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex ... 2023-10-21T04:36:25.292+05:30 [Info] switched currmeta from 206 -> 207 force true 2023-10-21T04:36:25.293+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:25.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.295+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_3.index destroyed 2023-10-21T04:36:25.296+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.296+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 3 Destroy Done 2023-10-21T04:36:25.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:36:25.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex ... 2023-10-21T04:36:25.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex] 2023-10-21T04:36:25.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex closed 2023-10-21T04:36:25.298+05:30 [Info] bucket_1/idx_partitioned/Mainstore#15864358084358521096:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:25.298+05:30 [Info] Indexer::cleanupIndexData IndexInst 3464125101225762400 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:25.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:36:25.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex ... 2023-10-21T04:36:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_8.index destroyed 2023-10-21T04:36:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.310+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:36:25.310+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 8 Destroy Done 2023-10-21T04:36:25.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex ... 2023-10-21T04:36:25.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:36:25.310+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex ... 2023-10-21T04:36:25.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_1.index destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3464125101225762400_0.index destroyed 2023-10-21T04:36:25.325+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.325+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 1 Destroy Done 2023-10-21T04:36:25.354+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3464125101225762400, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:36:25.354+05:30 [Info] Indexer::cleanupIndexData IndexInst 3464125101225762400 Partition 0 Destroy Done 2023-10-21T04:36:25.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:36:25.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex ... 2023-10-21T04:36:25.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:36:25.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex ... 2023-10-21T04:36:25.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/mainIndex sucessfully destroyed 2023-10-21T04:36:25.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_5.index destroyed 2023-10-21T04:36:25.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:36:25.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex ... 2023-10-21T04:36:25.366+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.366+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 5 Destroy Done 2023-10-21T04:36:25.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : metadata saved successfully 2023-10-21T04:36:25.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_10032435514095102923_4.index destroyed 2023-10-21T04:36:25.372+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10032435514095102923, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:36:25.372+05:30 [Info] Indexer::cleanupIndexData IndexInst 10032435514095102923 Partition 4 Destroy Done 2023-10-21T04:36:25.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:25.517+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:25.543+05:30 [Info] switched currmeta from 207 -> 207 force true 2023-10-21T04:36:25.543+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:25.558+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:25.564+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:25.565+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:25.584+05:30 [Info] switched currmeta from 207 -> 207 force true 2023-10-21T04:36:25.590+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:25.590+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:25.593+05:30 [Info] switched currmeta from 207 -> 207 force true 2023-10-21T04:36:25.600+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:25.600+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:25.604+05:30 [Info] switched currmeta from 207 -> 207 force true 2023-10-21T04:36:25.726+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 16787445029868804812 2023-10-21T04:36:25.726+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 16787445029868804812 2023-10-21T04:36:25.726+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 2925063509317026816 2023-10-21T04:36:25.726+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 2925063509317026816 2023-10-21T04:36:25.726+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [16787445029868804812 2925063509317026816], elapsed: 467.229589ms 2023-10-21T04:36:25.726+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:36:25.726+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:36:25.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:36:25.726+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:36:25.727+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper stopped 2023-10-21T04:36:25.727+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': stopped 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:36:25.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:36:25.727+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:36:25.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:36:25.730+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:36:25.731+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:36:25.731+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:36:25.731+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:36:25.732+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Begin close 2023-10-21T04:36:25.733+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.Close: Done close 2023-10-21T04:36:25.733+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:36:25.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:36:25.734+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812/data/recovery(shard16787445029868804812) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:36:25.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:36:25.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shutdown completed 2023-10-21T04:36:25.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:36:25.735+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812 (16787445029868804812) removed 2023-10-21T04:36:25.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All daemons stopped 2023-10-21T04:36:25.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance closed 2023-10-21T04:36:25.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : All instance destroyed 2023-10-21T04:36:25.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard16787445029868804812(16787445029868804812)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:36:25.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:36:25.739+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:36:25.740+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:36:25.740+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:36:25.740+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:36:25.740+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:36:25.741+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:36:25.741+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Received stop message 2023-10-21T04:36:25.741+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoveryCleaner.lssCleanerDaemon: Stopped daemon 2023-10-21T04:36:25.741+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:36:25.742+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:36:25.742+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Begin stopping data LSS Cleaner 2023-10-21T04:36:25.742+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCleaner.Stop: Done stopping data LSS Cleaner 2023-10-21T04:36:25.742+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:36:25.742+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:36:25.743+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:36:25.743+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:36:25.743+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Begin stopping recovery LSS Cleaner 2023-10-21T04:36:25.743+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : RecoverCleaner.Stop: Done stopping recovery LSS Cleaner 2023-10-21T04:36:25.743+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Received stop message 2023-10-21T04:36:25.743+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.runtimeStats: Stopped daemon 2023-10-21T04:36:25.744+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:36:25.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:36:25.744+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:36:25.744+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:36:25.747+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:36:25.748+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:36:25.748+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:36:25.749+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:36:25.749+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:36:25.749+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Begin close 2023-10-21T04:36:25.750+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.Close: Done close 2023-10-21T04:36:25.750+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:36:25.750+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:36:25.750+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816/data/recovery(shard2925063509317026816) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:36:25.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:36:25.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shutdown completed 2023-10-21T04:36:25.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:36:25.751+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816 (2925063509317026816) removed 2023-10-21T04:36:25.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All daemons stopped 2023-10-21T04:36:25.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance closed 2023-10-21T04:36:25.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : All instance destroyed 2023-10-21T04:36:25.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard2925063509317026816(2925063509317026816)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:36:25.760+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Cleaned up Tenant with name:bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:36:25.760+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [16787445029868804812 2925063509317026816], elapsed(sec): 0.501174633 2023-10-21T04:36:25.763+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokene9:6c:98:3e:fb:d7:77:38 MasterId: 268b83ffbd4df37978153ecb76d33816 SourceId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) DestId: 268b83ffbd4df37978153ecb76d33816 (127.0.0.1:9001) RebalId: 2a6a1478070bd9349853af27a661ad2b ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: Shards: [17406544375239214575 10170029854712240877] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12127044935527081292 RealInstId: 8732144615635911668 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3127033699511712809 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14327413444160233932 RealInstId: 8965923595543532942 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 17434304260397056806 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:25.779+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokene9:6c:98:3e:fb:d7:77:38 2023-10-21T04:36:25.779+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 2a6a1478070bd9349853af27a661ad2b TransferTokenId: ShardTokene9:6c:98:3e:fb:d7:77:38 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:36:25.780+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000116466 2023-10-21T04:36:25.780+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38, shardId :0 2023-10-21T04:36:25.780+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/2a6a1478070bd9349853af27a661ad2b_ShardTokene9_6c_98_3e_fb_d7_77_38, shardId :0 2023-10-21T04:36:25.780+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:36:25.780+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 3127033699511712809, partnId: 0 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 7 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 8 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 1 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 2 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 3 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 4 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 5 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8965923595543532942, partnId: 6 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 1 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 2 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 3 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 4 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 5 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 6 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 7 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8732144615635911668, partnId: 8 2023-10-21T04:36:25.780+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17434304260397056806, partnId: 0 2023-10-21T04:36:25.780+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.675µs 2023-10-21T04:36:25.780+05:30 [Info] ShardTransferManager::handleRestoreAndUnlockShards Initiating unlock for shardId: 17406544375239214575 2023-10-21T04:36:25.780+05:30 [Info] ShardTransferManager::handleRestoreAndUnlockShards Initiating unlock for shardId: 10170029854712240877 2023-10-21T04:36:25.780+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:36:25.780+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:36:25.780+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:36:25.788+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:36:25.788+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:36:25.788+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:36:25.789+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:36:25.789+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 36.639µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:25.790+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:36:25.790+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:36:25.790+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:36:25.790+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:36:25.790+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:36:25.790+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:36:25.791+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:36:25.791+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 32.656µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:25.792+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:36:25.792+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:36:25.792+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.052281697s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:36:25.792+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/2a6a1478070bd9349853af27a661ad2b, rev: [] call. err: 2023-10-21T04:36:25.793+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[268b83ffbd4df37978153ecb76d33816 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:25.793+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 6] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[]} 2023-10-21T04:36:25.794+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:36:25.794+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[]} 2023-10-21T04:36:25.795+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:36:25.795+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[268b83ffbd4df37978153ecb76d33816 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:25.797+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 7] 2023-10-21T04:36:25.797+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 7] 2023-10-21T04:36:27.480+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:37994 2023-10-21T04:36:27.505+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 20.074175ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:27.506+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 40.576µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:27.552+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49588" doAuth() ... 2023-10-21T04:36:27.554+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:49588" doReceive() ... 2023-10-21T04:36:27.726+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:36:28.007+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:36:28.050+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3127033699511712809 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:36:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3127033699511712809 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.216µs SnapOpenDur 120.079µs 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:36:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 SnapshotId 13 Creating recovery point ... 2023-10-21T04:36:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17434304260397056806 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:6 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.557µs SnapOpenDur 69.606µs 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:36:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 7 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:131 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.402µs SnapOpenDur 59.645µs 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:36:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 8 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.565µs SnapOpenDur 41.273µs 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:36:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 1 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.413µs SnapOpenDur 35.404µs 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:36:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 2 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.75µs SnapOpenDur 34.064µs 2023-10-21T04:36:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 7 Creating recovery point ... 2023-10-21T04:36:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:36:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.051+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 3 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.057µs SnapOpenDur 225.464µs 2023-10-21T04:36:28.051+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Creating New Snapshot SnapshotInfo: count:118 committed:true 2023-10-21T04:36:28.052+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 4 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:118 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.021µs SnapOpenDur 60.094µs 2023-10-21T04:36:28.052+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:36:28.052+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 5 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.226µs SnapOpenDur 36.716µs 2023-10-21T04:36:28.052+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:36:28.052+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 6 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.388µs SnapOpenDur 35.801µs 2023-10-21T04:36:28.052+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.052+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.052+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 4 Creating recovery point ... 2023-10-21T04:36:28.098+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 7 Created recovery point (took 46.572459ms) 2023-10-21T04:36:28.104+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:36:28.194+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 SnapshotId 13 Created recovery point (took 142.878795ms) 2023-10-21T04:36:28.196+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:36:28.197+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0 Cleanup backstore recovery point SnapshotInfo: count:1000 committed:false. num RPs 3. 2023-10-21T04:36:28.202+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 4 Created recovery point (took 150.375356ms) 2023-10-21T04:36:28.202+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:36:28.203+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:36:28.213+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 4 Created recovery point (took 162.107828ms) 2023-10-21T04:36:28.215+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:36:28.216+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:36:28.226+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 4 Created recovery point (took 175.214746ms) 2023-10-21T04:36:28.228+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:36:28.229+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:36:28.237+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 4 Created recovery point (took 185.598926ms) 2023-10-21T04:36:28.238+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:36:28.238+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:36:28.255+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 4 Created recovery point (took 203.894066ms) 2023-10-21T04:36:28.256+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:36:28.257+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:36:28.278+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 4 Created recovery point (took 225.636479ms) 2023-10-21T04:36:28.279+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:36:28.279+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:36:28.298+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 4 Created recovery point (took 246.175134ms) 2023-10-21T04:36:28.299+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:36:28.300+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:36:28.311+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 4 Created recovery point (took 258.636609ms) 2023-10-21T04:36:28.311+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:36:28.312+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:36:30.299+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:30.304+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:30.304+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:30.306+05:30 [Info] switched currmeta from 207 -> 209 force true 2023-10-21T04:36:32.068+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51262 2023-10-21T04:36:32.103+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51262 2023-10-21T04:36:32.238+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697843187283507212, Old rollbacktime: 1697843177420910984 2023-10-21T04:36:32.238+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697843187283507212, Old rollbacktime: 1697843177420910984 {Index inst : [partitions]}: map[15864358084358521096:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697843187283507212, Old rollbacktime: 0 2023-10-21T04:36:32.238+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3464125101225762400:[0]]. New rollbacktime: 1697843187283507212, Old rollbacktime: 1697843177420910984 {Index inst : [partitions]}: map[15055366427687156590:[0] 15864358084358521096:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697843187283507212, Old rollbacktime: 0 2023-10-21T04:36:46.795+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:36:47.035+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:47.075+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {ea726006a91d328b09c6b7ff3c444060 [] topology-change-failover [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] []} 2023-10-21T04:36:47.075+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:36:47.076+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 8] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:47.076+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 7] call. taskList: &{Rev:[0 0 0 0 0 0 0 8] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ea726006a91d328b09c6b7ff3c444060 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ea726006a91d328b09c6b7ff3c444060]}]} 2023-10-21T04:36:47.092+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 8] 2023-10-21T04:36:47.103+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 8] 2023-10-21T04:36:47.104+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {ea726006a91d328b09c6b7ff3c444060 [] topology-change-failover [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] []} 2023-10-21T04:36:47.104+05:30 [Info] NewRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken master true localaddr runPlanner false runParam clusterVersion 9 2023-10-21T04:36:47.104+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:36:47.104+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 8] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ea726006a91d328b09c6b7ff3c444060 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ea726006a91d328b09c6b7ff3c444060]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/ea726006a91d328b09c6b7ff3c444060 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ea726006a91d328b09c6b7ff3c444060]}]} 2023-10-21T04:36:47.104+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:47.110+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:36:47.110+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. taskList: &{Rev:[0 0 0 0 0 0 0 10] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ea726006a91d328b09c6b7ff3c444060 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ea726006a91d328b09c6b7ff3c444060]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/ea726006a91d328b09c6b7ff3c444060 Type:task-rebalance Status:task-running IsCancelable:true Progress:1 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ea726006a91d328b09c6b7ff3c444060]}]} 2023-10-21T04:36:47.113+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:36:47.113+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 10] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ea726006a91d328b09c6b7ff3c444060 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ea726006a91d328b09c6b7ff3c444060]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/ea726006a91d328b09c6b7ff3c444060 Type:task-rebalance Status:task-running IsCancelable:true Progress:1 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ea726006a91d328b09c6b7ff3c444060]}]} 2023-10-21T04:36:47.115+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:36:47.115+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 10] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:47.125+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 10] 2023-10-21T04:36:47.130+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 10] 2023-10-21T04:36:47.130+05:30 [Info] Rebalancer::doFinish cleanup started 2023-10-21T04:36:47.130+05:30 [Info] Rebalancer::doFinish Cleanup 2023-10-21T04:36:47.130+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:36:47.130+05:30 [Info] Rebalancer::processDropIndexQueue Done Received 2023-10-21T04:36:47.215+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:47.215+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:47.215+05:30 [Info] UnwatchMetadata(): indexer 268b83ffbd4df37978153ecb76d33816 2023-10-21T04:36:47.215+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:36:47.216+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:57526->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:36:47.216+05:30 [Info] UnwatchMetadata(): finish for indexer 268b83ffbd4df37978153ecb76d33816 2023-10-21T04:36:47.219+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:36:47.219+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:36:47.219+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.935µs 2023-10-21T04:36:47.219+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:36:47.219+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:36:47.219+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:36:47.221+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:36:47.222+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:36:47.222+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:36:47.222+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:36:47.222+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 37.536µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:47.223+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:36:47.223+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:36:47.223+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced false, isMaster true, forceUnbalanced true, err: 2023-10-21T04:36:47.223+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 10] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[]} 2023-10-21T04:36:47.223+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:47.223+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:36:47.225+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 11] 2023-10-21T04:36:47.225+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 11] 2023-10-21T04:36:47.228+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:36:47.228+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[]} 2023-10-21T04:36:47.234+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:36:47.234+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:36:47.251+05:30 [Info] switched currmeta from 209 -> 210 force true 2023-10-21T04:36:47.251+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:36:47.251+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] ... stopped 2023-10-21T04:36:47.274+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 11] 2023-10-21T04:36:47.275+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 11] 2023-10-21T04:36:47.276+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:36:47.276+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:36:47.299+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38488 2023-10-21T04:36:47.300+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:36:47.334+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Stopping releaseConnsRoutine 2023-10-21T04:36:47.347+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 44.621118ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:47.348+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:36:47.348+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:36:47.348+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:36:47.364+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:47.419+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:36:47.486+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:47.486+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:47.495+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:47.496+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:47.646+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:47.681+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:36:47.681+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:36:47.681+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:36:47.706+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:36:47.706+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:36:47.706+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:36:47.706+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:36:47.706+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:36:47.706+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:38488->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:36:47.707+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:34872->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:36:47.707+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38488. Reason = EOF. Cleaning Pipe. 2023-10-21T04:36:47.707+05:30 [Info] messageListener.start(): message channel closed. Remove peer d7:40:b:15:5:a0:42:73 as follower. 2023-10-21T04:36:47.743+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 1 unhealthy node 0 add node 0 2023-10-21T04:36:47.743+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:47.743+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:47.748+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:47.769+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:47.769+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:47.775+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:47.789+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:37994. Reason = EOF. Cleaning Pipe. 2023-10-21T04:36:47.789+05:30 [Info] messageListener.start(): message channel closed. Remove peer d1:ca:f5:ba:f8:cf:50:6 as follower. 2023-10-21T04:36:47.793+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:49588 closed 2023-10-21T04:36:47.849+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:36:47.878+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:47.902+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:47.911+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:47.911+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:47.925+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:47.968+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:47.969+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:48.010+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:48.071+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:48.072+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:48.187+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:48.187+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:48.190+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:48.196+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:48.196+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:48.198+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:48.287+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:48.294+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:48.294+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:48.295+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:48.339+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:48.368+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:48.368+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:48.379+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:48.579+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:48.711+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:48.711+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:48.714+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:48.741+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 216] updated cluster rebalancing status to true 2023-10-21T04:36:48.741+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 216] updating rebalancing status to true 2023-10-21T04:36:48.752+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:48.756+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:48.757+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:48.760+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:48.986+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:49.048+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:49.048+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:49.051+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:49.280+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:36:49.368+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:36:49.368+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:49.368+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:49.385+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:49.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9591316647008257, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:36:49.498+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:49.516+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:49.516+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:49.523+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:50.311+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:50.315+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:50.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:50.318+05:30 [Info] switched currmeta from 210 -> 210 force true 2023-10-21T04:36:50.378+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {f3a5b85675e4f53a445d2e873c593537 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] []} 2023-10-21T04:36:50.382+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 7.91µs 2023-10-21T04:36:50.382+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:36:50.382+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:36:50.382+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:36:50.384+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:36:50.384+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:50.384+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 1.891535ms 2023-10-21T04:36:50.384+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:36:50.384+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:36:50.384+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:50.384+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 39.238µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:50.384+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 11] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/f3a5b85675e4f53a445d2e873c593537 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]}]} 2023-10-21T04:36:50.385+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 11] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/f3a5b85675e4f53a445d2e873c593537 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]}]} 2023-10-21T04:36:50.385+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:50.387+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 12] 2023-10-21T04:36:50.387+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 12] 2023-10-21T04:36:50.391+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {f3a5b85675e4f53a445d2e873c593537 [] topology-change-rebalance [{{18194302e3f10df59bd076e1c4142d0b 7060000 } recovery-full} {{8806f5706747af4e635b8da5fcd8aeee 7060000 } recovery-full}] []} 2023-10-21T04:36:50.393+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:36:50.413+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"f3a5b85675e4f53a445d2e873c593537","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:36:50.415+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:36:50.418+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:36:50.418+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:36:50.488+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7919327262683501, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:36:51.428+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:36:51.428+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:36:51.428+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:36:51.428+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.037223049s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:36:51.428+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:36:51.428+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:51.428+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 12] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/f3a5b85675e4f53a445d2e873c593537 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/f3a5b85675e4f53a445d2e873c593537 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]}]} 2023-10-21T04:36:51.429+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:36:51.432+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:36:51.433+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:51.432+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:36:51.432+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:36:51.432+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:36:51.433+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/f3a5b85675e4f53a445d2e873c593537 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/f3a5b85675e4f53a445d2e873c593537 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]}]} 2023-10-21T04:36:51.435+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:36:51.435+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:36:51.483+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{18194302e3f10df59bd076e1c4142d0b 18194302e3f10df59bd076e1c4142d0b plasma map[] [{64 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{8965923595543532942 4 1 [{1 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{8732144615635911668 1 0 [{1 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {2 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {3 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {4 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {5 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {6 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {7 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []} {8 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 14549775068266281786 [{3127033699511712809 4 1 [{0 0 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575 10170029854712240877] []}] 1 0 0 1 false plasma 0}]}]} {15 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{17434304260397056806 4 1 [{0 1 {[{0 0 18194302e3f10df59bd076e1c4142d0b}]} {[] []} [17406544375239214575] []}] 1 0 1 1 false plasma 0}]}]} {76 bucket_1 _default c1 []} {29 bucket_1 _system _query []}] [DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {8806f5706747af4e635b8da5fcd8aeee 8806f5706747af4e635b8da5fcd8aeee plasma map[] [{64 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned_defer 10606939961365971246 [{6225851069678382014 1 0 [{1 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 1 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned 8910459001587546151 [{6043349427976659583 4 1 [{1 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {2 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {3 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {4 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {5 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {6 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {7 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []} {8 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []}] 8 0 1 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 14549775068266281786 [{9702649553456094532 4 1 [{0 0 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593 1249143295319901191] []}] 1 0 0 0 false plasma 0}]}]} {19 bucket_%2 _system _query [{bucket_%2 _system _query #primary 3381524789655566900 [{2535881715450691825 4 1 [{0 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [7434639758576776593] []}] 1 0 1 0 false plasma 0}]}]} {87 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 16961575949607441659 [{3345094951175719474 4 1 [{1 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16337157693634799600 [{6377153279641237139 1 0 [{1 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {2 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {3 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {4 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {5 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {6 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {7 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []} {8 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 12582561312368048736 [{1361754384643230279 4 1 [{0 0 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908 15531034489713578274] []}] 1 0 0 0 false plasma 0}]}]} {24 bucket_1 _system _query [{bucket_1 _system _query #primary 11276338709617092948 [{10601152754683910907 4 1 [{0 1 {[{0 0 8806f5706747af4e635b8da5fcd8aeee}]} {[] []} [4718895791748289908] []}] 1 0 1 1 false plasma 0}]}]}] [DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false}] map[]} 2023-10-21T04:36:51.590+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44900 2023-10-21T04:36:51.596+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:36:51.596+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:36:51.597+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44900 2023-10-21T04:36:51.618+05:30 [Info] Planner::processDeleteToken: processing 42 tokens for node 127.0.0.1:9003 2023-10-21T04:36:51.618+05:30 [Info] Planner::processDeleteToken: processing 42 tokens for node 127.0.0.1:9004 2023-10-21T04:36:51.618+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:44900 2023-10-21T04:36:51.624+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:36:51.624+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:36:51.668+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 18206354 Units 2 Tenants 2 2023-10-21T04:36:51.668+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 9102406 Units 2 Tenants 1 2023-10-21T04:36:51.668+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:36:51.668+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_system,_query,#primary,0) on 127.0.0.1:9003 2023-10-21T04:36:51.668+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_secondary,1) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned_defer,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::placeMissingReplicaOnTarget Rebuilding lost replica for (bucket_1,_default,c1,idx_partitioned,0) on 127.0.0.1:9003 2023-10-21T04:36:51.669+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:36:51.669+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:36:51.669+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:36:51.669+05:30 [Info] ---------------------------------------- 2023-10-21T04:36:51.669+05:30 [Info] 2023-10-21T04:36:51.669+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:8806f5706747af4e635b8da5fcd8aeee, useLiveData:false 2023-10-21T04:36:51.669+05:30 [Info] Indexer total memory:5768598634 (5.37243G), mem:798040 (779.336K), overhead:5767800594 (5.37168G), min:5768598634 (5.37243G), data:798040 (779.336K) cpu:0.2500, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:36:51.669+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.669+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.669+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.669+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.669+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.669+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.669+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.669+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.669+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.669+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.669+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.669+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.669+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.669+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.669+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.669+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.669+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:6225851069678382014, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:10601152754683910907, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:157293350 (150.007M), mem:6018 (5.87695K), overhead:157287332 (150.001M), min:157293350 (150.007M), data:6018 (5.87695K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:12582561312368048736, instId:1361754384643230279, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0901 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:6377153279641237139, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:157326885 (150.039M), mem:35052 (34.2305K), overhead:157291833 (150.005M), min:157326885 (150.039M), data:35052 (34.2305K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.670+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.670+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.670+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.670+05:30 [Info] Index total memory:157327712 (150.039M), mem:35768 (34.9297K), overhead:157291944 (150.005M), min:157327712 (150.039M), data:35768 (34.9297K) cpu:0.0056 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157326825 (150.039M), mem:35000 (34.1797K), overhead:157291825 (150.005M), min:157326825 (150.039M), data:35000 (34.1797K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157327546 (150.039M), mem:35625 (34.79K), overhead:157291921 (150.005M), min:157327546 (150.039M), data:35625 (34.79K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157327598 (150.039M), mem:35670 (34.834K), overhead:157291928 (150.005M), min:157327598 (150.039M), data:35670 (34.834K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157327545 (150.039M), mem:35624 (34.7891K), overhead:157291921 (150.005M), min:157327545 (150.039M), data:35624 (34.7891K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0056 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:3345094951175719474, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157325700 (150.037M), mem:34026 (33.2285K), overhead:157291674 (150.005M), min:157325700 (150.037M), data:34026 (33.2285K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:2535881715450691825, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157294043 (150.007M), mem:6618 (6.46289K), overhead:157287425 (150.001M), min:157294043 (150.007M), data:6618 (6.46289K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:14549775068266281786, instId:9702649553456094532, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157671030 (150.367M), mem:111000 (108.398K), overhead:157560030 (150.261M), min:157671030 (150.367M), data:111000 (108.398K) cpu:0.0795 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157326908 (150.039M), mem:35072 (34.25K), overhead:157291836 (150.005M), min:157326908 (150.039M), data:35072 (34.25K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157327314 (150.039M), mem:35424 (34.5938K), overhead:157291890 (150.005M), min:157327314 (150.039M), data:35424 (34.5938K) cpu:0.0048 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157327598 (150.039M), mem:35670 (34.834K), overhead:157291928 (150.005M), min:157327598 (150.039M), data:35670 (34.834K) cpu:0.0048 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.671+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.671+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.671+05:30 [Info] Index total memory:157326878 (150.039M), mem:35046 (34.2246K), overhead:157291832 (150.005M), min:157326878 (150.039M), data:35046 (34.2246K) cpu:0.0046 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.671+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157327682 (150.039M), mem:35742 (34.9043K), overhead:157291940 (150.005M), min:157327682 (150.039M), data:35742 (34.9043K) cpu:0.0054 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157326591 (150.038M), mem:34798 (33.9824K), overhead:157291793 (150.005M), min:157326591 (150.038M), data:34798 (33.9824K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157327101 (150.039M), mem:35239 (34.4131K), overhead:157291862 (150.005M), min:157327101 (150.039M), data:35239 (34.4131K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:6043349427976659583, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [7434639758576776593 1249143295319901191], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157326372 (150.038M), mem:34608 (33.7969K), overhead:157291764 (150.005M), min:157326372 (150.038M), data:34608 (33.7969K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] 2023-10-21T04:36:51.672+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:18194302e3f10df59bd076e1c4142d0b, useLiveData:false 2023-10-21T04:36:51.672+05:30 [Info] Indexer total memory:2936725506 (2.73504G), mem:398468 (389.129K), overhead:2936327038 (2.73467G), min:2936725506 (2.73504G), data:398468 (389.129K) cpu:0.1475, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:36:51.672+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:14549775068266281786, instId:3127033699511712809, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0863 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:3381524789655566900, instId:17434304260397056806, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157293316 (150.007M), mem:5988 (5.84766K), overhead:157287328 (150.001M), min:157293316 (150.007M), data:5988 (5.84766K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157326760 (150.038M), mem:34944 (34.125K), overhead:157291816 (150.005M), min:157326760 (150.038M), data:34944 (34.125K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.672+05:30 [Info] Index total memory:157327598 (150.039M), mem:35670 (34.834K), overhead:157291928 (150.005M), min:157327598 (150.039M), data:35670 (34.834K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.672+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.672+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.672+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.673+05:30 [Info] Index total memory:157327882 (150.04M), mem:35916 (35.0742K), overhead:157291966 (150.005M), min:157327882 (150.04M), data:35916 (35.0742K) cpu:0.0076 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.673+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.676+05:30 [Info] Index total memory:157327014 (150.039M), mem:35164 (34.3398K), overhead:157291850 (150.005M), min:157327014 (150.039M), data:35164 (34.3398K) cpu:0.0072 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.676+05:30 [Info] Index total memory:157328000 (150.04M), mem:36018 (35.1738K), overhead:157291982 (150.005M), min:157328000 (150.04M), data:36018 (35.1738K) cpu:0.0084 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.676+05:30 [Info] Index total memory:157326885 (150.039M), mem:35052 (34.2305K), overhead:157291833 (150.005M), min:157326885 (150.039M), data:35052 (34.2305K) cpu:0.0078 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.676+05:30 [Info] Index total memory:157326949 (150.039M), mem:35108 (34.2852K), overhead:157291841 (150.005M), min:157326949 (150.039M), data:35108 (34.2852K) cpu:0.0080 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:8910459001587546151, instId:8965923595543532942, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.676+05:30 [Info] Index total memory:157326372 (150.038M), mem:34608 (33.7969K), overhead:157291764 (150.005M), min:157326372 (150.038M), data:34608 (33.7969K) cpu:0.0068 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.676+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:10606939961365971246, instId:8732144615635911668, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17406544375239214575 10170029854712240877], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:11276338709617092948, instId:3643393537124379309, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:157293350 (150.007M), mem:6018 (5.87695K), overhead:157287332 (150.001M), min:157293350 (150.007M), data:6018 (5.87695K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:12582561312368048736, instId:4443661069773468873, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.677+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0901 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.677+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:3580008349810282285, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:3713514391612952033, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:18027006066212494029, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:8596886222467472262, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:7731545429291016377, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:7085324877944716265, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:11615515309843986700, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16337157693634799600, instId:10092313590244003404, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:7516828288031590951, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:157326885 (150.039M), mem:35052 (34.2305K), overhead:157291833 (150.005M), min:157326885 (150.039M), data:35052 (34.2305K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:15143545425026414670, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:157327712 (150.039M), mem:35768 (34.9297K), overhead:157291944 (150.005M), min:157327712 (150.039M), data:35768 (34.9297K) cpu:0.0056 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:5790753322281801334, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.678+05:30 [Info] Index total memory:157326825 (150.039M), mem:35000 (34.1797K), overhead:157291825 (150.005M), min:157326825 (150.039M), data:35000 (34.1797K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.678+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:10659414698947748564, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.679+05:30 [Info] Index total memory:157327546 (150.039M), mem:35625 (34.79K), overhead:157291921 (150.005M), min:157327546 (150.039M), data:35625 (34.79K) cpu:0.0052 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.679+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:6229156650115639552, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.679+05:30 [Info] Index total memory:157327598 (150.039M), mem:35670 (34.834K), overhead:157291928 (150.005M), min:157327598 (150.039M), data:35670 (34.834K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.679+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:11967158353644191343, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.679+05:30 [Info] Index total memory:157327545 (150.039M), mem:35624 (34.7891K), overhead:157291921 (150.005M), min:157327545 (150.039M), data:35624 (34.7891K) cpu:0.0050 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.679+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:18243258950605066295, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.679+05:30 [Info] Index total memory:157328026 (150.04M), mem:36040 (35.1953K), overhead:157291986 (150.005M), min:157328026 (150.04M), data:36040 (35.1953K) cpu:0.0056 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:36:51.679+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16961575949607441659, instId:16747354934383523994, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [4718895791748289908 15531034489713578274], shardIdsInDefn: map[] 2023-10-21T04:36:51.679+05:30 [Info] Index total memory:157325700 (150.037M), mem:34026 (33.2285K), overhead:157291674 (150.005M), min:157325700 (150.037M), data:34026 (33.2285K) cpu:0.0044 io:0 (0) scan:0 drain:0 2023-10-21T04:36:51.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:36:51.679+05:30 [Info] **************************************** 2023-10-21T04:36:51.679+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 18206354 Units 2 Tenants 2 2023-10-21T04:36:51.679+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 9346458 Units 2 Tenants 1 2023-10-21T04:36:51.679+05:30 [Info] Generating Shard Transfer Token (ShardToken28:ac:25:bb:38:c:fb:af) for rebalance ( MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: Region: InstRenameMap: map[4718895791748289908:map[bucket_1_#primary_10601152754683910907_0.index/mainIndex:bucket_1_#primary_3643393537124379309_0.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_1.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_2.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_3.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_4.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_5.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_6.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_7.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_8.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_1.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_2.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_3.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_4.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_5.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_6.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_7.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_8.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex bucket_1_idx_secondary_1361754384643230279_0.index/mainIndex:bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex] 15531034489713578274:map[bucket_1_idx_partitioned_3345094951175719474_1.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex bucket_1_idx_partitioned_3345094951175719474_2.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex bucket_1_idx_partitioned_3345094951175719474_3.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex bucket_1_idx_partitioned_3345094951175719474_4.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex bucket_1_idx_partitioned_3345094951175719474_5.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex bucket_1_idx_partitioned_3345094951175719474_6.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex bucket_1_idx_partitioned_3345094951175719474_7.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex bucket_1_idx_partitioned_3345094951175719474_8.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_1.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_2.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_3.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_4.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_5.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_6.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_7.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_8.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex bucket_1_idx_secondary_1361754384643230279_0.index/docIndex:bucket_1_idx_secondary_4443661069773468873_0.index/docIndex]] InstId: 4443661069773468873 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 14010044466748520509 RealInstId: 3580008349810282285 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 3580008349810282285 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 6643622002716756177 RealInstId: 7516828288031590951 Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] Inst: InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: InstId: 3643393537124379309 RealInstId: 0 Partitions: [0] Versions: [2] Inst: InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:36:51.680+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:36:51.681+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:36:51.685+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken28:ac:25:bb:38:c:fb:af 2023-10-21T04:36:51.685+05:30 [Info] ShardRebalancer::observeRebalance &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:36:51.689+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken28:ac:25:bb:38:c:fb:af MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4443661069773468873 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14010044466748520509 RealInstId: 3580008349810282285 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6643622002716756177 RealInstId: 7516828288031590951 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3643393537124379309 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:51.689+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"f3a5b85675e4f53a445d2e873c593537","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:36:51.689+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:36:51.730+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken28:ac:25:bb:38:c:fb:af MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4443661069773468873 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14010044466748520509 RealInstId: 3580008349810282285 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6643622002716756177 RealInstId: 7516828288031590951 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3643393537124379309 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:51.731+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:36:51.731+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:36:51.731+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:36:51.731+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 52.12µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:51.737+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken28:ac:25:bb:38:c:fb:af MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4443661069773468873 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14010044466748520509 RealInstId: 3580008349810282285 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6643622002716756177 RealInstId: 7516828288031590951 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3643393537124379309 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:51.745+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:51.745+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:36:51.758+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:36:51.758+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 1 2023-10-21T04:36:51.776+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[8806f5706747af4e635b8da5fcd8aeee:ShardToken28:ac:25:bb:38:c:fb:af], currActiveTransfers: map[8806f5706747af4e635b8da5fcd8aeee:1] 2023-10-21T04:36:51.776+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"f3a5b85675e4f53a445d2e873c593537","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:36:51.776+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:36:51.776+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:36:51.776+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:36:51.776+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:36:51.776+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:36:51.777+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken28:ac:25:bb:38:c:fb:af MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4443661069773468873 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14010044466748520509 RealInstId: 3580008349810282285 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6643622002716756177 RealInstId: 7516828288031590951 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3643393537124379309 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:51.777+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 16.016µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:51.917+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken28:ac:25:bb:38:c:fb:af MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4443661069773468873 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14010044466748520509 RealInstId: 3580008349810282285 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6643622002716756177 RealInstId: 7516828288031590951 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3643393537124379309 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:51.917+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:1 8806f5706747af4e635b8da5fcd8aeee:0], batchSize: 1 2023-10-21T04:36:51.917+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:36:51.917+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:36:51.918+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:36:51.918+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.242µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:51.918+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[4718895791748289908:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908 15531034489713578274:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274] f3a5b85675e4f53a445d2e873c593537 ShardToken28:ac:25:bb:38:c:fb:af /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00d39ee40 0xc00d39eea0 map[4718895791748289908:map[bucket_1_#primary_10601152754683910907_0.index/mainIndex:bucket_1_#primary_3643393537124379309_0.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_1.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_2.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_3.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_4.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_5.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_6.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_7.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex bucket_1_idx_partitioned_3345094951175719474_8.index/mainIndex:bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_1.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_2.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_3.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_4.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_5.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_6.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_7.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex bucket_1_idx_partitioned_defer_6377153279641237139_8.index/mainIndex:bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex bucket_1_idx_secondary_1361754384643230279_0.index/mainIndex:bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex] 15531034489713578274:map[bucket_1_idx_partitioned_3345094951175719474_1.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex bucket_1_idx_partitioned_3345094951175719474_2.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex bucket_1_idx_partitioned_3345094951175719474_3.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex bucket_1_idx_partitioned_3345094951175719474_4.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex bucket_1_idx_partitioned_3345094951175719474_5.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex bucket_1_idx_partitioned_3345094951175719474_6.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex bucket_1_idx_partitioned_3345094951175719474_7.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex bucket_1_idx_partitioned_3345094951175719474_8.index/docIndex:bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_1.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_2.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_3.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_4.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_5.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_6.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_7.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex bucket_1_idx_partitioned_defer_6377153279641237139_8.index/docIndex:bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex bucket_1_idx_secondary_1361754384643230279_0.index/docIndex:bucket_1_idx_secondary_4443661069773468873_0.index/docIndex]] 0xc0001a4b40 0xc004fc2240 false} 2023-10-21T04:36:51.918+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908 2023-10-21T04:36:51.919+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274 2023-10-21T04:36:51.923+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274/data: super block snapshot head offset :0 tail offset:1556480 sbgen :261 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274/data/recovery: super block snapshot head offset :0 tail offset:1417216 sbgen :261 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_1.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_4.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_6.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_7.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_secondary_1361754384643230279_0.index/docIndex new indexPath :bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_2.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_3.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_5.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_8.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_3.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_5.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_4.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_6.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_7.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_1.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_8.index/docIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:36:51.924+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_2.index/docIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:51.925+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908/data: super block snapshot head offset :0 tail offset:1794048 sbgen :284 2023-10-21T04:36:51.925+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908/data/recovery: super block snapshot head offset :0 tail offset:1703936 sbgen :284 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_1.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_4.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_6.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_#primary_10601152754683910907_0.index/mainIndex new indexPath :bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_3.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_5.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_7.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_8.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_secondary_1361754384643230279_0.index/mainIndex new indexPath :bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_2.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_4.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_1.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_5.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_8.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_2.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_defer_6377153279641237139_3.index/mainIndex new indexPath :bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_6.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:36:51.926+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: remap instance in shard metadata : indexPath :bucket_1_idx_partitioned_3345094951175719474_7.index/mainIndex new indexPath :bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:36:51.930+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: metadata saved successfully 2023-10-21T04:36:51.932+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: metadata saved successfully 2023-10-21T04:36:51.937+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908: metadata saved successfully 2023-10-21T04:36:51.938+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 (4718895791748289908) added 2023-10-21T04:36:51.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:36:51.938+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 185116421 totalBytes: 3517212 numShards: 2 duration: 19.911585ms 2023-10-21T04:36:51.938+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 4718895791748289908 bytesWritten: 3517212, totalBytes: 3517212, transferRate: 1.851164210526316e+08 2023-10-21T04:36:51.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/4718895791748289908/shards/shard4718895791748289908, newshardId 4718895791748289908, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908, shardsize 3517212, duration 20.219424ms 2023-10-21T04:36:51.938+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 4718895791748289908, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908, err: , elapsed(sec): 0.020555936 2023-10-21T04:36:51.941+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274: metadata saved successfully 2023-10-21T04:36:51.941+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 (15531034489713578274) added 2023-10-21T04:36:51.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 from metadata completed, tenant :bucket_1_1fc3689e709c62b9ee184f2ec13e6f43 2023-10-21T04:36:51.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/f3a5b85675e4f53a445d2e873c593537_ShardToken28_ac_25_bb_38_c_fb_af/15531034489713578274/shards/shard15531034489713578274, newshardId 15531034489713578274, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274, shardsize 2992765, duration 22.852912ms 2023-10-21T04:36:51.942+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 15531034489713578274, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274, err: , elapsed(sec): 0.023978455 2023-10-21T04:36:51.942+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 15531034489713578274 bytesWritten: 2992765, totalBytes: 2992765, transferRate: 1.3603477272727272e+08 2023-10-21T04:36:51.942+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: f3a5b85675e4f53a445d2e873c593537, transferId: ShardToken28:ac:25:bb:38:c:fb:af 2023-10-21T04:36:51.942+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: f3a5b85675e4f53a445d2e873c593537, transferId: ShardToken28:ac:25:bb:38:c:fb:af, elapsed(sec): 0.000136503 2023-10-21T04:36:51.942+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[4718895791748289908: 15531034489713578274:], shardPaths: map[4718895791748289908:/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 15531034489713578274:/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.024176182 2023-10-21T04:36:51.942+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [4718895791748289908 15531034489713578274], ttid: ShardToken28:ac:25:bb:38:c:fb:af, elapsed(sec): 0.024590471 2023-10-21T04:36:51.945+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:51.957+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken28:ac:25:bb:38:c:fb:af MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4443661069773468873 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14010044466748520509 RealInstId: 3580008349810282285 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6643622002716756177 RealInstId: 7516828288031590951 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3643393537124379309 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:51.957+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[18194302e3f10df59bd076e1c4142d0b:0 8806f5706747af4e635b8da5fcd8aeee:0], batchSize: 1 2023-10-21T04:36:51.957+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken28:ac:25:bb:38:c:fb:af 2023-10-21T04:36:51.957+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[4718895791748289908:true 15531034489713578274:true] 2023-10-21T04:36:51.957+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 2.045µs 2023-10-21T04:36:51.957+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [4718895791748289908 15531034489713578274], isLockedForRecovery: true 2023-10-21T04:36:51.957+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [4718895791748289908 15531034489713578274], errMap: map[4718895791748289908: 15531034489713578274:], elapsed: 10.007µs 2023-10-21T04:36:51.958+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:36:51.958+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 11276338709617092948, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:36:51.986+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3643393537124379309,realInstId: 0, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:36:51.986+05:30 [Info] Indexer::handleRecoverIndex InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:36:51.986+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 141.266µs 2023-10-21T04:36:51.986+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 218.305µs 2023-10-21T04:36:51.986+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3643393537124379309 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:51.986+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3643393537124379309 PartitionId: 0 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:51.986+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3643393537124379309, indexDefn DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:36:51.986+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 28.53442ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:51.987+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:36:51.990+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : LSSCleaner initialized 2023-10-21T04:36:51.991+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:36:51.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:51.993+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : LSSCleaner initialized for recovery 2023-10-21T04:36:51.993+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:36:51.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery 2023-10-21T04:36:51.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:51.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data], Shared [true] 2023-10-21T04:36:51.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1703936] Data log Head [0] Tail [1794048] 2023-10-21T04:36:51.994+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1703936] 2023-10-21T04:36:51.997+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:51.997+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:36:51.998+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:36:51.998+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:51.998+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:36:51.999+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:51.999+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:36:51.999+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.000+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:36:52.000+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.000+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:36:52.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.001+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:36:52.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.001+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:36:52.002+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.003+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:36:52.004+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.005+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:36:52.008+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.024+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:52.012+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:52.025+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.026+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:52.027+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.027+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:52.027+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.027+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:52.027+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.027+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:52.027+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.027+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:52.027+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.027+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:52.027+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.027+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:52.028+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:52.028+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:36:52.029+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : replayOffset: 1785856 (RP:1774574) (CP:1785856) 2023-10-21T04:36:52.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1785856] took [35.197071ms] 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12721, memSzIndex 0, numRecords 125 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12401, memSzIndex 0, numRecords 122 2023-10-21T04:36:52.029+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13797, memSzIndex 0, numRecords 136 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12889, memSzIndex 0, numRecords 127 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12513, memSzIndex 0, numRecords 123 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 10832, memSzIndex 0, numRecords 106 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12723, memSzIndex 0, numRecords 125 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1055940, resident 1, memSz 70472, memSzIndex 353, numRecords 1000 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1055476, resident 0.2, memSz 438, memSzIndex 0, numRecords 10 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13778, memSzIndex 0, numRecords 136 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935225 controller mem 1458 2023-10-21T04:36:52.030+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1794048] replayOffset [1785856] 2023-10-21T04:36:52.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [36.078634ms] 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.030+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.031+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.031+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.031+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.031+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.032+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.032+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.032+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.032+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.032+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.032+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.033+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.033+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.033+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.033+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.033+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:36:52.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data], Shared [true]. Built [18] plasmas, took [39.156452ms] 2023-10-21T04:36:52.033+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1055476 memSz 64 2023-10-21T04:36:52.033+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 360005 count: 6 metaSz: 2483 rp [1] sn: 720008 count: 6 metaSz: 2483 2023-10-21T04:36:52.034+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 97084 recovery UsedSpace 165982 2023-10-21T04:36:52.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:36:52.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:52.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:52.043+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:36:52.043+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:36:52.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex started 2023-10-21T04:36:52.044+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3643393537124379309 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:36:52.044+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:52.044+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:52.065+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:52.065+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:52.072+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3643393537124379309 PartitionId 0 Warmup took 85.399961ms 2023-10-21T04:36:52.072+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3643393537124379309 partitionId 0 WriterThreads 4 2023-10-21T04:36:52.072+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3643393537124379309 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index Index: 3643393537124379309 Partition: 0 2023-10-21T04:36:52.072+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3643393537124379309, shardIdMap: map[0:[4718895791748289908]] err: 2023-10-21T04:36:52.072+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3643393537124379309, partnShardIdMap: map[0:[4718895791748289908]], err: 2023-10-21T04:36:52.072+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:52.072+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3643393537124379309 2023-10-21T04:36:52.072+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:52.073+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:36:52.073+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:52.073+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:52.073+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:52.073+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3643393537124379309 Partitions [0] 2023-10-21T04:36:52.074+05:30 [Info] StorageMgr::openSnapshot IndexInst:3643393537124379309 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:36:52.075+05:30 [Info] switched currmeta from 210 -> 211 force true 2023-10-21T04:36:52.080+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.176074ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:52.081+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 83.503µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:36:52.096+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:52.096+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:52.102+05:30 [Info] switched currmeta from 211 -> 212 force true 2023-10-21T04:36:53.054+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3643393537124379309:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:36:53.054+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [11276338709617092948] belonging to collectionId: 9, ttid: ShardToken28:ac:25:bb:38:c:fb:af. Initiating build 2023-10-21T04:36:53.061+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [3643393537124379309] &{2} 2023-10-21T04:36:53.061+05:30 [Info] Indexer::handleBuildRecoveredIndexes [3643393537124379309] 2023-10-21T04:36:53.061+05:30 [Info] bulkUpdateRState: Index instance 3643393537124379309 rstate moved to PENDING 2023-10-21T04:36:53.061+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [3643393537124379309] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:36:53.061+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:53.061+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:53.061+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:53.062+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:36:53.062+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.062+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.062+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 7300 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:53.062+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:36:53.062+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:36:53.062+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:36:53.062+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 7300 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:53.062+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:36:53.062+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.062+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.063+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:36:53.063+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:36:53.063+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 7301 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:36:53.063+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.063+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.063+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:36:53.063+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:36:53.063+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 3643393537124379309 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:36:53.063+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:36:53.063+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=23,1=27,2=28,3=24,4=18,5=23,6=23,7=17,8=22,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=30,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=25,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=25,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=34,63=17] IndexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 7301 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:36:53.064+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:36:53.064+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:36:53.064+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:53.064+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 3.614269ms 2023-10-21T04:36:53.065+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [3643393537124379309] 2023-10-21T04:36:53.065+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 10.039728ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:53.066+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[3643393537124379309:true] 2023-10-21T04:36:53.064+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 3.655015ms 2023-10-21T04:36:53.075+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:36:53.078+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.757427ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:53.079+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:36:53.090+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:53.100+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39442" doAuth() ... 2023-10-21T04:36:53.101+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39442" auth successful 2023-10-21T04:36:53.102+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:39442" +1 2023-10-21T04:36:53.109+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:53.110+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:53.113+05:30 [Info] switched currmeta from 212 -> 214 force true 2023-10-21T04:36:53.121+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:53.122+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:53.125+05:30 [Info] switched currmeta from 214 -> 214 force true 2023-10-21T04:36:53.411+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 0 1949031200982 23 7301 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:53.418+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 2 52522903947059 28 7301 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:36:53.418+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 1 245782287380001 27 7301 172.31.5.112:9000. HWT [26-27,27]. 2023-10-21T04:36:53.418+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 4 93079113162334 18 7301 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:53.418+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 3 205201940883916 24 7301 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:53.421+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 5 244791485011286 23 7301 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:53.424+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 6 167580527121081 23 7301 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:53.424+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 7 177815184127237 17 7301 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:53.425+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 8 54236768606397 22 7301 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:53.431+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 11 58522600746999 24 7301 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:53.431+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 9 134409829206048 26 7301 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:53.432+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 10 160381752747972 18 7301 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:53.433+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 12 142387448675642 30 7301 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:36:53.433+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 13 90115353725987 21 7301 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:53.439+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 15 85302001459467 21 7301 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:53.439+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 14 161230548567164 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.442+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 16 123957315902602 30 7301 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:36:53.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 23 53829314402679 32 7301 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:36:53.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 20 151273673850546 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 17 213108110408291 23 7301 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:53.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 18 163460438181870 34 7301 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:53.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 19 150690956328758 26 7301 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:53.455+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 21 147946970349859 24 7301 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:53.458+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 25 114823206239184 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.461+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 22 258671969827141 19 7301 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:53.461+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 27 232028101514735 19 7301 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:53.465+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 28 25510109770064 28 7301 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:36:53.466+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 29 8305763409275 21 7301 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:36:53.466+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 24 19989312661772 15 7301 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:36:53.466+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 26 68847799765163 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.468+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 31 207489375637709 12 7301 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:36:53.469+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 30 124811039198764 18 7301 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:53.472+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 33 107407811917992 20 7301 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:53.475+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 32 33310272161722 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.478+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 34 100746255343609 19 7301 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:53.481+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 35 117298435979365 24 7301 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:53.484+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 36 228857796432725 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.484+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 37 203503670520010 24 7301 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:53.486+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 39 158357249947806 21 7301 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:53.489+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 38 47156288300713 24 7301 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:53.489+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 41 236904976023311 22 7301 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:53.494+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 43 196269910725961 26 7301 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:53.494+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 40 104136033125373 29 7301 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:36:53.496+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 42 124310544527292 21 7301 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:53.497+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 45 276463747117997 25 7301 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:36:53.502+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 44 246959094981553 34 7301 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:53.502+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 47 172365063201867 21 7301 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:53.503+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 49 212884822297038 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.506+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 46 65885584526743 20 7301 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:53.506+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 51 213334533059126 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.508+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 48 79077905592013 17 7301 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:53.509+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 53 254944616021293 24 7301 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:53.509+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 50 246582249071123 20 7301 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:53.513+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 54 55145461136105 16 7301 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:36:53.513+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 52 142324128597367 22 7301 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:53.514+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 55 219528837936411 25 7301 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:53.517+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 57 80145536651832 17 7301 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:53.517+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 56 194568213794435 29 7301 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:36:53.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 59 106620692996001 23 7301 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:53.522+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 58 25511316571199 16 7301 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:36:53.525+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 61 223881084081370 17 7301 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:53.525+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 60 279733370822238 23 7301 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:53.531+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 63 25657368013413 17 7301 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:53.531+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 62 36300629322478 34 7301 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:53.531+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_system:_query InstanceIds [3643393537124379309] 2023-10-21T04:36:53.531+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 7301 2023-10-21T04:36:53.531+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7301 2023-10-21T04:36:53.531+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:36:53.531+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:36:53.532+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 3643393537124379309 INDEX_STATE_CATCHUP 2023-10-21T04:36:53.532+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 3643393537124379309 true 2023-10-21T04:36:53.532+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 3643393537124379309 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query Session: 7301 BuildTS: [0=23,1=27,2=28,3=24,4=18,5=23,6=23,7=17,8=22,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=30,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=25,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=25,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=34,63=17] 2023-10-21T04:36:53.532+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_system:_query, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} , fetchKVSeq false 2023-10-21T04:36:53.532+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:36:53.532+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:36:53.532+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7301 STREAM_ACTIVE 2023-10-21T04:36:53.532+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 7301 2023-10-21T04:36:53.532+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1:_system:_query 7301 2023-10-21T04:36:53.532+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:53.532+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:53.532+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:53.533+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:53.533+05:30 [Info] send progress stats to clients 2023-10-21T04:36:53.533+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 7301 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:53.533+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 3643393537124379309 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:36:53.533+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:36:53.533+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:36:53.533+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:36:53.533+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 7301 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:53.533+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:36:53.533+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:36:53.533+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:36:53.533+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:36:53.533+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:39442: use of closed network connection 2023-10-21T04:36:53.533+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:39442" exit: read tcp4 127.0.0.1:9121->127.0.0.1:39442: use of closed network connection 2023-10-21T04:36:53.533+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39442" closed ! 2023-10-21T04:36:53.533+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:36:53.534+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:39442" already gone 2023-10-21T04:36:53.534+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:36:53.534+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:36:53.535+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.535+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.535+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.535+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.535+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1:_system:_query 2023-10-21T04:36:53.535+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:36:53.553+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_system:_query]. Err genServer.closed 2023-10-21T04:36:53.553+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:36:53.553+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Treating genServer.closed As Success 2023-10-21T04:36:53.553+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 7301 2023-10-21T04:36:53.553+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1:_system:_query 2023-10-21T04:36:53.553+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:36:53.553+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.553+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.553+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:53.553+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:36:53.553+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:36:53.553+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:36:53.553+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 61685 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:53.553+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0046acf00 }] 2023-10-21T04:36:53.553+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.553+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.553+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:36:53.553+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:36:53.553+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 61686 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:36:53.554+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:53.554+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:53.554+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:36:53.554+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:36:53.554+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:36:53.554+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 61686 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:36:53.554+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0046acf00 } bucket_1:{0xc00b520180 }] 2023-10-21T04:36:53.557+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:36:53.561+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.709875ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:53.569+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:53.569+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:53.573+05:30 [Info] switched currmeta from 214 -> 215 force true 2023-10-21T04:36:54.095+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:54.095+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 2 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc000356540 2023-10-21T04:36:54.095+05:30 [Info] handleUpdateIndexRState: Index instance 3643393537124379309 rstate moved to ACTIVE 2023-10-21T04:36:54.108+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.58542ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:54.108+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[3643393537124379309:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:36:54.109+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12582561312368048736, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:36:54.109+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12582561312368048736, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:36:54.127+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:54.127+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:54.132+05:30 [Info] switched currmeta from 215 -> 216 force true 2023-10-21T04:36:54.138+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4443661069773468873,realInstId: 0, indexDefn DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:36:54.138+05:30 [Info] Indexer::handleRecoverIndex InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:36:54.139+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 316.221µs 2023-10-21T04:36:54.139+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 359.722µs 2023-10-21T04:36:54.139+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4443661069773468873 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:54.139+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4443661069773468873 PartitionId: 0 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:54.140+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4443661069773468873, indexDefn DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:36:54.140+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 31.666561ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:54.144+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:54.151+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : LSSCleaner initialized 2023-10-21T04:36:54.151+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:36:54.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:54.152+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:54.153+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:54.155+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : LSSCleaner initialized for recovery 2023-10-21T04:36:54.155+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:36:54.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery 2023-10-21T04:36:54.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:54.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data], Shared [true] 2023-10-21T04:36:54.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1417216] Data log Head [0] Tail [1556480] 2023-10-21T04:36:54.155+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1417216] 2023-10-21T04:36:54.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:54.158+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1055940 memSz 882 2023-10-21T04:36:54.158+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 21 count: 1000 metaSz: 2511 rp [1] sn: 22 count: 1000 metaSz: 2519 2023-10-21T04:36:54.159+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 204569 recovery UsedSpace 22523 2023-10-21T04:36:54.160+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.160+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:36:54.160+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.160+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:36:54.161+05:30 [Info] switched currmeta from 216 -> 217 force true 2023-10-21T04:36:54.160+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.161+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:36:54.161+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.161+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:36:54.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.163+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:36:54.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.163+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:36:54.163+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.163+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:36:54.166+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.166+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:36:54.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:36:54.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:54.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.173+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.176+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:54.176+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.176+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:54.177+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.177+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:54.177+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.177+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:54.177+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.178+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:54.178+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.178+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:54.179+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.179+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:54.170+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:36:54.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex started 2023-10-21T04:36:54.187+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:54.187+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:54.189+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:54.189+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:36:54.190+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : replayOffset: 1545194 (RP:1545194) (CP:1511424) 2023-10-21T04:36:54.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1545194] took [34.745849ms] 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9253, memSzIndex 76, numRecords 125 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1056174, resident 1, memSz 68004, memSzIndex 887, numRecords 1000 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9573, memSzIndex 76, numRecords 127 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10516, memSzIndex 93, numRecords 136 2023-10-21T04:36:54.190+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5124 2023-10-21T04:36:54.190+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1556480] replayOffset [1545194] 2023-10-21T04:36:54.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [35.738081ms] 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:36:54.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data], Shared [true]. Built [17] plasmas, took [36.493002ms] 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1056174 memSz 2102 2023-10-21T04:36:54.191+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 21 count: 1000 metaSz: 2511 rp [1] sn: 22 count: 1000 metaSz: 2519 2023-10-21T04:36:54.192+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 170312 recovery UsedSpace 24966 2023-10-21T04:36:54.196+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 1949031200982 23 61686 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:54.198+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 245782287380001 27 61686 172.31.5.112:9000. HWT [26-27,27]. 2023-10-21T04:36:54.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:36:54.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:54.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:54.199+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:36:54.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex started 2023-10-21T04:36:54.199+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4443661069773468873 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:36:54.200+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:54.200+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:54.202+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 205201940883916 24 61686 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:54.203+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 93079113162334 18 61686 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:54.205+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 52522903947059 28 61686 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:36:54.205+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 244791485011286 23 61686 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:54.208+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 167580527121081 23 61686 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:54.208+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 177815184127237 17 61686 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:54.210+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 54236768606397 22 61686 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:54.213+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 134409829206048 26 61686 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:54.213+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 160381752747972 18 61686 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:54.216+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 142387448675642 30 61686 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:36:54.216+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 58522600746999 24 61686 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:54.219+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 90115353725987 21 61686 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:54.219+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 161230548567164 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.221+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 123957315902602 30 61686 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:36:54.221+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 85302001459467 21 61686 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:54.224+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 213108110408291 23 61686 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:54.225+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4443661069773468873 PartitionId 0 Warmup took 85.785675ms 2023-10-21T04:36:54.225+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4443661069773468873 partitionId 0 WriterThreads 4 2023-10-21T04:36:54.225+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4443661069773468873 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index Index: 4443661069773468873 Partition: 0 2023-10-21T04:36:54.225+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4443661069773468873, shardIdMap: map[0:[4718895791748289908 15531034489713578274]] err: 2023-10-21T04:36:54.225+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4443661069773468873, partnShardIdMap: map[0:[4718895791748289908 15531034489713578274]], err: 2023-10-21T04:36:54.225+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:54.226+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4443661069773468873 2023-10-21T04:36:54.226+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:36:54.226+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:36:54.226+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:54.226+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:36:54.226+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:54.227+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4443661069773468873 Partitions [0] 2023-10-21T04:36:54.227+05:30 [Info] StorageMgr::openSnapshot IndexInst:4443661069773468873 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:36:54.229+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 150690956328758 26 61686 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:54.229+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 151273673850546 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.231+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 258671969827141 19 61686 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:54.231+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 147946970349859 24 61686 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:54.231+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 163460438181870 34 61686 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:54.234+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 53829314402679 32 61686 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:36:54.238+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 114823206239184 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.238+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 19989312661772 15 61686 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:36:54.240+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 232028101514735 19 61686 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:54.241+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 25510109770064 28 61686 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:36:54.242+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.785663ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:54.242+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 74.896µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:36:54.244+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 8305763409275 21 61686 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:36:54.248+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:54.248+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:54.254+05:30 [Info] switched currmeta from 217 -> 218 force true 2023-10-21T04:36:54.261+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 68847799765163 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.264+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 124811039198764 18 61686 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:54.264+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 100746255343609 19 61686 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:54.264+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 47156288300713 24 61686 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:54.264+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 207489375637709 12 61686 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:36:54.264+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 117298435979365 24 61686 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:54.265+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 203503670520010 24 61686 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:54.265+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 107407811917992 20 61686 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:54.266+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 228857796432725 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.266+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 33310272161722 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.266+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 158357249947806 21 61686 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:54.266+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 104136033125373 29 61686 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:36:54.267+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 236904976023311 22 61686 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:54.269+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 124310544527292 21 61686 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:54.271+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 276463747117997 25 61686 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:36:54.271+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 246959094981553 34 61686 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:54.271+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 196269910725961 26 61686 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:54.275+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 65885584526743 20 61686 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:54.279+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 172365063201867 21 61686 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:54.280+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 246582249071123 20 61686 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:54.280+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 79077905592013 17 61686 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:54.283+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 55145461136105 16 61686 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:36:54.283+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 254944616021293 24 61686 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:54.284+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 212884822297038 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.284+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 213334533059126 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.285+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 219528837936411 25 61686 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:54.285+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 142324128597367 22 61686 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:54.288+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 80145536651832 17 61686 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:54.289+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 194568213794435 29 61686 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:36:54.292+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 25511316571199 16 61686 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:36:54.293+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 106620692996001 23 61686 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:54.300+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 InstanceIds [17434304260397056806 3127033699511712809 8965923595543532942 3643393537124379309] 2023-10-21T04:36:54.300+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 61686 2023-10-21T04:36:54.300+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 61686 2023-10-21T04:36:54.300+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:36:54.300+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:36:54.300+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:36:54.300+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:36:54.300+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 61686 2023-10-21T04:36:54.300+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 61686 STREAM_ACTIVE 2023-10-21T04:36:54.300+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:36:54.302+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 279733370822238 23 61686 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:54.302+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 223881084081370 17 61686 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:54.302+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 36300629322478 34 61686 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:54.302+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 25657368013413 17 61686 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:54.534+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1:_system:_query In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:36:54.570+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3643393537124379309:[0]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:36:54.570+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3643393537124379309:[0]]. New rollbacktime: 1697842989607052637, Old rollbacktime: 0 2023-10-21T04:36:55.148+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4443661069773468873:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:36:55.148+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:36:55.148+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 16337157693634799600, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:36:55.167+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 3580008349810282285,realInstId: 0, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:36:55.171+05:30 [Info] Indexer::handleRecoverIndex InstId: 3580008349810282285 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} 2023-10-21T04:36:55.173+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 1.69454ms 2023-10-21T04:36:55.173+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.173+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 2.117131ms 2023-10-21T04:36:55.173+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 3580008349810282285, indexDefn DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:36:55.176+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 27.550052ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:55.180+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:55.183+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:55.183+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:55.186+05:30 [Info] switched currmeta from 218 -> 219 force true 2023-10-21T04:36:55.193+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 1 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.194+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.194+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.195+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:36:55.196+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:36:55.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:55.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.204+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:55.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex started 2023-10-21T04:36:55.204+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.204+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:55.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.205+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:55.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex started 2023-10-21T04:36:55.206+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:36:55.206+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:36:55.206+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:55.206+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:36:55.206+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:36:55.206+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:55.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex stopped 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.206+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:36:55.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex closed 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:36:55.212+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:55.212+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:36:55.212+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:36:55.212+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:55.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex stopped 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.212+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.218+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:36:55.218+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex closed 2023-10-21T04:36:55.219+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex ... 2023-10-21T04:36:55.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex ... 2023-10-21T04:36:55.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex ... 2023-10-21T04:36:55.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex ... 2023-10-21T04:36:55.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index destroyed 2023-10-21T04:36:55.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.227+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.227+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.229+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.229+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:55.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.232+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:55.232+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:36:55.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex started 2023-10-21T04:36:55.232+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.232+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:55.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.236+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:55.236+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:36:55.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex started 2023-10-21T04:36:55.236+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 1 Warmup took 42.468073ms 2023-10-21T04:36:55.236+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 1 WriterThreads 1 2023-10-21T04:36:55.236+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 1 Stopped Billing WUs 2023-10-21T04:36:55.236+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index Index: 3580008349810282285 Partition: 1 2023-10-21T04:36:55.236+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.237+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 2 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.237+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.237+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:36:55.237+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.237+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.238+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.238+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:36:55.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:55.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.244+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:55.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex started 2023-10-21T04:36:55.244+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.244+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:55.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.246+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:55.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex started 2023-10-21T04:36:55.246+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.246+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.246+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:36:55.246+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.246+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.246+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.247+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:36:55.250+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:55.250+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:36:55.250+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:36:55.250+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:55.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex stopped 2023-10-21T04:36:55.250+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.250+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.251+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.251+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.255+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:36:55.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex closed 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:36:55.256+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:55.256+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:36:55.256+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:36:55.256+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:55.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex stopped 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.256+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.267+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:36:55.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex closed 2023-10-21T04:36:55.267+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex ... 2023-10-21T04:36:55.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex ... 2023-10-21T04:36:55.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex ... 2023-10-21T04:36:55.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex ... 2023-10-21T04:36:55.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index destroyed 2023-10-21T04:36:55.288+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.289+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.289+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.297+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.298+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.298+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.300+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:36:55.300+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:36:55.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:55.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.301+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:55.301+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:36:55.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex started 2023-10-21T04:36:55.301+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.302+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:55.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.306+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:55.306+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:36:55.306+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex started 2023-10-21T04:36:55.306+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 2 Warmup took 69.718368ms 2023-10-21T04:36:55.307+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 2 WriterThreads 1 2023-10-21T04:36:55.307+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 2 Stopped Billing WUs 2023-10-21T04:36:55.307+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index Index: 3580008349810282285 Partition: 2 2023-10-21T04:36:55.307+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.307+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 3 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.307+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.307+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.307+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:36:55.307+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.307+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.308+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.308+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:36:55.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:55.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.314+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:55.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex started 2023-10-21T04:36:55.314+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.314+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:55.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.315+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:55.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex started 2023-10-21T04:36:55.315+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:36:55.315+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:36:55.315+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:55.315+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:36:55.315+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:36:55.315+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:55.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex stopped 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.315+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.322+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:36:55.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex closed 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:36:55.323+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:55.323+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:36:55.323+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:36:55.323+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:55.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex stopped 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.323+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.330+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:36:55.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex closed 2023-10-21T04:36:55.331+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex ... 2023-10-21T04:36:55.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex ... 2023-10-21T04:36:55.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex ... 2023-10-21T04:36:55.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex ... 2023-10-21T04:36:55.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index destroyed 2023-10-21T04:36:55.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.339+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.339+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.342+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.342+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.342+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:55.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.347+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:55.347+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:36:55.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex started 2023-10-21T04:36:55.347+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.347+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:55.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.349+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:55.349+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:36:55.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex started 2023-10-21T04:36:55.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.350+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.350+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 3 Warmup took 42.730174ms 2023-10-21T04:36:55.350+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 3 WriterThreads 1 2023-10-21T04:36:55.350+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 3 Stopped Billing WUs 2023-10-21T04:36:55.350+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index Index: 3580008349810282285 Partition: 3 2023-10-21T04:36:55.350+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.350+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 4 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.350+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.351+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:36:55.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.351+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.351+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:36:55.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:55.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.362+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:55.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex started 2023-10-21T04:36:55.362+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.362+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.365+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:36:55.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:55.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.365+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:55.365+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex started 2023-10-21T04:36:55.365+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:36:55.365+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.365+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.365+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.365+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:36:55.366+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:55.366+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:36:55.366+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:36:55.366+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:55.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex stopped 2023-10-21T04:36:55.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.376+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:36:55.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex closed 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:36:55.377+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:55.377+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:36:55.377+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:36:55.377+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:55.377+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex stopped 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.377+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.378+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:55.378+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:55.381+05:30 [Info] switched currmeta from 219 -> 219 force true 2023-10-21T04:36:55.383+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:36:55.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex closed 2023-10-21T04:36:55.383+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex ... 2023-10-21T04:36:55.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex ... 2023-10-21T04:36:55.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex ... 2023-10-21T04:36:55.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex ... 2023-10-21T04:36:55.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index destroyed 2023-10-21T04:36:55.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.404+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.404+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.408+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.408+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:55.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.420+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:55.420+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:36:55.420+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex started 2023-10-21T04:36:55.420+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.420+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.423+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:55.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.424+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:55.424+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:36:55.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex started 2023-10-21T04:36:55.424+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 4 Warmup took 73.545666ms 2023-10-21T04:36:55.424+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.424+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.424+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 4 WriterThreads 1 2023-10-21T04:36:55.424+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 4 Stopped Billing WUs 2023-10-21T04:36:55.424+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index Index: 3580008349810282285 Partition: 4 2023-10-21T04:36:55.424+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.425+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 5 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.426+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.426+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:36:55.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.427+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.427+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:36:55.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:55.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.438+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:55.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex started 2023-10-21T04:36:55.438+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.438+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:55.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.441+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:55.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex started 2023-10-21T04:36:55.441+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:36:55.441+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.441+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.441+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.441+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.441+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:36:55.441+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.441+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:55.442+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:36:55.442+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:36:55.442+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:55.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex stopped 2023-10-21T04:36:55.442+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.442+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.442+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.442+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:36:55.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex closed 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:36:55.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:55.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:36:55.450+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:36:55.450+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:55.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex stopped 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.450+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.458+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:36:55.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex closed 2023-10-21T04:36:55.458+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex ... 2023-10-21T04:36:55.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex ... 2023-10-21T04:36:55.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex ... 2023-10-21T04:36:55.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex ... 2023-10-21T04:36:55.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index destroyed 2023-10-21T04:36:55.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.472+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.473+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.477+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.477+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:55.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.480+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:55.480+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:36:55.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex started 2023-10-21T04:36:55.480+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.480+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:55.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.485+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:55.485+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:36:55.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex started 2023-10-21T04:36:55.485+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 5 Warmup took 60.429007ms 2023-10-21T04:36:55.486+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.486+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.486+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 5 WriterThreads 1 2023-10-21T04:36:55.486+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 5 Stopped Billing WUs 2023-10-21T04:36:55.486+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index Index: 3580008349810282285 Partition: 5 2023-10-21T04:36:55.486+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.486+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 6 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.486+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.486+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:36:55.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.487+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.487+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:36:55.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:55.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.491+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:55.491+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex started 2023-10-21T04:36:55.492+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.492+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:55.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.494+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:55.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex started 2023-10-21T04:36:55.494+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.495+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:36:55.495+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:36:55.495+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:55.495+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:36:55.495+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:36:55.495+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:55.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex stopped 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.495+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.500+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:36:55.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex closed 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:36:55.501+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:55.501+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:36:55.501+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:36:55.501+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:55.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex stopped 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.501+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.507+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:36:55.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex closed 2023-10-21T04:36:55.507+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex ... 2023-10-21T04:36:55.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex ... 2023-10-21T04:36:55.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex ... 2023-10-21T04:36:55.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex ... 2023-10-21T04:36:55.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index destroyed 2023-10-21T04:36:55.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.519+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.519+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.521+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.521+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.521+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:55.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.529+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:55.529+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:36:55.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex started 2023-10-21T04:36:55.529+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.529+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:55.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.531+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:55.531+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:36:55.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex started 2023-10-21T04:36:55.531+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.531+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.531+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 6 Warmup took 45.004088ms 2023-10-21T04:36:55.531+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 6 WriterThreads 1 2023-10-21T04:36:55.531+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 6 Stopped Billing WUs 2023-10-21T04:36:55.531+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index Index: 3580008349810282285 Partition: 6 2023-10-21T04:36:55.531+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.531+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 7 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.532+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.532+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:36:55.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.532+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.532+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:36:55.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.543+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex started 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.543+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex started 2023-10-21T04:36:55.543+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:36:55.543+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:36:55.543+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:55.543+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:36:55.543+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:36:55.543+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:55.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex stopped 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.543+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:36:55.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex closed 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:36:55.555+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:55.555+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:36:55.555+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:36:55.555+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:55.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex stopped 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.555+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.569+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:36:55.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex closed 2023-10-21T04:36:55.576+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex ... 2023-10-21T04:36:55.577+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex ... 2023-10-21T04:36:55.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex ... 2023-10-21T04:36:55.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex ... 2023-10-21T04:36:55.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index destroyed 2023-10-21T04:36:55.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.595+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.595+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.601+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.601+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:55.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.611+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:55.611+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:36:55.611+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex started 2023-10-21T04:36:55.611+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.611+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:55.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.614+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:55.614+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:36:55.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex started 2023-10-21T04:36:55.614+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 7 Warmup took 82.420046ms 2023-10-21T04:36:55.614+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.614+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.614+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 7 WriterThreads 1 2023-10-21T04:36:55.614+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 7 Stopped Billing WUs 2023-10-21T04:36:55.614+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index Index: 3580008349810282285 Partition: 7 2023-10-21T04:36:55.614+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:55.614+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 3580008349810282285 PartitionId: 8 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:55.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.615+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.615+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:36:55.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.615+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:36:55.615+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:36:55.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:55.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.624+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:55.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex started 2023-10-21T04:36:55.624+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.624+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:55.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.625+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:55.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex started 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.625+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:36:55.625+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:36:55.625+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:55.625+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:36:55.625+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:36:55.625+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:55.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex stopped 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:36:55.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex closed 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:36:55.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:36:55.632+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:55.632+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:36:55.632+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:36:55.632+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:55.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex stopped 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:36:55.632+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:36:55.636+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:36:55.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:36:55.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex closed 2023-10-21T04:36:55.636+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:36:55.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex ... 2023-10-21T04:36:55.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex ... 2023-10-21T04:36:55.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex sucessfully destroyed 2023-10-21T04:36:55.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex ... 2023-10-21T04:36:55.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex ... 2023-10-21T04:36:55.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index destroyed 2023-10-21T04:36:55.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:55.645+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.645+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:36:55.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:55.648+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:36:55.648+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:36:55.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:55.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:55.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:55.652+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:55.652+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:36:55.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex started 2023-10-21T04:36:55.652+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.652+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:55.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:55.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:55.655+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:55.655+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:36:55.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex started 2023-10-21T04:36:55.655+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 8 Warmup took 40.150552ms 2023-10-21T04:36:55.655+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:55.655+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:55.655+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 8 WriterThreads 1 2023-10-21T04:36:55.655+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 3580008349810282285 PartitionId 8 Stopped Billing WUs 2023-10-21T04:36:55.655+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index Index: 3580008349810282285 Partition: 8 2023-10-21T04:36:55.655+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 3580008349810282285, shardIdMap: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] err: 2023-10-21T04:36:55.655+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 3580008349810282285, partnShardIdMap: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]], err: 2023-10-21T04:36:55.655+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3580008349810282285 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:55.656+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3580008349810282285 2023-10-21T04:36:55.656+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3580008349810282285 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:55.656+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:36:55.656+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:55.656+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3580008349810282285 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:55.656+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 3580008349810282285 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:55.656+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3580008349810282285 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:36:55.657+05:30 [Info] StorageMgr::openSnapshot IndexInst:3580008349810282285 Partition:6 No Snapshot Found. 2023-10-21T04:36:55.657+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3580008349810282285 Adding Nil Snapshot. 2023-10-21T04:36:55.668+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.268443ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:55.668+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 118.848µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:36:55.686+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:55.686+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:55.689+05:30 [Info] switched currmeta from 219 -> 220 force true 2023-10-21T04:36:55.938+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_default:c1:idx_secondary Progress: 100 InstId: 4443661069773468873 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:36:55.938+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_1:_system:_query:#primary Progress: 100 InstId: 3643393537124379309 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:36:55.938+05:30 [Info] ShardRebalancer::computeProgress 0.7022499999999999 2023-10-21T04:36:56.184+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[3580008349810282285:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:36:56.184+05:30 [Info] Indexer::handleMergePartition Source 14010044466748520509 Target 3580008349810282285 2023-10-21T04:36:56.184+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 3580008349810282285 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00d6bbce0 2023-10-21T04:36:56.185+05:30 [Info] MergePartition: sent async request to update index instance 3580008349810282285 rstate moved to ACTIVE 2023-10-21T04:36:56.190+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.101571ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:56.190+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 14010044466748520509, realInst: 3580008349810282285 2023-10-21T04:36:56.195+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:36:56.196+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 16961575949607441659, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:36:56.198+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:56.198+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:56.205+05:30 [Info] switched currmeta from 220 -> 221 force true 2023-10-21T04:36:56.228+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7516828288031590951,realInstId: 0, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:36:56.229+05:30 [Info] Indexer::handleRecoverIndex InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} 2023-10-21T04:36:56.229+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 323.871µs 2023-10-21T04:36:56.229+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 371.513µs 2023-10-21T04:36:56.229+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.229+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 7 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.229+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7516828288031590951, indexDefn DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:36:56.230+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 34.315436ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:56.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.230+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 272 2023-10-21T04:36:56.230+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 21 count: 136 metaSz: 2510 rp [1] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:36:56.230+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 104140 recovery UsedSpace 145546 2023-10-21T04:36:56.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.230+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:36:56.230+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 21 count: 136 metaSz: 2510 rp [1] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:36:56.244+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 125675 recovery UsedSpace 144968 2023-10-21T04:36:56.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:36:56.238+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:56.241+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:56.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.247+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:56.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:36:56.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex started 2023-10-21T04:36:56.252+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.253+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.253+05:30 [Info] switched currmeta from 221 -> 222 force true 2023-10-21T04:36:56.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:36:56.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.255+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:36:56.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex started 2023-10-21T04:36:56.255+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:36:56.255+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.255+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.275+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 7 Warmup took 45.5031ms 2023-10-21T04:36:56.275+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 7 WriterThreads 1 2023-10-21T04:36:56.275+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 7 Stopped Billing WUs 2023-10-21T04:36:56.275+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index Index: 7516828288031590951 Partition: 7 2023-10-21T04:36:56.275+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.276+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 8 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.276+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:36:56.276+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 21 count: 106 metaSz: 2508 rp [1] sn: 720006 count: 106 metaSz: 2508 2023-10-21T04:36:56.276+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 96800 recovery UsedSpace 149064 2023-10-21T04:36:56.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.277+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:36:56.277+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 21 count: 106 metaSz: 2508 rp [1] sn: 720006 count: 106 metaSz: 2508 2023-10-21T04:36:56.277+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 121250 recovery UsedSpace 149064 2023-10-21T04:36:56.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:36:56.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.281+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:36:56.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex started 2023-10-21T04:36:56.281+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.281+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:36:56.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.282+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.282+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:36:56.282+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex started 2023-10-21T04:36:56.282+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.282+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.282+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:36:56.299+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 8 Warmup took 23.56553ms 2023-10-21T04:36:56.300+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 8 WriterThreads 1 2023-10-21T04:36:56.300+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 8 Stopped Billing WUs 2023-10-21T04:36:56.300+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index Index: 7516828288031590951 Partition: 8 2023-10-21T04:36:56.300+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.300+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 1 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.300+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:36:56.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.301+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:36:56.301+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 19 count: 127 metaSz: 2508 rp [1] sn: 20 count: 127 metaSz: 2510 rp [2] sn: 21 count: 127 metaSz: 2510 rp [3] sn: 720006 count: 127 metaSz: 2510 2023-10-21T04:36:56.301+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 139435 recovery UsedSpace 133214 2023-10-21T04:36:56.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.301+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:36:56.302+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 21 count: 127 metaSz: 2510 rp [1] sn: 720006 count: 127 metaSz: 2510 2023-10-21T04:36:56.302+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 124266 recovery UsedSpace 144968 2023-10-21T04:36:56.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:36:56.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.316+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:36:56.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex started 2023-10-21T04:36:56.316+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.316+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:36:56.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.317+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:36:56.317+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex started 2023-10-21T04:36:56.318+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:36:56.318+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.320+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.345+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 1 Warmup took 45.193927ms 2023-10-21T04:36:56.346+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 1 WriterThreads 1 2023-10-21T04:36:56.346+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 1 Stopped Billing WUs 2023-10-21T04:36:56.346+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index Index: 7516828288031590951 Partition: 1 2023-10-21T04:36:56.346+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.347+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 2 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.347+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.347+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:36:56.347+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 20 count: 136 metaSz: 2508 rp [1] sn: 21 count: 136 metaSz: 2510 rp [2] sn: 22 count: 136 metaSz: 2510 rp [3] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:36:56.347+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 141100 recovery UsedSpace 133222 2023-10-21T04:36:56.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.348+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:36:56.348+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 22 count: 136 metaSz: 2510 rp [1] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:36:56.348+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 125916 recovery UsedSpace 144968 2023-10-21T04:36:56.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:36:56.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.364+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:36:56.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex started 2023-10-21T04:36:56.364+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.364+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:36:56.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.366+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:36:56.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex started 2023-10-21T04:36:56.366+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:36:56.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.366+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.389+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 2 Warmup took 41.952673ms 2023-10-21T04:36:56.390+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 2 WriterThreads 1 2023-10-21T04:36:56.390+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 2 Stopped Billing WUs 2023-10-21T04:36:56.390+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index Index: 7516828288031590951 Partition: 2 2023-10-21T04:36:56.390+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.391+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 3 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.391+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.391+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:36:56.391+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 18 count: 125 metaSz: 2508 rp [1] sn: 19 count: 125 metaSz: 2510 rp [2] sn: 20 count: 125 metaSz: 2510 rp [3] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:36:56.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.392+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:36:56.392+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 16 numRPs: 3 rp [0] sn: 19 count: 125 metaSz: 2510 rp [1] sn: 20 count: 125 metaSz: 2510 rp [2] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:36:56.392+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 138435 recovery UsedSpace 133032 2023-10-21T04:36:56.392+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 124275 recovery UsedSpace 136754 2023-10-21T04:36:56.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:36:56.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.397+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:36:56.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:36:56.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex started 2023-10-21T04:36:56.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.397+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:36:56.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex started 2023-10-21T04:36:56.397+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.397+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.398+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.399+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.399+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:36:56.415+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 3 Warmup took 23.934311ms 2023-10-21T04:36:56.415+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 3 WriterThreads 1 2023-10-21T04:36:56.415+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 3 Stopped Billing WUs 2023-10-21T04:36:56.415+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index Index: 7516828288031590951 Partition: 3 2023-10-21T04:36:56.415+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.415+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 4 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.416+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 256 2023-10-21T04:36:56.417+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 17 count: 125 metaSz: 2508 rp [1] sn: 18 count: 125 metaSz: 2510 rp [2] sn: 19 count: 125 metaSz: 2510 rp [3] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:36:56.417+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 137750 recovery UsedSpace 133026 2023-10-21T04:36:56.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.417+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:36:56.417+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 16 numRPs: 3 rp [0] sn: 18 count: 125 metaSz: 2510 rp [1] sn: 19 count: 125 metaSz: 2510 rp [2] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:36:56.418+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 124016 recovery UsedSpace 136754 2023-10-21T04:36:56.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:36:56.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.426+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:36:56.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex started 2023-10-21T04:36:56.426+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.427+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:36:56.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.427+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:36:56.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex started 2023-10-21T04:36:56.427+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:36:56.427+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.427+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.429+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 14] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:56.429+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. taskList: &{Rev:[0 0 0 0 0 0 0 14] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/f3a5b85675e4f53a445d2e873c593537 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/f3a5b85675e4f53a445d2e873c593537 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7022499999999999 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]}]} 2023-10-21T04:36:56.429+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 14] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:true Messages:[]} 2023-10-21T04:36:56.429+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. taskList: &{Rev:[0 0 0 0 0 0 0 14] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/f3a5b85675e4f53a445d2e873c593537 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/f3a5b85675e4f53a445d2e873c593537 Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7022499999999999 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:f3a5b85675e4f53a445d2e873c593537]}]} 2023-10-21T04:36:56.432+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 14] 2023-10-21T04:36:56.432+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 14] 2023-10-21T04:36:56.441+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 4 Warmup took 25.69818ms 2023-10-21T04:36:56.442+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 4 WriterThreads 1 2023-10-21T04:36:56.442+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 4 Stopped Billing WUs 2023-10-21T04:36:56.442+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index Index: 7516828288031590951 Partition: 4 2023-10-21T04:36:56.442+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.442+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 5 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.442+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:36:56.442+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 20 count: 123 metaSz: 2508 rp [1] sn: 21 count: 123 metaSz: 2509 rp [2] sn: 22 count: 123 metaSz: 2510 rp [3] sn: 720006 count: 123 metaSz: 2510 2023-10-21T04:36:56.442+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 135450 recovery UsedSpace 132636 2023-10-21T04:36:56.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.443+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:36:56.443+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 22 count: 123 metaSz: 2510 rp [1] sn: 720006 count: 123 metaSz: 2510 2023-10-21T04:36:56.443+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 135236 recovery UsedSpace 144968 2023-10-21T04:36:56.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:36:56.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.447+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:36:56.447+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex started 2023-10-21T04:36:56.447+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.447+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:36:56.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.449+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:36:56.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex started 2023-10-21T04:36:56.449+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:36:56.449+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.449+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.459+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 5 Warmup took 17.220039ms 2023-10-21T04:36:56.459+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 5 WriterThreads 1 2023-10-21T04:36:56.459+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 5 Stopped Billing WUs 2023-10-21T04:36:56.459+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index Index: 7516828288031590951 Partition: 5 2023-10-21T04:36:56.459+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] 2023-10-21T04:36:56.460+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7516828288031590951 PartitionId: 6 using shardIds: [4718895791748289908 15531034489713578274], rebalance running: true 2023-10-21T04:36:56.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:36:56.460+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:36:56.460+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 15 numRPs: 4 rp [0] sn: 19 count: 122 metaSz: 2508 rp [1] sn: 20 count: 122 metaSz: 2509 rp [2] sn: 21 count: 122 metaSz: 2510 rp [3] sn: 720006 count: 122 metaSz: 2510 2023-10-21T04:36:56.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:36:56.460+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:36:56.460+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 17 numRPs: 2 rp [0] sn: 21 count: 122 metaSz: 2510 rp [1] sn: 720006 count: 122 metaSz: 2510 2023-10-21T04:36:56.460+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 145638 recovery UsedSpace 132636 2023-10-21T04:36:56.460+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 134963 recovery UsedSpace 144968 2023-10-21T04:36:56.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:36:56.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:36:56.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:36:56.467+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:36:56.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex started 2023-10-21T04:36:56.468+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:36:56.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:36:56.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:36:56.468+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.468+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:36:56.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex started 2023-10-21T04:36:56.468+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:36:56.468+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:36:56.468+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:36:56.506+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 6 Warmup took 46.025508ms 2023-10-21T04:36:56.506+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 6 WriterThreads 1 2023-10-21T04:36:56.506+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 7516828288031590951 PartitionId 6 Stopped Billing WUs 2023-10-21T04:36:56.506+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index Index: 7516828288031590951 Partition: 6 2023-10-21T04:36:56.507+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7516828288031590951, shardIdMap: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]] err: 2023-10-21T04:36:56.507+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7516828288031590951, partnShardIdMap: map[1:[4718895791748289908 15531034489713578274] 2:[4718895791748289908 15531034489713578274] 3:[4718895791748289908 15531034489713578274] 4:[4718895791748289908 15531034489713578274] 5:[4718895791748289908 15531034489713578274] 6:[4718895791748289908 15531034489713578274] 7:[4718895791748289908 15531034489713578274] 8:[4718895791748289908 15531034489713578274]], err: 2023-10-21T04:36:56.507+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:56.508+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7516828288031590951 2023-10-21T04:36:56.508+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:56.509+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:36:56.509+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:56.509+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:56.510+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:56.511+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7516828288031590951 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:36:56.511+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:5 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:36:56.512+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:6 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:36:56.513+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:7 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:36:56.514+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:8 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:36:56.515+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:1 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:36:56.516+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:2 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:36:56.517+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:3 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:36:56.518+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:4 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:36:56.519+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.680348ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:56.519+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 214.275µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:36:56.544+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:56.544+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:56.548+05:30 [Info] switched currmeta from 222 -> 223 force true 2023-10-21T04:36:57.252+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4443661069773468873:INDEX_STATE_RECOVERED 7516828288031590951:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:36:57.252+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [12582561312368048736 16961575949607441659] belonging to collectionId: c, ttid: ShardToken28:ac:25:bb:38:c:fb:af. Initiating build 2023-10-21T04:36:57.268+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [4443661069773468873 7516828288031590951] &{2} 2023-10-21T04:36:57.268+05:30 [Info] Indexer::handleBuildRecoveredIndexes [4443661069773468873 7516828288031590951] 2023-10-21T04:36:57.268+05:30 [Info] bulkUpdateRState: Index instance 4443661069773468873 rstate moved to PENDING 2023-10-21T04:36:57.268+05:30 [Info] bulkUpdateRState: Index instance 7516828288031590951 rstate moved to PENDING 2023-10-21T04:36:57.268+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [4443661069773468873 7516828288031590951] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:36:57.269+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:57.269+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:57.269+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:57.271+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:57.271+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:57.276+05:30 [Info] switched currmeta from 223 -> 225 force true 2023-10-21T04:36:57.285+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:57.286+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:57.290+05:30 [Info] switched currmeta from 225 -> 225 force true 2023-10-21T04:36:57.301+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:36:57.301+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:57.301+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:57.302+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66598 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:57.302+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:36:57.302+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:36:57.302+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:36:57.302+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66598 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:57.302+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:36:57.302+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:57.302+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:57.302+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:36:57.302+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:36:57.302+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId 66599 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:36:57.303+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:57.303+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:57.303+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:36:57.303+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:1] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:36:57.303+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4443661069773468873 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:36:57.303+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:1] INIT_STREAM:map[bucket_1:_default:c1:2]] 2023-10-21T04:36:57.303+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 7516828288031590951 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:36:57.303+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:36:57.303+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=27,2=28,3=24,4=18,5=23,6=23,7=17,8=22,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=30,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=25,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=25,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=34,63=17] IndexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0}] Async: true SessionId: 66599 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:36:57.304+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:36:57.304+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:36:57.304+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:57.305+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 36.440955ms 2023-10-21T04:36:57.305+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 36.473564ms 2023-10-21T04:36:57.305+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [4443661069773468873 7516828288031590951] 2023-10-21T04:36:57.305+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 52.167598ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:57.305+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4443661069773468873:true 7516828288031590951:true] 2023-10-21T04:36:57.310+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:36:57.312+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.310656ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:57.313+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:36:57.329+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.619943ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:57.338+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39642" doAuth() ... 2023-10-21T04:36:57.338+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39642" auth successful 2023-10-21T04:36:57.339+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:39642" +1 2023-10-21T04:36:57.342+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:57.344+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:57.344+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:57.356+05:30 [Info] switched currmeta from 225 -> 227 force true 2023-10-21T04:36:57.366+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:57.366+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:57.379+05:30 [Info] switched currmeta from 227 -> 227 force true 2023-10-21T04:36:57.663+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 1949031200982 23 66599 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:57.665+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 245782287380001 27 66599 172.31.5.112:9000. HWT [26-27,27]. 2023-10-21T04:36:57.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 205201940883916 24 66599 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:57.672+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 52522903947059 28 66599 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:36:57.675+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 93079113162334 18 66599 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:57.677+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 244791485011286 23 66599 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:57.679+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 177815184127237 17 66599 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:57.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 167580527121081 23 66599 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:57.682+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 54236768606397 22 66599 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:57.685+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 134409829206048 26 66599 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:57.688+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 160381752747972 18 66599 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:57.689+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 58522600746999 24 66599 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:57.693+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 142387448675642 30 66599 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:36:57.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 90115353725987 21 66599 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:57.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 123957315902602 30 66599 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:36:57.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 85302001459467 21 66599 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:57.699+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 161230548567164 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.701+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 163460438181870 34 66599 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:57.701+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 213108110408291 23 66599 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:57.704+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 150690956328758 26 66599 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:57.706+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 151273673850546 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.713+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 147946970349859 24 66599 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:57.719+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 53829314402679 32 66599 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:36:57.719+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 258671969827141 19 66599 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:57.724+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 114823206239184 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.724+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 19989312661772 15 66599 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:36:57.735+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 25510109770064 28 66599 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:36:57.735+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 68847799765163 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.735+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 232028101514735 19 66599 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:57.738+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 207489375637709 12 66599 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:36:57.738+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 124811039198764 18 66599 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:36:57.738+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 8305763409275 21 66599 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:36:57.741+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 33310272161722 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.741+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 107407811917992 20 66599 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:57.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 117298435979365 24 66599 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:57.745+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 100746255343609 19 66599 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:36:57.746+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 228857796432725 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.750+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 203503670520010 24 66599 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:57.751+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 158357249947806 21 66599 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:57.751+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 47156288300713 24 66599 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:57.753+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 104136033125373 29 66599 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:36:57.755+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 236904976023311 22 66599 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:57.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 196269910725961 26 66599 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:36:57.758+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 124310544527292 21 66599 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:57.760+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 246959094981553 34 66599 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:57.762+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 276463747117997 25 66599 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:36:57.764+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 65885584526743 20 66599 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:57.766+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 172365063201867 21 66599 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:36:57.769+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 212884822297038 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.770+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 79077905592013 17 66599 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:57.771+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 246582249071123 20 66599 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:36:57.773+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 213334533059126 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.774+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 142324128597367 22 66599 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:36:57.777+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 254944616021293 24 66599 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:36:57.778+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 55145461136105 16 66599 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:36:57.780+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 219528837936411 25 66599 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:36:57.782+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 194568213794435 29 66599 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:36:57.785+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 80145536651832 17 66599 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:57.788+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 106620692996001 23 66599 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:57.788+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 25511316571199 16 66599 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:36:57.789+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 279733370822238 23 66599 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:36:57.792+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 36300629322478 34 66599 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:36:57.792+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 223881084081370 17 66599 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:57.795+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 25657368013413 17 66599 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:36:57.796+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 InstanceIds [4443661069773468873 7516828288031590951] 2023-10-21T04:36:57.796+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 66599 2023-10-21T04:36:57.796+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66599 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 4443661069773468873 INDEX_STATE_CATCHUP 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 7516828288031590951 INDEX_STATE_CATCHUP 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 4443661069773468873 true 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 7516828288031590951 true 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 4443661069773468873 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 66599 BuildTS: [0=23,1=27,2=28,3=24,4=18,5=23,6=23,7=17,8=22,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=30,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=21,30=18,31=12,32=25,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=26,44=34,45=25,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=34,63=17] 2023-10-21T04:36:57.796+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} , fetchKVSeq false 2023-10-21T04:36:57.797+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:36:57.797+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:36:57.797+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:36:57.797+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:36:57.797+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66599 STREAM_ACTIVE 2023-10-21T04:36:57.797+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 66599 2023-10-21T04:36:57.797+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 66599 2023-10-21T04:36:57.797+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:57.797+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:57.797+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:57.798+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:57.798+05:30 [Info] send progress stats to clients 2023-10-21T04:36:57.798+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:36:57.798+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:36:57.798+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:2] INIT_STREAM:map[bucket_1:_default:c1:2]] 2023-10-21T04:36:57.798+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:5 bucket_1:3] INIT_STREAM:map[bucket_1:_default:c1:2]] 2023-10-21T04:36:57.798+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0}] Async: false SessionId: 66599 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:36:57.799+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:57.802+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:36:57.808+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 2023-10-21T04:36:57.808+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66599 2023-10-21T04:36:57.808+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66599 2023-10-21T04:36:57.808+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:36:57.809+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:36:57.809+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 7516828288031590951 false 2023-10-21T04:36:57.809+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 4443661069773468873 false 2023-10-21T04:36:57.809+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 4443661069773468873 2023-10-21T04:36:57.809+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 7516828288031590951 2023-10-21T04:36:57.809+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:36:57.809+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 66599 2023-10-21T04:36:57.810+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.343527ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:57.816+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.270625ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:57.831+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:57.831+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:57.844+05:30 [Info] switched currmeta from 227 -> 229 force true 2023-10-21T04:36:57.851+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:57.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:57.853+05:30 [Info] switched currmeta from 229 -> 229 force true 2023-10-21T04:36:58.346+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:58.798+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:36:58.798+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:36:58.798+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 4443661069773468873 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 66599, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:36:58.798+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 4443661069773468873 INDEX_STATE_ACTIVE 2023-10-21T04:36:58.798+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 7516828288031590951 INDEX_STATE_ACTIVE 2023-10-21T04:36:58.798+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:36:58.798+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:36:58.798+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:36:58.798+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:36:58.798+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 66599 2023-10-21T04:36:58.798+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:58.799+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697842989598078587 bucket_1:1697842989607052637] 2023-10-21T04:36:58.799+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 2 ReplicaId: 0 2023-10-21T04:36:58.799+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66599 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:58.799+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 7516828288031590951 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:36:58.799+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4443661069773468873 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:36:58.799+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:36:58.799+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:36:58.799+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:36:58.799+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 66599 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:36:58.799+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:36:58.799+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:36:58.799+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:36:58.800+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:36:58.800+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:39642: use of closed network connection 2023-10-21T04:36:58.800+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:39642" exit: read tcp4 127.0.0.1:9121->127.0.0.1:39642: use of closed network connection 2023-10-21T04:36:58.800+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39642" closed ! 2023-10-21T04:36:58.800+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:36:58.800+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:39642" already gone 2023-10-21T04:36:58.800+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:58.800+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:58.801+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:36:58.801+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:36:58.801+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:36:58.801+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:36:58.803+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 66599 2023-10-21T04:36:58.804+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:36:58.818+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.678937ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:58.834+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.327733ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:58.843+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:36:58.843+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:36:58.843+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:36:58.844+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66599 2023-10-21T04:36:58.844+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 66599 2023-10-21T04:36:58.844+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:36:58.905+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:58.906+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:58.910+05:30 [Info] switched currmeta from 229 -> 231 force true 2023-10-21T04:36:58.925+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:58.925+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:58.929+05:30 [Info] switched currmeta from 231 -> 231 force true 2023-10-21T04:36:59.351+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 2023-10-21T04:36:59.351+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc004ed4d80 2023-10-21T04:36:59.352+05:30 [Info] handleUpdateIndexRState: Index instance 4443661069773468873 rstate moved to ACTIVE 2023-10-21T04:36:59.356+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 4.687225ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:59.356+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[4443661069773468873:true 7516828288031590951:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:36:59.357+05:30 [Info] Indexer::handleMergePartition Source 6643622002716756177 Target 7516828288031590951 2023-10-21T04:36:59.357+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 2 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] []} 2:{2 2 [:9123] []} 3:{3 2 [:9123] []} 4:{4 2 [:9123] []} 5:{5 2 [:9123] []} 6:{6 2 [:9123] []} 7:{7 2 [:9123] []} 8:{8 2 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc004ec2c00 2023-10-21T04:36:59.357+05:30 [Info] MergePartition: sent async request to update index instance 7516828288031590951 rstate moved to ACTIVE 2023-10-21T04:36:59.367+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:36:59.367+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:36:59.373+05:30 [Info] switched currmeta from 231 -> 232 force true 2023-10-21T04:36:59.375+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.556121ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:36:59.376+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 6643622002716756177, realInst: 7516828288031590951 2023-10-21T04:36:59.377+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken28:ac:25:bb:38:c:fb:af, elapsed(sec): 7.419375779 2023-10-21T04:36:59.386+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken28:ac:25:bb:38:c:fb:af 2023-10-21T04:36:59.387+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken28:ac:25:bb:38:c:fb:af MasterId: 18194302e3f10df59bd076e1c4142d0b SourceId: 8806f5706747af4e635b8da5fcd8aeee (127.0.0.1:9004) DestId: 18194302e3f10df59bd076e1c4142d0b (127.0.0.1:9003) RebalId: f3a5b85675e4f53a445d2e873c593537 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Copy SiblingTokenId: Shards: [4718895791748289908 15531034489713578274] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4443661069773468873 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 14010044466748520509 RealInstId: 3580008349810282285 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 6643622002716756177 RealInstId: 7516828288031590951 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [2 2 2 2 2 2 2 2] InstId: 3643393537124379309 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [2] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:36:59.389+05:30 [Info] TestCode::TestActionAtTag: currTag: 7, actionAtTag: 7, actionOnNode: 127.0.0.1:9002, currNode: 127.0.0.1:9003, action: 1 panic: TestCode::TestActionAtTag - Inducing artificial panic at tag: 7 as wished goroutine 17638 [running]: github.com/couchbase/indexing/secondary/testcode.TestActionAtTag(0xc00d93b680?, 0x7) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/testcode/test_action_ci.go:45 +0x5e5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferTokenAsMaster(0xc00d93b680, {0xc00a759614, 0x20}, 0xc00af1e000) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:847 +0x5f8 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTransferToken(0xc00d93b680, {0xc00a759614, 0x20}, 0xc00af1e000) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:724 +0x2a5 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).processShardTokens(0xc00d93b680, {{0xc00a759600, 0x34}, {0xc008453000, 0x2673, 0x2673}, {0x176e520, 0xc00a67e450}, 0x0}) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:644 +0x134 github.com/couchbase/cbauth/metakv.RunObserveChildren.func1({{0xc00a759600, 0x34}, {0xc008453000, 0x2673, 0x2673}, {0xc00a644800, 0x77, 0x78}, 0x0}) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:400 +0x149 github.com/couchbase/cbauth/metakv.doRunObserveChildren(0x1a8c321?, {0x1aa523a, 0x14}, 0xc0046cdf00, 0xc00d39ef60) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:316 +0x4eb github.com/couchbase/cbauth/metakv.(*store).runObserveChildren(...) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:268 github.com/couchbase/cbauth/metakv.RunObserveChildren({0x1aa523a?, 0xc00b5e0004?}, 0x1aee743?, 0x2e?) /opt/build/goproj/src/github.com/couchbase/cbauth/metakv/metakv.go:398 +0x58 github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).observeRebalance(0xc00d93b680) /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:235 +0x111 created by github.com/couchbase/indexing/secondary/indexer.(*ShardRebalancer).doRebalance /opt/build/goproj/src/github.com/couchbase/indexing/secondary/indexer/shard_rebalancer.go:2689 +0x14a 2023-10-21T04:37:00.123+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=18194302e3f10df59bd076e1c4142d0b -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:37:00.123+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:37:00.124+05:30 [Info] Setting ipv6=false 2023-10-21T04:37:00.139+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:37:00.161+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:37:00.165+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:37:00.165+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:37:00.165+05:30 [Info] Setting log level to Info 2023-10-21T04:37:00.165+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:37:00.165+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:37:00.166+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:37:00.166+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:37:00.186+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:37:00 audit: created new audit service 2023-10-21T04:37:00.321+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:37:00.321+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:37:00.321+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:37:00.322+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:37:00.322+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:37:00.322+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:37:00.365+05:30 [Info] security context initialized 2023-10-21T04:37:00.365+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:37:00.365+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:37:00.365+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:37:00.366+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:37:00.366+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:37:00.372+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:37:00.372+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:37:00.372+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:37:00.376+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:37:00.377+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:37:00.377+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:37:00.377+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:37:00.381+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:37:00.384+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:37:00.385+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:37:00.385+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:37:00.386+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:37:00.386+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:37:00.425+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:37:00.425+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:37:00.425+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:37:00.428+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:37:00.428+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:37:00.430+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:37:00.431+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:37:00.431+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:37:00.431+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:37:00.432+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:37:00.435+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 195 us 2023-10-21T04:37:00.436+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:37:00.446+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:37:00.447+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:37:00.458+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39064 2023-10-21T04:37:00.466+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:37:00.466+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:37:00.466+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:37:00.466+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:37:00.466+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:37:00.466+05:30 [Info] Indexer Id 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:37:00.466+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:37:00.466+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:37:00.482+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:37:00.485+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:37:00.487+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 182] updating cluster rev epoch 0 -> 1 2023-10-21T04:37:00.487+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 182] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:37:00.492+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 218] bucketNames has been updated (was map[], now map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43]) 2023-10-21T04:37:00.492+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 188] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:37:00.493+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 189] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:37:00.502+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 218] updated cluster rebalancing status to false 2023-10-21T04:37:00.502+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:37:00.503+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:37:00.503+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:37:00.505+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:37:00.505+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:37:00.505+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:37:00.506+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:37:00.506+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning true RebalanceToken &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:37:00.506+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:37:00.507+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:37:00.507+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:37:00.507+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:37:00.523+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:37:00.523+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:37:00.525+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:37:00.526+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:37:00.526+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:37:00.548+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 2 ReplicaId: 0 InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 InstanceId: 8732144615635911668 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_READY Stream: NIL_STREAM RState: RebalActive Version: 1 ReplicaId: 0 InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 2 ReplicaId: 0 InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 InstanceId: 3580008349810282285 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_READY Stream: NIL_STREAM RState: RebalActive Version: 2 ReplicaId: 0 , elapsed: 41.413652ms 2023-10-21T04:37:00.549+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:37:00.549+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:37:00.549+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:37:00.550+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager true &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:37:00.550+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:37:00.550+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:37:00.550+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:37:00.551+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:37:00.581+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:37:00.581+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:37:00.589+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:37:00.589+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:false Messages:[]} 2023-10-21T04:37:00.599+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:37:00.600+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:37:00.631+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err genServer.closed 2023-10-21T04:37:00.631+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:37:00.631+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating genServer.closed As Success 2023-10-21T04:37:00.631+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:37:00.632+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b. Err projector.topicMissing 2023-10-21T04:37:00.632+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:37:00.632+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:37:00.632+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:00.639+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:37:00.639+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:37:00.641+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.642+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:37:00.642+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 (10170029854712240877) added 2023-10-21T04:37:00.642+05:30 [Info] : Recover shard shard10170029854712240877 from metadata completed. 2023-10-21T04:37:00.643+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 (15531034489713578274) added 2023-10-21T04:37:00.643+05:30 [Info] : Recover shard shard15531034489713578274 from metadata completed. 2023-10-21T04:37:00.643+05:30 [Info] tenant 'bucket_%2_b9538b29d7e025c819e734b646db039c': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 (17406544375239214575) added 2023-10-21T04:37:00.644+05:30 [Info] : Recover shard shard17406544375239214575 from metadata completed. 2023-10-21T04:37:00.644+05:30 [Info] tenant 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 (4718895791748289908) added 2023-10-21T04:37:00.646+05:30 [Info] : Recover shard shard4718895791748289908 from metadata completed. 2023-10-21T04:37:00.646+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': added 2023-10-21T04:37:00.646+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:37:00.647+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner initialized 2023-10-21T04:37:00.648+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:00.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.649+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCleaner initialized for recovery 2023-10-21T04:37:00.649+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:00.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery 2023-10-21T04:37:00.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true] 2023-10-21T04:37:00.649+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1966080] Data log Head [0] Tail [2101248] 2023-10-21T04:37:00.650+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1966080] 2023-10-21T04:37:00.651+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner initialized 2023-10-21T04:37:00.651+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:00.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.654+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCleaner initialized for recovery 2023-10-21T04:37:00.654+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:00.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery 2023-10-21T04:37:00.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true] 2023-10-21T04:37:00.655+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:37:00.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[2011136] Data log Head [0] Tail [2134016] 2023-10-21T04:37:00.656+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [2011136] 2023-10-21T04:37:00.658+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.659+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:37:00.659+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:37:00.660+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.660+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:37:00.660+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.660+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:37:00.660+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.660+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:37:00.661+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.661+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:37:00.661+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.662+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:37:00.662+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.663+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:37:00.663+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.663+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:37:00.663+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.665+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:37:00.665+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.666+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:37:00.666+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.667+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:37:00.668+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.668+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:37:00.668+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.668+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:37:00.669+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.670+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:37:00.670+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.670+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:37:00.671+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.674+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:37:00.674+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.674+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:37:00.678+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.681+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:00.681+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.681+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:00.682+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.682+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:00.682+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.682+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:00.682+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.682+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:00.683+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.683+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:00.683+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.683+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:00.684+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.684+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:00.681+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.685+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:00.686+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.686+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:00.686+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.686+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:00.687+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.687+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:00.687+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.688+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:00.688+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.688+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:00.688+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.688+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:00.689+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.689+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:37:00.689+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.690+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:00.691+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:00.692+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:37:00.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : replayOffset: 2125824 (RP:2117632) (CP:2125824) 2023-10-21T04:37:00.692+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : replayOffset: 2093056 (RP:2021388) (CP:2093056) 2023-10-21T04:37:00.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [2093056] took [44.800952ms] 2023-10-21T04:37:00.694+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.694+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.694+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.694+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.694+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053288, resident 1, memSz 8667, memSzIndex 0, numRecords 112 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053288, resident 1, memSz 9669, memSzIndex 76, numRecords 128 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053288, resident 1, memSz 9511, memSzIndex 76, numRecords 127 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053288, resident 1, memSz 9997, memSzIndex 77, numRecords 131 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053364, resident 1, memSz 9853, memSzIndex 0, numRecords 123 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053288, resident 1, memSz 9287, memSzIndex 0, numRecords 118 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1056060, resident 1, memSz 66997, memSzIndex 872, numRecords 1000 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053364, resident 1, memSz 9834, memSzIndex 0, numRecords 123 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1053288, resident 1, memSz 10687, memSzIndex 75, numRecords 138 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.695+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.695+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [2101248] replayOffset [2093056] 2023-10-21T04:37:00.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [45.880939ms] 2023-10-21T04:37:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.696+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.697+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [2125824] took [39.612201ms] 2023-10-21T04:37:00.697+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057276, resident 1, memSz 11483, memSzIndex 0, numRecords 112 2023-10-21T04:37:00.697+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.698+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.698+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.698+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.697+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.701+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.701+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.701+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.701+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057276, resident 1, memSz 12979, memSzIndex 0, numRecords 127 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057276, resident 1, memSz 14041, memSzIndex 0, numRecords 138 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1055776, resident 1, memSz 66045, memSzIndex 508, numRecords 1000 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data], Shared [true]. Built [17] plasmas, took [53.551366ms] 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 128 metaSz: 2520 rp [1] sn: 1080008 count: 128 metaSz: 2520 rp [2] sn: 1080009 count: 128 metaSz: 2520 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 141790 recovery UsedSpace 190834 2023-10-21T04:37:00.703+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057426, resident 1, memSz 12606, memSzIndex 0, numRecords 123 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057426, resident 1, memSz 12625, memSzIndex 0, numRecords 123 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057276, resident 1, memSz 12079, memSzIndex 0, numRecords 118 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1055176, resident 1, memSz 816, memSzIndex 0, numRecords 10 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057276, resident 1, memSz 13374, memSzIndex 0, numRecords 131 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1057276, resident 1, memSz 13061, memSzIndex 0, numRecords 128 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:00.704+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: controller ('bucket_%2_b9538b29d7e025c819e734b646db039c') quota after header replay 45801990 controller mem 4952 2023-10-21T04:37:00.704+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [2134016] replayOffset [2125824] 2023-10-21T04:37:00.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [48.184656ms] 2023-10-21T04:37:00.705+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.705+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.705+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.706+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.706+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.706+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.706+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.707+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.707+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.707+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.707+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.708+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.711+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.711+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.712+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.712+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.712+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.712+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:00.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data], Shared [true]. Built [18] plasmas, took [58.011894ms] 2023-10-21T04:37:00.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:37:00.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.717+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swapper started 2023-10-21T04:37:00.717+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:37:00.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex started 2023-10-21T04:37:00.717+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.717+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.714+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:00.719+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 1080008 count: 128 metaSz: 2520 rp [1] sn: 1080009 count: 128 metaSz: 2520 2023-10-21T04:37:00.720+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 135818 recovery UsedSpace 165572 2023-10-21T04:37:00.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:37:00.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.725+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:37:00.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex started 2023-10-21T04:37:00.725+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:37:00.726+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.726+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.748+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Warmup took 106.705659ms 2023-10-21T04:37:00.749+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 1 WriterThreads 1 2023-10-21T04:37:00.749+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index Index: 8965923595543532942 Partition: 1 2023-10-21T04:37:00.749+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.750+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:37:00.750+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 1080008 count: 123 metaSz: 2520 rp [1] sn: 1080009 count: 123 metaSz: 2520 2023-10-21T04:37:00.750+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 135245 recovery UsedSpace 165572 2023-10-21T04:37:00.750+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:37:00.750+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 123 metaSz: 2520 rp [1] sn: 1080008 count: 123 metaSz: 2520 rp [2] sn: 1080009 count: 123 metaSz: 2520 2023-10-21T04:37:00.751+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 135536 recovery UsedSpace 190126 2023-10-21T04:37:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:37:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.770+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:37:00.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex started 2023-10-21T04:37:00.770+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.770+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:37:00.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.772+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:37:00.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex started 2023-10-21T04:37:00.772+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:37:00.772+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.773+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.807+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Warmup took 57.727793ms 2023-10-21T04:37:00.807+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 2 WriterThreads 1 2023-10-21T04:37:00.807+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index Index: 8965923595543532942 Partition: 2 2023-10-21T04:37:00.807+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.808+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:37:00.808+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 123 metaSz: 2520 rp [1] sn: 1080008 count: 123 metaSz: 2520 rp [2] sn: 1080009 count: 123 metaSz: 2520 2023-10-21T04:37:00.808+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 135200 recovery UsedSpace 190126 2023-10-21T04:37:00.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.808+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:37:00.808+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 1080008 count: 123 metaSz: 2520 rp [1] sn: 1080009 count: 123 metaSz: 2520 2023-10-21T04:37:00.809+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 135300 recovery UsedSpace 165572 2023-10-21T04:37:00.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:37:00.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.813+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:37:00.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex started 2023-10-21T04:37:00.813+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.813+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:37:00.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.814+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:37:00.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex started 2023-10-21T04:37:00.814+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.814+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.814+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:37:00.832+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Warmup took 23.958116ms 2023-10-21T04:37:00.832+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 3 WriterThreads 1 2023-10-21T04:37:00.832+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index Index: 8965923595543532942 Partition: 3 2023-10-21T04:37:00.832+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.833+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:37:00.833+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 118 metaSz: 2520 rp [1] sn: 1080008 count: 118 metaSz: 2520 rp [2] sn: 1080009 count: 118 metaSz: 2520 2023-10-21T04:37:00.833+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 147680 recovery UsedSpace 190126 2023-10-21T04:37:00.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.834+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:00.834+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 1080008 count: 118 metaSz: 2520 rp [1] sn: 1080009 count: 118 metaSz: 2520 2023-10-21T04:37:00.834+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 134427 recovery UsedSpace 165572 2023-10-21T04:37:00.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:37:00.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.838+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:37:00.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex started 2023-10-21T04:37:00.839+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.839+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:37:00.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.840+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:37:00.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex started 2023-10-21T04:37:00.840+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:37:00.841+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.842+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.851+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Warmup took 18.305276ms 2023-10-21T04:37:00.851+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 4 WriterThreads 1 2023-10-21T04:37:00.851+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index Index: 8965923595543532942 Partition: 4 2023-10-21T04:37:00.851+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 238 2023-10-21T04:37:00.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 138 metaSz: 2520 rp [1] sn: 1080008 count: 138 metaSz: 2520 rp [2] sn: 1080009 count: 138 metaSz: 2520 2023-10-21T04:37:00.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.852+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:00.852+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 157940 recovery UsedSpace 190824 2023-10-21T04:37:00.852+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 1080008 count: 138 metaSz: 2520 rp [1] sn: 1080009 count: 138 metaSz: 2520 2023-10-21T04:37:00.852+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 137427 recovery UsedSpace 169668 2023-10-21T04:37:00.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:37:00.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.859+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:37:00.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex started 2023-10-21T04:37:00.859+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:37:00.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.860+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:37:00.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex started 2023-10-21T04:37:00.860+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:37:00.862+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.862+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.862+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.894+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Warmup took 42.757546ms 2023-10-21T04:37:00.895+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 5 WriterThreads 1 2023-10-21T04:37:00.895+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index Index: 8965923595543532942 Partition: 5 2023-10-21T04:37:00.895+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 256 2023-10-21T04:37:00.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 127 metaSz: 2520 rp [1] sn: 1080008 count: 127 metaSz: 2520 rp [2] sn: 1080009 count: 127 metaSz: 2520 2023-10-21T04:37:00.895+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 154870 recovery UsedSpace 190740 2023-10-21T04:37:00.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.896+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:00.896+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 127 metaSz: 2520 rp [1] sn: 1080008 count: 127 metaSz: 2520 rp [2] sn: 1080009 count: 127 metaSz: 2520 2023-10-21T04:37:00.896+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 135627 recovery UsedSpace 157358 2023-10-21T04:37:00.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:37:00.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.906+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:37:00.906+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex started 2023-10-21T04:37:00.907+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.907+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:37:00.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.908+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:37:00.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex started 2023-10-21T04:37:00.908+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:37:00.908+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.908+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.924+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Warmup took 29.271775ms 2023-10-21T04:37:00.925+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 6 WriterThreads 1 2023-10-21T04:37:00.925+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index Index: 8965923595543532942 Partition: 6 2023-10-21T04:37:00.925+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.926+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:00.926+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 1080008 count: 131 metaSz: 2520 rp [1] sn: 1080009 count: 131 metaSz: 2520 2023-10-21T04:37:00.926+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 150390 recovery UsedSpace 165572 2023-10-21T04:37:00.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.927+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:37:00.927+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 131 metaSz: 2520 rp [1] sn: 1080008 count: 131 metaSz: 2520 rp [2] sn: 1080009 count: 131 metaSz: 2520 2023-10-21T04:37:00.927+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 156560 recovery UsedSpace 190844 2023-10-21T04:37:00.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:37:00.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.934+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:37:00.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex started 2023-10-21T04:37:00.934+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.934+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:37:00.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.935+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:37:00.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex started 2023-10-21T04:37:00.935+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:37:00.935+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.935+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.946+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Warmup took 21.141725ms 2023-10-21T04:37:00.947+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 7 WriterThreads 1 2023-10-21T04:37:00.947+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index Index: 8965923595543532942 Partition: 7 2023-10-21T04:37:00.947+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8965923595543532942 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.947+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:37:00.947+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 1080007 count: 112 metaSz: 2520 rp [1] sn: 1080008 count: 112 metaSz: 2520 rp [2] sn: 1080009 count: 112 metaSz: 2520 2023-10-21T04:37:00.948+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 146470 recovery UsedSpace 190126 2023-10-21T04:37:00.948+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.948+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:00.948+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 1080008 count: 112 metaSz: 2520 rp [1] sn: 1080009 count: 112 metaSz: 2520 2023-10-21T04:37:00.948+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 146630 recovery UsedSpace 165572 2023-10-21T04:37:00.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:37:00.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:00.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:00.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:37:00.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:00.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:00.953+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:37:00.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex started 2023-10-21T04:37:00.954+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.954+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.953+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:37:00.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex started 2023-10-21T04:37:00.954+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:37:00.955+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:00.955+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:00.977+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Warmup took 30.207047ms 2023-10-21T04:37:00.978+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8965923595543532942 partitionId 8 WriterThreads 1 2023-10-21T04:37:00.978+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8965923595543532942 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index Index: 8965923595543532942 Partition: 8 2023-10-21T04:37:00.978+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8965923595543532942 2023-10-21T04:37:00.978+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8965923595543532942 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:37:00.980+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:7 Attempting to open snapshot (SnapshotInfo: count:131 committed:false) 2023-10-21T04:37:00.981+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:8 Attempting to open snapshot (SnapshotInfo: count:112 committed:false) 2023-10-21T04:37:00.982+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:1 Attempting to open snapshot (SnapshotInfo: count:128 committed:false) 2023-10-21T04:37:00.983+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:2 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:37:00.984+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:3 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:37:00.985+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:4 Attempting to open snapshot (SnapshotInfo: count:118 committed:false) 2023-10-21T04:37:00.987+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:5 Attempting to open snapshot (SnapshotInfo: count:138 committed:false) 2023-10-21T04:37:00.988+05:30 [Info] StorageMgr::openSnapshot IndexInst:8965923595543532942 Partition:6 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:37:00.988+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {8965923595543532942 {8910459001587546151 idx_partitioned plasma bucket_%2 b9538b29d7e025c819e734b646db039c false [`emailid`] N1QL KEY [false] false true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc005b19740 [] 1 1 false plasma 0} map[1:{{1 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc004cdf5a0} 2:{{2 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc0059f70d0} 3:{{3 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc0059f73c0} 4:{{4 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005ba8c30} 5:{{5 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005aa7760} 6:{{6 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc004f39920} 7:{{7 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc0058f9240} 8:{{8 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005cd86b0}] 0xc004ef1200 0xc005a68a80} 2023-10-21T04:37:00.989+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:00.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:00.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:00.990+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:00.990+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:00.990+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:00.990+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:01.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.007+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:01.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:37:01.007+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.007+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:01.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.016+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:01.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:37:01.016+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:37:01.016+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:37:01.016+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:37:01.016+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:01.016+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.016+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.017+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.017+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:37:01.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:37:01.026+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:37:01.026+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:01.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.026+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.047+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:37:01.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:37:01.047+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:37:01.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:37:01.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:37:01.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:37:01.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:37:01.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.059+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.059+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.064+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.064+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:01.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.070+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:01.070+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:37:01.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex started 2023-10-21T04:37:01.070+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.070+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:01.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.072+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:01.072+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:37:01.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex started 2023-10-21T04:37:01.072+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 2 Warmup took 82.898753ms 2023-10-21T04:37:01.073+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 2 WriterThreads 1 2023-10-21T04:37:01.073+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index Index: 8732144615635911668 Partition: 2 2023-10-21T04:37:01.073+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.073+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.073+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:01.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.079+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:01.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:37:01.079+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.079+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:01.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.080+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:01.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:37:01.080+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:37:01.080+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:37:01.080+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:37:01.080+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:01.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.080+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.081+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.081+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:37:01.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:37:01.085+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:37:01.085+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:01.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.085+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.089+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:37:01.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:37:01.090+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:37:01.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:37:01.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:37:01.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:37:01.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:37:01.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.097+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.103+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.103+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:01.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.107+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:01.107+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:37:01.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex started 2023-10-21T04:37:01.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:01.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.109+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:01.109+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:37:01.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex started 2023-10-21T04:37:01.109+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 3 Warmup took 36.176738ms 2023-10-21T04:37:01.109+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.109+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.109+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 3 WriterThreads 1 2023-10-21T04:37:01.109+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index Index: 8732144615635911668 Partition: 3 2023-10-21T04:37:01.110+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.110+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.110+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.110+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.110+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:01.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.118+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:01.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:37:01.118+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.118+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:01.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.118+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:01.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:37:01.119+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:37:01.119+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:37:01.119+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:37:01.119+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:01.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.119+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.121+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 16] will report aggregate recorder stats every 5m0s 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:37:01.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:37:01.124+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:37:01.124+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:01.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.124+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.129+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:37:01.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:37:01.129+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:37:01.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:37:01.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:37:01.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:37:01.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:37:01.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.140+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.141+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.151+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:01.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.173+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:01.173+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:37:01.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex started 2023-10-21T04:37:01.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:01.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.175+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:01.175+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:37:01.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex started 2023-10-21T04:37:01.175+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 4 Warmup took 65.178288ms 2023-10-21T04:37:01.175+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.175+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.175+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 4 WriterThreads 1 2023-10-21T04:37:01.175+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index Index: 8732144615635911668 Partition: 4 2023-10-21T04:37:01.175+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.176+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.176+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.176+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.176+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:37:01.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.181+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:37:01.181+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:37:01.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:37:01.181+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:01.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.182+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.181+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.190+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:37:01.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:37:01.191+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:37:01.191+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:01.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.191+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.197+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:37:01.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:37:01.198+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:37:01.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:37:01.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:37:01.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:37:01.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:37:01.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.209+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.210+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.214+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:01.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.222+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:01.222+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:37:01.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex started 2023-10-21T04:37:01.222+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.223+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:01.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.225+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:01.226+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:37:01.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex started 2023-10-21T04:37:01.226+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 5 Warmup took 50.065066ms 2023-10-21T04:37:01.226+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 5 WriterThreads 1 2023-10-21T04:37:01.226+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index Index: 8732144615635911668 Partition: 5 2023-10-21T04:37:01.226+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.226+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.226+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.227+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.227+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.227+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.227+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:01.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.233+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:01.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:37:01.233+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.234+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:01.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.234+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:01.234+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:37:01.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:37:01.235+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:37:01.235+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:01.235+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.235+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:37:01.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:37:01.240+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:37:01.240+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:01.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.240+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.245+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:37:01.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:37:01.245+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:37:01.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:37:01.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:37:01.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:37:01.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:37:01.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.253+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.253+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.258+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.258+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:01.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.263+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:01.263+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:37:01.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex started 2023-10-21T04:37:01.263+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.263+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:01.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.271+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:01.271+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:37:01.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex started 2023-10-21T04:37:01.271+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 6 Warmup took 44.83217ms 2023-10-21T04:37:01.271+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.271+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.272+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 6 WriterThreads 1 2023-10-21T04:37:01.272+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index Index: 8732144615635911668 Partition: 6 2023-10-21T04:37:01.272+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.272+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.272+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.272+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.272+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:01.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.278+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:01.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:37:01.278+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:01.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.278+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.278+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:01.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:37:01.279+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:37:01.279+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:37:01.279+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:37:01.279+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:01.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.279+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.290+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:37:01.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:37:01.291+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:37:01.291+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:01.291+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.291+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.305+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:37:01.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:37:01.305+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:37:01.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:37:01.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:37:01.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:37:01.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.313+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:37:01.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.319+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.320+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.322+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.322+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:01.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.328+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:01.328+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:37:01.328+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex started 2023-10-21T04:37:01.328+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.328+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:01.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.329+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:01.330+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:37:01.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex started 2023-10-21T04:37:01.330+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 7 Warmup took 57.756638ms 2023-10-21T04:37:01.330+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 7 WriterThreads 1 2023-10-21T04:37:01.330+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index Index: 8732144615635911668 Partition: 7 2023-10-21T04:37:01.330+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.330+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.330+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.331+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.331+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.331+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.331+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:01.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.338+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:01.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:37:01.339+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.339+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:01.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.339+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:01.339+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:37:01.339+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:37:01.339+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.339+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.339+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.339+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:37:01.340+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:37:01.340+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:01.340+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.340+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.345+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:37:01.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:37:01.345+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.345+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.345+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.345+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:37:01.346+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:01.346+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:37:01.346+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:37:01.346+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:01.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:37:01.346+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.346+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.346+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.346+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.351+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:37:01.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:37:01.351+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:37:01.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:37:01.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:37:01.354+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:37:01.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:37:01.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.361+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.361+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.364+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.364+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:01.366+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.367+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:01.367+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:37:01.367+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex started 2023-10-21T04:37:01.367+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.367+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:01.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.369+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:01.369+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:37:01.369+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex started 2023-10-21T04:37:01.369+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 8 Warmup took 39.379321ms 2023-10-21T04:37:01.370+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.370+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.370+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 8 WriterThreads 1 2023-10-21T04:37:01.370+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index Index: 8732144615635911668 Partition: 8 2023-10-21T04:37:01.370+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8732144615635911668 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.370+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.370+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.370+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.370+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.370+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:37:01.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:01.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.380+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:01.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:37:01.380+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.380+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 36 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:01.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.380+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:01.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:37:01.381+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:37:01.381+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.382+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.381+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.383+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.383+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:37:01.383+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:01.383+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:37:01.383+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:37:01.383+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:01.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:37:01.383+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.384+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.384+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.384+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:37:01.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:37:01.395+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:37:01.395+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:01.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.395+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:37:01.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:37:01.406+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:37:01.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 36 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:37:01.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:37:01.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:37:01.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:37:01.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.418+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.418+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.424+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.424+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:01.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.434+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:01.434+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:37:01.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex started 2023-10-21T04:37:01.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:01.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.436+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:01.436+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:37:01.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex started 2023-10-21T04:37:01.436+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8732144615635911668 PartitionId 1 Warmup took 66.138329ms 2023-10-21T04:37:01.437+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8732144615635911668 partitionId 1 WriterThreads 1 2023-10-21T04:37:01.437+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8732144615635911668 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index Index: 8732144615635911668 Partition: 1 2023-10-21T04:37:01.437+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8732144615635911668 2023-10-21T04:37:01.437+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Partitions [2 3 4 5 6 7 8 1] 2023-10-21T04:37:01.437+05:30 [Info] StorageMgr::openSnapshot IndexInst:8732144615635911668 Partition:4 No Snapshot Found. 2023-10-21T04:37:01.437+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8732144615635911668 Adding Nil Snapshot. 2023-10-21T04:37:01.437+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {8732144615635911668 {10606939961365971246 idx_partitioned_defer plasma bucket_%2 b9538b29d7e025c819e734b646db039c false [`balance`] N1QL KEY [false] true true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 0 0 0 0xc0060bf1d0 [] 1 0 false plasma 0} map[1:{{1 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005e37a80} 2:{{2 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005df5f10} 3:{{3 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005e365f0} 4:{{4 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005c5bc10} 5:{{5 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005d21780} 6:{{6 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005eb9940} 7:{{7 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005d6f410} 8:{{8 1 [:9123] [17406544375239214575 10170029854712240877]} 0xc005e373c0}] 0xc0060f5900 0xc00609dc20} 2023-10-21T04:37:01.437+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3643393537124379309 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.437+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': added 2023-10-21T04:37:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.440+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : LSSCleaner initialized 2023-10-21T04:37:01.440+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:01.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.441+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : LSSCleaner initialized for recovery 2023-10-21T04:37:01.441+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:01.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery 2023-10-21T04:37:01.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data], Shared [true] 2023-10-21T04:37:01.441+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[2007040] Data log Head [0] Tail [2097152] 2023-10-21T04:37:01.441+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [2007040] 2023-10-21T04:37:01.444+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.444+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:37:01.444+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:37:01.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.445+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:37:01.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.445+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:37:01.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.445+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:37:01.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.445+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:37:01.445+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.445+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:37:01.446+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.446+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:37:01.446+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.446+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:37:01.446+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.446+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:37:01.452+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.452+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:37:01.454+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.454+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:01.454+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.454+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:01.455+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.455+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.455+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.455+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.456+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.456+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.456+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.456+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.457+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.457+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.459+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.461+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.461+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : replayOffset: 2088960 (RP:2088960) (CP:1785856) 2023-10-21T04:37:01.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [2088960] took [20.179843ms] 2023-10-21T04:37:01.461+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13778, memSzIndex 0, numRecords 136 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12721, memSzIndex 0, numRecords 125 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 13797, memSzIndex 0, numRecords 136 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12723, memSzIndex 0, numRecords 125 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1055940, resident 1, memSz 70472, memSzIndex 353, numRecords 1000 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12401, memSzIndex 0, numRecords 122 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057276, resident 1, memSz 12889, memSzIndex 0, numRecords 127 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 12513, memSzIndex 0, numRecords 123 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1057426, resident 1, memSz 10832, memSzIndex 0, numRecords 106 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: request quota after header replay 1055476, resident 0.2, memSz 438, memSzIndex 0, numRecords 10 2023-10-21T04:37:01.462+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 27935222 controller mem 1442 2023-10-21T04:37:01.462+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [2097152] replayOffset [2088960] 2023-10-21T04:37:01.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [21.21249ms] 2023-10-21T04:37:01.463+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.464+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.469+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.469+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.469+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.469+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.470+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.470+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.470+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.470+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data], Shared [true]. Built [18] plasmas, took [29.613721ms] 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1055476 memSz 64 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 360005 count: 6 metaSz: 2483 rp [1] sn: 720008 count: 6 metaSz: 2483 2023-10-21T04:37:01.471+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 115154 recovery UsedSpace 186586 2023-10-21T04:37:01.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:37:01.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.480+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swapper started 2023-10-21T04:37:01.480+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:37:01.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex started 2023-10-21T04:37:01.480+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3643393537124379309 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:37:01.484+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.484+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.493+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3643393537124379309 PartitionId 0 Warmup took 55.868588ms 2023-10-21T04:37:01.493+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3643393537124379309 partitionId 0 WriterThreads 4 2023-10-21T04:37:01.493+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3643393537124379309 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index Index: 3643393537124379309 Partition: 0 2023-10-21T04:37:01.494+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3643393537124379309 2023-10-21T04:37:01.494+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3643393537124379309 Partitions [0] 2023-10-21T04:37:01.495+05:30 [Info] StorageMgr::openSnapshot IndexInst:3643393537124379309 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:37:01.495+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {3643393537124379309 {11276338709617092948 #primary plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 true [] N1QL SINGLE [] false false [] false false 1 [] false 0 {true 1 0 0} _system _query 8 9 false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc006137770 [] 2 0 false plasma 0} map[0:{{0 2 [:9123] [4718895791748289908]} 0xc005eed9f0}] 0xc00615a000 0xc005e77aa0} 2023-10-21T04:37:01.495+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4443661069773468873 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.498+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : LSSCleaner initialized 2023-10-21T04:37:01.498+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:01.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.499+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : LSSCleaner initialized for recovery 2023-10-21T04:37:01.499+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:37:01.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery 2023-10-21T04:37:01.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data], Shared [true] 2023-10-21T04:37:01.499+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1748992] Data log Head [0] Tail [1888256] 2023-10-21T04:37:01.500+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1748992] 2023-10-21T04:37:01.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.501+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1055940 memSz 866 2023-10-21T04:37:01.501+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 21 count: 1000 metaSz: 2511 rp [1] sn: 22 count: 1000 metaSz: 2519 2023-10-21T04:37:01.501+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 246345 recovery UsedSpace 43745 2023-10-21T04:37:01.505+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.506+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:37:01.506+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.506+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:37:01.507+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.507+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:37:01.507+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.507+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:37:01.508+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.508+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:37:01.508+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.508+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:37:01.509+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.509+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:37:01.509+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:37:01.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.509+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:37:01.509+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:37:01.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex started 2023-10-21T04:37:01.509+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.509+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.515+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 259] aggregate storage info: map[bucket_%2:map[kv:2377920 index:13676804] bucket_1:map[kv:2443456 index:13119672]] 2023-10-21T04:37:01.521+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.521+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:37:01.522+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.522+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:01.523+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.523+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:01.523+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.523+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.523+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.524+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.524+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.524+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.524+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.524+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.524+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.524+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.524+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.525+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.525+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : replayOffset: 1875968 (RP:1875968) (CP:1511424) 2023-10-21T04:37:01.525+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1875968] took [25.260845ms] 2023-10-21T04:37:01.525+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9257, memSzIndex 77, numRecords 125 2023-10-21T04:37:01.525+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9821, memSzIndex 0, numRecords 123 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9705, memSzIndex 0, numRecords 122 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.526+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053288, resident 1, memSz 10516, memSzIndex 93, numRecords 136 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 10517, memSzIndex 76, numRecords 136 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 8071, memSzIndex 0, numRecords 106 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1056174, resident 1, memSz 68004, memSzIndex 887, numRecords 1000 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053288, resident 1, memSz 9573, memSzIndex 76, numRecords 127 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: request quota after header replay 1053364, resident 1, memSz 9253, memSzIndex 76, numRecords 125 2023-10-21T04:37:01.528+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: controller ('bucket_1_1fc3689e709c62b9ee184f2ec13e6f43') quota after header replay 43705944 controller mem 5060 2023-10-21T04:37:01.528+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1888256] replayOffset [1875968] 2023-10-21T04:37:01.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [28.512543ms] 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.529+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.530+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.530+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.530+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.531+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.531+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.531+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:37:01.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data], Shared [true]. Built [17] plasmas, took [31.439357ms] 2023-10-21T04:37:01.532+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1056174 memSz 2102 2023-10-21T04:37:01.532+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 21 count: 1000 metaSz: 2511 rp [1] sn: 22 count: 1000 metaSz: 2519 2023-10-21T04:37:01.532+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 218184 recovery UsedSpace 47006 2023-10-21T04:37:01.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 37 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:37:01.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.538+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:37:01.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex started 2023-10-21T04:37:01.538+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.538+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.538+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4443661069773468873 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:37:01.561+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4443661069773468873 PartitionId 0 Warmup took 66.122187ms 2023-10-21T04:37:01.562+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4443661069773468873 partitionId 0 WriterThreads 4 2023-10-21T04:37:01.562+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4443661069773468873 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index Index: 4443661069773468873 Partition: 0 2023-10-21T04:37:01.562+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4443661069773468873 2023-10-21T04:37:01.563+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4443661069773468873 Partitions [0] 2023-10-21T04:37:01.564+05:30 [Info] StorageMgr::openSnapshot IndexInst:4443661069773468873 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:37:01.564+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {4443661069773468873 {12582561312368048736 idx_secondary plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 false [`age`] N1QL SINGLE [false] false false [] false false 1 [] false 0 {true 1 0 0} _default c1 0 c false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc00617d290 [] 1 1 false plasma 0} map[0:{{0 1 [:9123] [4718895791748289908 15531034489713578274]} 0xc006143d80}] 0xc006998c80 0xc006172e40} 2023-10-21T04:37:01.564+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3127033699511712809 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:01.564+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1056060 memSz 2040 2023-10-21T04:37:01.564+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: RP version: 4 numRPs: 3 rp [0] sn: 11 count: 1000 metaSz: 2517 rp [1] sn: 12 count: 1000 metaSz: 2525 rp [2] sn: 13 count: 1000 metaSz: 2524 2023-10-21T04:37:01.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:01.565+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1104 2023-10-21T04:37:01.565+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 5 numRPs: 2 rp [0] sn: 12 count: 1000 metaSz: 2525 rp [1] sn: 13 count: 1000 metaSz: 2524 2023-10-21T04:37:01.565+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 270480 recovery UsedSpace 33360 2023-10-21T04:37:01.565+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 263790 recovery UsedSpace 39571 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:01.571+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex started 2023-10-21T04:37:01.571+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:37:01.571+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.571+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex started 2023-10-21T04:37:01.571+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.571+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3127033699511712809 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:37:01.571+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.571+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.605+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3127033699511712809 PartitionId 0 Warmup took 41.158811ms 2023-10-21T04:37:01.606+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3127033699511712809 partitionId 0 WriterThreads 4 2023-10-21T04:37:01.606+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3127033699511712809 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index Index: 3127033699511712809 Partition: 0 2023-10-21T04:37:01.606+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3127033699511712809 2023-10-21T04:37:01.606+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3127033699511712809 Partitions [0] 2023-10-21T04:37:01.607+05:30 [Info] StorageMgr::openSnapshot IndexInst:3127033699511712809 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:37:01.607+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {3127033699511712809 {14549775068266281786 idx_secondary plasma bucket_%2 b9538b29d7e025c819e734b646db039c false [`age`] N1QL SINGLE [false] false false [] false false 1 [] false 0 {true 1 0 0} _default c1 0 c false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc006e10780 [] 0 1 false plasma 0} map[0:{{0 0 [:9123] [17406544375239214575 10170029854712240877]} 0xc00617f8f0}] 0xc006e18580 0xc005fd2c60} 2023-10-21T04:37:01.608+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.608+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.608+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:01.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:01.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.616+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex started 2023-10-21T04:37:01.616+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.616+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.624+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex started 2023-10-21T04:37:01.625+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:37:01.625+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.625+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:37:01.625+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.625+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.626+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.626+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:37:01.626+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:37:01.626+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex stopped 2023-10-21T04:37:01.626+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.626+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.626+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.626+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.638+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:37:01.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex closed 2023-10-21T04:37:01.638+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.638+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.638+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.638+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:37:01.639+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.639+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:37:01.639+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:37:01.639+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex stopped 2023-10-21T04:37:01.639+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.639+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.639+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.639+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.647+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:37:01.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex closed 2023-10-21T04:37:01.648+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex ... 2023-10-21T04:37:01.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex ... 2023-10-21T04:37:01.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex ... 2023-10-21T04:37:01.652+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex ... 2023-10-21T04:37:01.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index destroyed 2023-10-21T04:37:01.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.665+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.665+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.665+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.668+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.669+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.669+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.669+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.672+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.672+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:37:01.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex started 2023-10-21T04:37:01.672+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.672+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.675+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.675+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:37:01.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex started 2023-10-21T04:37:01.675+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 3 Warmup took 67.435715ms 2023-10-21T04:37:01.676+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 3 WriterThreads 1 2023-10-21T04:37:01.676+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index Index: 3580008349810282285 Partition: 3 2023-10-21T04:37:01.676+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.676+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.676+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:01.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.681+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.681+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:01.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 41 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.682+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex started 2023-10-21T04:37:01.682+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.682+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.689+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex started 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.689+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:37:01.689+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:37:01.689+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.689+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:37:01.689+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:37:01.689+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex stopped 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.689+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:37:01.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex closed 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:37:01.699+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.699+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:37:01.699+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:37:01.699+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex stopped 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.699+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.706+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:37:01.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex closed 2023-10-21T04:37:01.706+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex ... 2023-10-21T04:37:01.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 41 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex ... 2023-10-21T04:37:01.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex ... 2023-10-21T04:37:01.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex ... 2023-10-21T04:37:01.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index destroyed 2023-10-21T04:37:01.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.720+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.720+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.720+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.725+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.725+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 47 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.737+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.737+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:37:01.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex started 2023-10-21T04:37:01.737+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.737+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.738+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.738+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:37:01.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex started 2023-10-21T04:37:01.739+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 4 Warmup took 62.576631ms 2023-10-21T04:37:01.739+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.739+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.739+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 4 WriterThreads 1 2023-10-21T04:37:01.739+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index Index: 3580008349810282285 Partition: 4 2023-10-21T04:37:01.739+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.739+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.740+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.740+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:01.739+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.754+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex started 2023-10-21T04:37:01.754+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.754+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 42 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.754+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex started 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.755+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:37:01.755+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:37:01.755+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.755+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:37:01.755+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:37:01.755+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex stopped 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.755+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.766+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:37:01.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex closed 2023-10-21T04:37:01.766+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.766+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.767+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.767+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:37:01.767+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.767+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:37:01.767+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:37:01.767+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex stopped 2023-10-21T04:37:01.767+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.767+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.767+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.767+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.776+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:37:01.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex closed 2023-10-21T04:37:01.777+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex ... 2023-10-21T04:37:01.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 42 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex ... 2023-10-21T04:37:01.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex ... 2023-10-21T04:37:01.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex ... 2023-10-21T04:37:01.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index destroyed 2023-10-21T04:37:01.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.788+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.788+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.788+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.792+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.792+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 48 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.797+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.798+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:37:01.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex started 2023-10-21T04:37:01.798+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.798+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.800+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.800+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:37:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex started 2023-10-21T04:37:01.800+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 5 Warmup took 61.119639ms 2023-10-21T04:37:01.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.800+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.801+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 5 WriterThreads 1 2023-10-21T04:37:01.801+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index Index: 3580008349810282285 Partition: 5 2023-10-21T04:37:01.801+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.801+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.801+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:01.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.801+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.802+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 43 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.807+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex started 2023-10-21T04:37:01.807+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex started 2023-10-21T04:37:01.807+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.807+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.807+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.807+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:37:01.807+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.808+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.807+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.808+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:37:01.808+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.808+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:37:01.808+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:37:01.808+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex stopped 2023-10-21T04:37:01.808+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.808+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.808+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.808+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.808+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.814+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:37:01.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex closed 2023-10-21T04:37:01.814+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.814+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.814+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.814+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:37:01.814+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.814+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:37:01.814+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:37:01.814+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex stopped 2023-10-21T04:37:01.815+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.815+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.815+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.815+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.822+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:37:01.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex closed 2023-10-21T04:37:01.822+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex ... 2023-10-21T04:37:01.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 43 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex ... 2023-10-21T04:37:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex ... 2023-10-21T04:37:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex ... 2023-10-21T04:37:01.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index destroyed 2023-10-21T04:37:01.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.832+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.832+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.835+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.835+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 49 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.838+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.838+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:37:01.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex started 2023-10-21T04:37:01.839+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.838+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.841+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.841+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:37:01.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex started 2023-10-21T04:37:01.842+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 6 Warmup took 40.764672ms 2023-10-21T04:37:01.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.842+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.842+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 6 WriterThreads 1 2023-10-21T04:37:01.842+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index Index: 3580008349810282285 Partition: 6 2023-10-21T04:37:01.842+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.842+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.843+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:01.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.843+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.843+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:01.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 44 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.861+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex started 2023-10-21T04:37:01.861+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.862+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.867+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex started 2023-10-21T04:37:01.867+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.867+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.868+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:37:01.868+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.868+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.868+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.868+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:37:01.868+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.868+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:37:01.868+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:37:01.868+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex stopped 2023-10-21T04:37:01.869+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.869+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.869+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.869+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:37:01.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex closed 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:37:01.876+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.876+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:37:01.876+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:37:01.876+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex stopped 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.876+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.890+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:37:01.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex closed 2023-10-21T04:37:01.890+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex ... 2023-10-21T04:37:01.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 44 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex ... 2023-10-21T04:37:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex ... 2023-10-21T04:37:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex ... 2023-10-21T04:37:01.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index destroyed 2023-10-21T04:37:01.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.903+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.903+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.910+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.910+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 50 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.915+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.915+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:37:01.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex started 2023-10-21T04:37:01.915+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.915+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.924+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.924+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:37:01.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex started 2023-10-21T04:37:01.924+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 7 Warmup took 81.53274ms 2023-10-21T04:37:01.924+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.924+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.927+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 7 WriterThreads 1 2023-10-21T04:37:01.927+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index Index: 3580008349810282285 Partition: 7 2023-10-21T04:37:01.928+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.929+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.929+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:01.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.930+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.930+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:01.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 46 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.937+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex started 2023-10-21T04:37:01.938+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.938+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 45 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.943+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex started 2023-10-21T04:37:01.944+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:37:01.944+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.944+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.944+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.944+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.944+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.945+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:37:01.946+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.946+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:37:01.946+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:37:01.946+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex stopped 2023-10-21T04:37:01.946+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.946+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.946+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.946+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.953+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:37:01.954+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex closed 2023-10-21T04:37:01.954+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:01.954+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.955+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.955+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:01.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:37:01.955+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.955+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:37:01.955+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:37:01.955+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex stopped 2023-10-21T04:37:01.955+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.955+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.955+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.955+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.961+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:01.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:37:01.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex closed 2023-10-21T04:37:01.961+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:01.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex ... 2023-10-21T04:37:01.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex ... 2023-10-21T04:37:01.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex sucessfully destroyed 2023-10-21T04:37:01.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex ... 2023-10-21T04:37:01.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex ... 2023-10-21T04:37:01.970+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.970+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index destroyed 2023-10-21T04:37:01.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.974+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.974+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.974+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:01.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.977+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:01.977+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:01.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 51 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.980+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.980+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:37:01.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex started 2023-10-21T04:37:01.980+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.980+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.983+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.983+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:37:01.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex started 2023-10-21T04:37:01.984+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 8 Warmup took 55.129368ms 2023-10-21T04:37:01.984+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 8 WriterThreads 1 2023-10-21T04:37:01.984+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index Index: 3580008349810282285 Partition: 8 2023-10-21T04:37:01.984+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:01.984+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:01.984+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.984+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.984+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:01.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:01.985+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:01.985+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:01.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 38 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:01.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:01.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:01.993+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:01.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex started 2023-10-21T04:37:01.993+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.993+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:01.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:01.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:01.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:01.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex started 2023-10-21T04:37:01.994+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:37:01.994+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:01.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:37:01.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:01.994+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:37:01.994+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:37:01.994+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:01.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex stopped 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:01.994+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:01.995+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:02.007+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:02.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:37:02.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex closed 2023-10-21T04:37:02.007+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:02.007+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:02.007+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:02.008+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:02.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:37:02.008+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:02.008+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:37:02.008+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:37:02.008+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:02.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex stopped 2023-10-21T04:37:02.008+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:02.008+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:02.008+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:02.008+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:02.018+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:02.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:37:02.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex closed 2023-10-21T04:37:02.019+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:02.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex ... 2023-10-21T04:37:02.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex ... 2023-10-21T04:37:02.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex sucessfully destroyed 2023-10-21T04:37:02.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex ... 2023-10-21T04:37:02.025+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex ... 2023-10-21T04:37:02.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index destroyed 2023-10-21T04:37:02.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.034+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:02.034+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:02.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.041+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:02.041+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:02.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 52 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:02.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.047+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:02.048+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:37:02.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex started 2023-10-21T04:37:02.048+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.048+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:02.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.051+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:02.051+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:37:02.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex started 2023-10-21T04:37:02.051+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.051+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 1 Warmup took 67.389548ms 2023-10-21T04:37:02.051+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.052+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 1 WriterThreads 1 2023-10-21T04:37:02.052+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index Index: 3580008349810282285 Partition: 1 2023-10-21T04:37:02.052+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 3580008349810282285 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.052+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:02.053+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:37:02.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.053+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:37:02.053+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 40 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 39 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.061+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:02.061+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex started 2023-10-21T04:37:02.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex started 2023-10-21T04:37:02.061+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.061+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:37:02.061+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.061+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.061+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:37:02.061+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:02.061+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.062+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:02.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:37:02.062+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:02.062+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:37:02.062+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:37:02.062+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:02.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex stopped 2023-10-21T04:37:02.062+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:02.062+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:02.062+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:02.062+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:02.062+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:02.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:37:02.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex closed 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:02.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:37:02.073+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:02.073+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:37:02.073+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:37:02.073+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:02.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex stopped 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:02.073+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:02.080+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:02.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:37:02.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex closed 2023-10-21T04:37:02.080+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:02.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex ... 2023-10-21T04:37:02.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 39 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex ... 2023-10-21T04:37:02.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex sucessfully destroyed 2023-10-21T04:37:02.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex ... 2023-10-21T04:37:02.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 40 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex ... 2023-10-21T04:37:02.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index destroyed 2023-10-21T04:37:02.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.089+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:02.090+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:02.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:02.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.092+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:02.092+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:02.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:02.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.095+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:02.095+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:37:02.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex started 2023-10-21T04:37:02.095+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.095+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 53 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:02.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.098+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:02.098+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:37:02.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex started 2023-10-21T04:37:02.098+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 3580008349810282285 PartitionId 2 Warmup took 45.938472ms 2023-10-21T04:37:02.098+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.098+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.098+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 3580008349810282285 partitionId 2 WriterThreads 1 2023-10-21T04:37:02.098+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 3580008349810282285 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index Index: 3580008349810282285 Partition: 2 2023-10-21T04:37:02.099+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 3580008349810282285 2023-10-21T04:37:02.099+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3580008349810282285 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:37:02.099+05:30 [Info] StorageMgr::openSnapshot IndexInst:3580008349810282285 Partition:7 No Snapshot Found. 2023-10-21T04:37:02.099+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 3580008349810282285 Adding Nil Snapshot. 2023-10-21T04:37:02.099+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {3580008349810282285 {16337157693634799600 idx_partitioned_defer plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 false [`balance`] N1QL KEY [false] true true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 0 0 0 0xc006ee4f30 [] 2 0 false plasma 0} map[1:{{1 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006dfd6a0} 2:{{2 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc00710c280} 3:{{3 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006e24d90} 4:{{4 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006e8efa0} 5:{{5 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006d87dd0} 6:{{6 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006dfc570} 7:{{7 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006d9fc50} 8:{{8 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006dfcef0}] 0xc007210580 0xc006ecec00} 2023-10-21T04:37:02.099+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.100+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 242 2023-10-21T04:37:02.100+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 18 count: 125 metaSz: 2508 rp [1] sn: 19 count: 125 metaSz: 2510 rp [2] sn: 20 count: 125 metaSz: 2510 rp [3] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:37:02.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.101+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:37:02.101+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 19 count: 125 metaSz: 2510 rp [1] sn: 20 count: 125 metaSz: 2510 rp [2] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:37:02.101+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 135990 recovery UsedSpace 157358 2023-10-21T04:37:02.101+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 150276 recovery UsedSpace 161960 2023-10-21T04:37:02.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:37:02.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.107+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:37:02.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex started 2023-10-21T04:37:02.107+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.107+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:37:02.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.108+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:37:02.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex started 2023-10-21T04:37:02.109+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.109+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.109+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:37:02.138+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 3 Warmup took 38.951726ms 2023-10-21T04:37:02.139+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 3 WriterThreads 1 2023-10-21T04:37:02.139+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index Index: 7516828288031590951 Partition: 3 2023-10-21T04:37:02.139+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.140+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 240 2023-10-21T04:37:02.140+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 17 count: 125 metaSz: 2508 rp [1] sn: 18 count: 125 metaSz: 2510 rp [2] sn: 19 count: 125 metaSz: 2510 rp [3] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:37:02.140+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 161991 recovery UsedSpace 161952 2023-10-21T04:37:02.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.141+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:37:02.141+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 18 count: 125 metaSz: 2510 rp [1] sn: 19 count: 125 metaSz: 2510 rp [2] sn: 720006 count: 125 metaSz: 2510 2023-10-21T04:37:02.141+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 149280 recovery UsedSpace 157358 2023-10-21T04:37:02.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:37:02.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:37:02.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.155+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:37:02.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex started 2023-10-21T04:37:02.155+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.155+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:37:02.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex started 2023-10-21T04:37:02.157+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:37:02.157+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.155+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.157+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.199+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 4 Warmup took 59.032143ms 2023-10-21T04:37:02.199+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 4 WriterThreads 1 2023-10-21T04:37:02.199+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index Index: 7516828288031590951 Partition: 4 2023-10-21T04:37:02.199+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.200+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:37:02.200+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 20 count: 123 metaSz: 2508 rp [1] sn: 21 count: 123 metaSz: 2509 rp [2] sn: 22 count: 123 metaSz: 2510 rp [3] sn: 720006 count: 123 metaSz: 2510 2023-10-21T04:37:02.200+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 158408 recovery UsedSpace 161432 2023-10-21T04:37:02.200+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.200+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:37:02.200+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 22 count: 123 metaSz: 2510 rp [1] sn: 720006 count: 123 metaSz: 2510 2023-10-21T04:37:02.200+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 149220 recovery UsedSpace 165572 2023-10-21T04:37:02.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:37:02.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.211+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:37:02.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex started 2023-10-21T04:37:02.211+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.211+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:37:02.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.213+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:37:02.213+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex started 2023-10-21T04:37:02.213+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:37:02.213+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.214+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.240+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 5 Warmup took 40.745663ms 2023-10-21T04:37:02.241+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 5 WriterThreads 1 2023-10-21T04:37:02.241+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index Index: 7516828288031590951 Partition: 5 2023-10-21T04:37:02.241+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.241+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 64 2023-10-21T04:37:02.241+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 19 count: 122 metaSz: 2508 rp [1] sn: 20 count: 122 metaSz: 2509 rp [2] sn: 21 count: 122 metaSz: 2510 rp [3] sn: 720006 count: 122 metaSz: 2510 2023-10-21T04:37:02.242+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 158158 recovery UsedSpace 161432 2023-10-21T04:37:02.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.242+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:02.242+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 21 count: 122 metaSz: 2510 rp [1] sn: 720006 count: 122 metaSz: 2510 2023-10-21T04:37:02.242+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 148920 recovery UsedSpace 165572 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex started 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.249+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.249+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:37:02.249+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex started 2023-10-21T04:37:02.257+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:37:02.257+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.257+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.282+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 6 Warmup took 40.811886ms 2023-10-21T04:37:02.282+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 6 WriterThreads 1 2023-10-21T04:37:02.282+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index Index: 7516828288031590951 Partition: 6 2023-10-21T04:37:02.282+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.283+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 240 2023-10-21T04:37:02.283+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 21 count: 136 metaSz: 2510 rp [1] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:37:02.283+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 131458 recovery UsedSpace 166280 2023-10-21T04:37:02.283+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.283+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:02.283+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 21 count: 136 metaSz: 2510 rp [1] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:37:02.283+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 151270 recovery UsedSpace 165572 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.289+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:37:02.289+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex started 2023-10-21T04:37:02.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex started 2023-10-21T04:37:02.289+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.289+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:37:02.289+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.289+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.290+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.325+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 7 Warmup took 42.124856ms 2023-10-21T04:37:02.325+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 7 WriterThreads 1 2023-10-21T04:37:02.325+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index Index: 7516828288031590951 Partition: 7 2023-10-21T04:37:02.325+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.325+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.326+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053364 memSz 64 2023-10-21T04:37:02.326+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 21 count: 106 metaSz: 2508 rp [1] sn: 720006 count: 106 metaSz: 2508 2023-10-21T04:37:02.326+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 121383 recovery UsedSpace 169668 2023-10-21T04:37:02.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.326+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057426 memSz 64 2023-10-21T04:37:02.326+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 21 count: 106 metaSz: 2508 rp [1] sn: 720006 count: 106 metaSz: 2508 2023-10-21T04:37:02.327+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 145960 recovery UsedSpace 169668 2023-10-21T04:37:02.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:37:02.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.333+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:37:02.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex started 2023-10-21T04:37:02.333+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.333+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:37:02.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.336+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:37:02.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex started 2023-10-21T04:37:02.336+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:37:02.336+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.337+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.349+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 8 Warmup took 23.765128ms 2023-10-21T04:37:02.349+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 8 WriterThreads 1 2023-10-21T04:37:02.350+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index Index: 7516828288031590951 Partition: 8 2023-10-21T04:37:02.350+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.351+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 240 2023-10-21T04:37:02.351+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 19 count: 127 metaSz: 2508 rp [1] sn: 20 count: 127 metaSz: 2510 rp [2] sn: 21 count: 127 metaSz: 2510 rp [3] sn: 720006 count: 127 metaSz: 2510 2023-10-21T04:37:02.351+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 163958 recovery UsedSpace 162140 2023-10-21T04:37:02.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.352+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:02.352+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 21 count: 127 metaSz: 2510 rp [1] sn: 720006 count: 127 metaSz: 2510 2023-10-21T04:37:02.352+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 149580 recovery UsedSpace 165572 2023-10-21T04:37:02.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:37:02.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:37:02.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.359+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:37:02.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex started 2023-10-21T04:37:02.359+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.359+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.360+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:37:02.360+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex started 2023-10-21T04:37:02.360+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:37:02.360+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.360+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.376+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 1 Warmup took 25.825289ms 2023-10-21T04:37:02.377+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 1 WriterThreads 1 2023-10-21T04:37:02.377+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index Index: 7516828288031590951 Partition: 1 2023-10-21T04:37:02.377+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7516828288031590951 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:02.378+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: after recovery: resident ratio 0 requestQuota 1057276 memSz 64 2023-10-21T04:37:02.378+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 22 count: 136 metaSz: 2510 rp [1] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:37:02.378+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: doInit: data UsedSpace 151560 recovery UsedSpace 165572 2023-10-21T04:37:02.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:02.378+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: after recovery: resident ratio 0 requestQuota 1053288 memSz 242 2023-10-21T04:37:02.379+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: RP version: 17 numRPs: 4 rp [0] sn: 20 count: 136 metaSz: 2508 rp [1] sn: 21 count: 136 metaSz: 2510 rp [2] sn: 22 count: 136 metaSz: 2510 rp [3] sn: 720006 count: 136 metaSz: 2510 2023-10-21T04:37:02.379+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: doInit: data UsedSpace 165908 recovery UsedSpace 162150 2023-10-21T04:37:02.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:37:02.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:02.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:02.396+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:37:02.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex started 2023-10-21T04:37:02.396+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.396+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:37:02.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:02.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:02.397+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:37:02.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex started 2023-10-21T04:37:02.397+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:37:02.398+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.398+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.413+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7516828288031590951 PartitionId 2 Warmup took 35.347324ms 2023-10-21T04:37:02.413+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7516828288031590951 partitionId 2 WriterThreads 1 2023-10-21T04:37:02.413+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7516828288031590951 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index Index: 7516828288031590951 Partition: 2 2023-10-21T04:37:02.414+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7516828288031590951 2023-10-21T04:37:02.414+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7516828288031590951 Partitions [6 7 8 1 2 3 4 5] 2023-10-21T04:37:02.415+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:7 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:37:02.416+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:8 Attempting to open snapshot (SnapshotInfo: count:106 committed:false) 2023-10-21T04:37:02.417+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:1 Attempting to open snapshot (SnapshotInfo: count:127 committed:false) 2023-10-21T04:37:02.418+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:2 Attempting to open snapshot (SnapshotInfo: count:136 committed:false) 2023-10-21T04:37:02.419+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:3 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:37:02.421+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:4 Attempting to open snapshot (SnapshotInfo: count:125 committed:false) 2023-10-21T04:37:02.422+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:5 Attempting to open snapshot (SnapshotInfo: count:123 committed:false) 2023-10-21T04:37:02.423+05:30 [Info] StorageMgr::openSnapshot IndexInst:7516828288031590951 Partition:6 Attempting to open snapshot (SnapshotInfo: count:122 committed:false) 2023-10-21T04:37:02.423+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {7516828288031590951 {16961575949607441659 idx_partitioned plasma bucket_1 1fc3689e709c62b9ee184f2ec13e6f43 false [`emailid`] N1QL KEY [false] false true [] false false 1 [(meta().`id`)] false 0 {true 1 0 0} _default c1 0 c false false true 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc0072fe840 [] 2 0 false plasma 0} map[1:{{1 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc00729b660} 2:{{2 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc0071bea80} 3:{{3 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006e258f0} 4:{{4 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc007225ad0} 5:{{5 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc006f61d10} 6:{{6 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc0071be1a0} 7:{{7 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc007275670} 8:{{8 2 [:9123] [4718895791748289908 15531034489713578274]} 0xc00733ae90}] 0xc00750cc80 0xc007107b60} 2023-10-21T04:37:02.423+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17434304260397056806 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:02.423+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:02.424+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: after recovery: resident ratio 0 requestQuota 1055176 memSz 64 2023-10-21T04:37:02.424+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 1080010 count: 6 metaSz: 2493 rp [1] sn: 1080012 count: 6 metaSz: 2494 2023-10-21T04:37:02.424+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: doInit: data UsedSpace 125720 recovery UsedSpace 186528 2023-10-21T04:37:02.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:37:02.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:02.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:02.435+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:37:02.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex started 2023-10-21T04:37:02.435+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:37:02.435+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:02.435+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:02.451+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Warmup took 27.599943ms 2023-10-21T04:37:02.451+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17434304260397056806 partitionId 0 WriterThreads 4 2023-10-21T04:37:02.451+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17434304260397056806 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index Index: 17434304260397056806 Partition: 0 2023-10-21T04:37:02.451+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17434304260397056806 2023-10-21T04:37:02.451+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17434304260397056806 Partitions [0] 2023-10-21T04:37:02.453+05:30 [Info] StorageMgr::openSnapshot IndexInst:17434304260397056806 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:37:02.453+05:30 [Info] ScanCoordinator::handleAddIndexInstance &{0xc0003589c0 ScanCoordinator {17434304260397056806 {3381524789655566900 #primary plasma bucket_%2 b9538b29d7e025c819e734b646db039c true [] N1QL SINGLE [] false false [] false false 1 [] false 0 {true 1 0 0} _system _query 8 9 false false false 0 0 0 0 0 0 0 0 [] [] 0 0 0 map[] map[]} 4 0 1 0xc0075cba70 [] 1 1 false plasma 0} map[0:{{0 1 [:9123] [17406544375239214575]} 0xc0075c0d70}] 0xc007615900 0xc00758cd20} 2023-10-21T04:37:02.455+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:37:02.455+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:02.455+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:37:02.455+05:30 [Info] RebalanceServiceManager::recoverRebalance: Init Pending Cleanup 2023-10-21T04:37:02.456+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:37:02.469+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:37:02.471+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:37:02.471+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:37:02.474+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:37:02.474+05:30 [Info] Setting log level to Info 2023-10-21T04:37:02.474+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:37:02.523+05:30 [Info] RebalanceServiceManager::doRecoverRebalance Found Global Rebalance Token &{18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:37:02.523+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:37:02.523+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:37:02.523+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:37:02.526+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {18194302e3f10df59bd076e1c4142d0b f3a5b85675e4f53a445d2e873c593537 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:37:02.568+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:37:02.622+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 55822 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:37:02.657+05:30 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2023-10-21T04:37:02.657+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:37:02.658+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:37:02.658+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:1]] 2023-10-21T04:37:02.659+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:2]] 2023-10-21T04:37:02.659+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3]] 2023-10-21T04:37:02.659+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:37:02.660+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] [4718895791748289908 15531034489713578274]} 2:{2 2 [:9123] [4718895791748289908 15531034489713578274]} 3:{3 2 [:9123] [4718895791748289908 15531034489713578274]} 4:{4 2 [:9123] [4718895791748289908 15531034489713578274]} 5:{5 2 [:9123] [4718895791748289908 15531034489713578274]} 6:{6 2 [:9123] [4718895791748289908 15531034489713578274]} 7:{7 2 [:9123] [4718895791748289908 15531034489713578274]} 8:{8 2 [:9123] [4718895791748289908 15531034489713578274]}] 8 KEY 0} InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] [4718895791748289908]}] 1 SINGLE 0} InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [4718895791748289908 15531034489713578274]}] 1 SINGLE 0}] Async: true SessionId: 55822 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 26 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 30 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 21 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 24 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} NumVBuckets: 64 2023-10-21T04:37:02.665+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:37:02.665+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:37:02.668+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 43196 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 28 28 28} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 20 20 20} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 19 19 19} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 28 0 28} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 34 34 34} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 22 22 22} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:37:02.668+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:02.668+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:02.669+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:37:02.669+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1 bucket_1:3]] 2023-10-21T04:37:02.669+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:3]] 2023-10-21T04:37:02.669+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3]] 2023-10-21T04:37:02.669+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:37:02.669+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 17434304260397056806 Defn: DefnId: 3381524789655566900 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [17406544375239214575]}] 1 SINGLE 0} InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] [17406544375239214575 10170029854712240877]} 2:{2 1 [:9123] [17406544375239214575 10170029854712240877]} 3:{3 1 [:9123] [17406544375239214575 10170029854712240877]} 4:{4 1 [:9123] [17406544375239214575 10170029854712240877]} 5:{5 1 [:9123] [17406544375239214575 10170029854712240877]} 6:{6 1 [:9123] [17406544375239214575 10170029854712240877]} 7:{7 1 [:9123] [17406544375239214575 10170029854712240877]} 8:{8 1 [:9123] [17406544375239214575 10170029854712240877]}] 8 KEY 0} InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] [17406544375239214575 10170029854712240877]}] 1 SINGLE 0}] Async: true SessionId: 43196 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 29 29} { 1 93622f3f2a49 2 28 28 28} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 20 20 20} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 19 19 19} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 28 0 28} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 34 34 34} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 22 22 22} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} NumVBuckets: 64 2023-10-21T04:37:02.670+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0001b0e40 } bucket_1:{0xc004dda3c0 }] 2023-10-21T04:37:02.671+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 Instances [indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : all deamons started 2023-10-21T04:37:02.671+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.671+05:30 [Info] ScanCoordinator::initialize rollback times on indexer resume: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:02.671+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:37:02.672+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.673+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:37:02.674+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:37:02.674+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:37:02.674+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.674+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data(shard17406544375239214575) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.675+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.675+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery(shard17406544375239214575) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.675+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.675+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data(shard4718895791748289908) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.675+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery(shard4718895791748289908) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.676+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:37:02.677+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:37:02.677+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.677+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.677+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery(shard10170029854712240877) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.677+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:37:02.672+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data(shard10170029854712240877) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.678+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery(shard15531034489713578274) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.678+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data(shard15531034489713578274) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:37:02.684+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:37:02.684+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:37:02.684+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:37:02.690+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emailid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:37:02.708+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:35652" doAuth() ... 2023-10-21T04:37:02.709+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:35652" auth successful 2023-10-21T04:37:02.709+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:35652" +1 2023-10-21T04:37:02.739+05:30 [Info] security context initialized 2023-10-21T04:37:02.739+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9118:9118 9119:9119 9120:19120 9121:9121 9122:9122 9123:9123 9124:9124 9125:9125 9126:19126 9127:9127 9128:9128 9129:9129 9499:19499 9999:9999] 2023-10-21T04:37:02.742+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:37:02.783+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 2 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:37:02.783+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:02.783+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:02.783+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:37:02.800+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39192 2023-10-21T04:37:02.803+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:37:02.828+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:37:02.828+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:37:02.830+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:37:02.833+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:37:02.833+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:37:02.837+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 27.418512ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:02.837+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:37:02.837+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:37:02.838+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 230.951µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:02.844+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:37:02.865+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:37:02.870+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:37:02.871+05:30 [Info] initialized currmeta 18 force true 2023-10-21T04:37:02.877+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:37:02.878+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:37:02.878+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:37:02.879+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:37:02.880+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50776" doAuth() ... 2023-10-21T04:37:02.881+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:50776" doReceive() ... 2023-10-21T04:37:02.887+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:37:02.887+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:37:02.887+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:37:02.888+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:37:02.890+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:37:02.890+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:37:02.890+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:37:02.890+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:37:02.890+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:37:02.890+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:37:02.890+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:37:02.893+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:37:02.893+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:37:02.893+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.58µs 2023-10-21T04:37:02.894+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:37:02.894+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:37:02.894+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:37:02.897+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:37:02.897+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:37:02.897+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:37:02.897+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:37:02.897+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 49.638µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:02.899+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:37:02.899+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:37:02.900+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{18194302e3f10df59bd076e1c4142d0b 7060000 } 2023-10-21T04:37:02.911+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 19.736893ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:02.963+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:56888:172.31.5.112:12000] 2023-10-21T04:37:03.113+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56922:172.31.5.112:12000] 2023-10-21T04:37:03.118+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56920:172.31.5.112:12000] 2023-10-21T04:37:03.163+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56924:172.31.5.112:12000] 2023-10-21T04:37:03.167+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56928:172.31.5.112:12000] 2023-10-21T04:37:03.195+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56926:172.31.5.112:12000] 2023-10-21T04:37:03.221+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56930:172.31.5.112:12000] 2023-10-21T04:37:03.235+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56932:172.31.5.112:12000] 2023-10-21T04:37:03.264+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56934:172.31.5.112:12000] 2023-10-21T04:37:03.284+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56938:172.31.5.112:12000] 2023-10-21T04:37:03.310+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56936:172.31.5.112:12000] 2023-10-21T04:37:03.324+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 2023-10-21T04:37:03.325+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1361754384643230279:[0] 10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 2023-10-21T04:37:03.327+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1361754384643230279:[0] 10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:37:03.328+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1361754384643230279:[0] 3345094951175719474:[8 1 2 3 4 5 6 7] 10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 {Index inst : [partitions]}: map[9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:37:03.329+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1361754384643230279:[0] 3345094951175719474:[8 1 2 3 4 5 6 7] 10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 {Index inst : [partitions]}: map[2535881715450691825:[0] 9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:37:03.330+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1361754384643230279:[0] 3345094951175719474:[8 1 2 3 4 5 6 7] 10601152754683910907:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 {Index inst : [partitions]}: map[2535881715450691825:[0] 6043349427976659583:[4 5 6 7 8 1 2 3] 9702649553456094532:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:37:03.359+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 1949031200982 23 55822 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:03.363+05:30 [Info] NewTsVbuuidCached: bucket_1, 64 TsVbuuidPoolMap is updated from: map[] to: map[64:0xc000676db0] 2023-10-21T04:37:03.376+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 245782287380001 27 55822 172.31.5.112:9000. HWT [26-27,27]. 2023-10-21T04:37:03.378+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 244791485011286 23 55822 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:03.378+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 205201940883916 24 55822 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:03.382+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 134409829206048 26 55822 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:37:03.382+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 177815184127237 17 55822 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:37:03.382+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 93079113162334 18 55822 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:37:03.386+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 58522600746999 24 55822 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:03.388+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 90115353725987 21 55822 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:03.388+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 167580527121081 23 55822 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:03.388+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 52522903947059 28 55822 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:03.392+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 85302001459467 21 55822 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:03.392+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 142387448675642 30 55822 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:37:03.392+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 54236768606397 22 55822 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:03.395+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 161230548567164 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.395+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 123957315902602 30 55822 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:37:03.395+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 160381752747972 18 55822 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:37:03.398+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 163460438181870 34 55822 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:37:03.400+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 150690956328758 26 55822 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:37:03.401+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 151273673850546 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.403+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 213108110408291 23 55822 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:03.403+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 147946970349859 24 55822 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:03.408+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 53829314402679 32 55822 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:37:03.408+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 114823206239184 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.408+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 258671969827141 19 55822 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:03.408+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 19989312661772 15 55822 172.31.5.112:9000. HWT [0-15,15]. 2023-10-21T04:37:03.410+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 232028101514735 19 55822 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:03.410+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 68847799765163 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.413+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 8305763409275 21 55822 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:37:03.413+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 25510109770064 28 55822 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:03.416+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 124811039198764 18 55822 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:37:03.417+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 207489375637709 12 55822 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:37:03.420+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 107407811917992 20 55822 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:37:03.422+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 33310272161722 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.422+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 117298435979365 24 55822 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:03.426+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 100746255343609 19 55822 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:03.426+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 203503670520010 24 55822 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:03.426+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno cache..., established connections: map[172.31.5.112:56954:172.31.5.112:12000] 2023-10-21T04:37:03.429+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 47156288300713 24 55822 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:03.430+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 228857796432725 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.432+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 158357249947806 21 55822 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:03.436+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 236904976023311 22 55822 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:03.436+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 104136033125373 29 55822 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:37:03.441+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 196269910725961 26 55822 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:37:03.444+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 276463747117997 25 55822 172.31.5.112:9000. HWT [24-25,25]. 2023-10-21T04:37:03.444+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 124310544527292 21 55822 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:03.449+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 172365063201867 21 55822 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:03.451+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 212884822297038 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.459+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 213334533059126 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.461+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 254944616021293 24 55822 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:03.463+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 219528837936411 25 55822 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:03.465+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 80145536651832 17 55822 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:37:03.465+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 65885584526743 20 55822 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:37:03.467+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 79077905592013 17 55822 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:37:03.467+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 246959094981553 34 55822 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:37:03.471+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 106620692996001 23 55822 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:03.473+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 142324128597367 22 55822 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:03.473+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 223881084081370 17 55822 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:37:03.473+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 194568213794435 29 55822 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:37:03.478+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1 InstanceIds [3643393537124379309 4443661069773468873 7516828288031590951] 2023-10-21T04:37:03.479+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 55822 2023-10-21T04:37:03.479+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 55822 2023-10-21T04:37:03.479+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:37:03.479+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:37:03.479+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:37:03.479+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:37:03.479+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 55822 2023-10-21T04:37:03.479+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 55822 STREAM_ACTIVE 2023-10-21T04:37:03.479+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:37:03.479+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 36300629322478 34 55822 172.31.5.112:9000. HWT [0-34,34]. 2023-10-21T04:37:03.479+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 25511316571199 16 55822 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:37:03.479+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 246582249071123 20 55822 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:37:03.479+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 55145461136105 16 55822 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:37:03.482+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 25657368013413 17 55822 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:37:03.484+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 279733370822238 23 55822 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:03.617+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56980:172.31.5.112:12000] 2023-10-21T04:37:03.640+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56982:172.31.5.112:12000] 2023-10-21T04:37:03.654+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56984:172.31.5.112:12000] 2023-10-21T04:37:03.683+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56986:172.31.5.112:12000] 2023-10-21T04:37:03.717+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56988:172.31.5.112:12000] 2023-10-21T04:37:03.732+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56992:172.31.5.112:12000] 2023-10-21T04:37:03.764+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56990:172.31.5.112:12000] 2023-10-21T04:37:03.785+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56994:172.31.5.112:12000] 2023-10-21T04:37:03.801+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56996:172.31.5.112:12000] 2023-10-21T04:37:03.828+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:56998:172.31.5.112:12000] 2023-10-21T04:37:03.830+05:30 [Info] progress stats collection done. 2023-10-21T04:37:03.830+05:30 [Info] send progress stats to clients 2023-10-21T04:37:03.833+05:30 [Info] Indexer::ReadMemstats Time Taken 91.281µs 2023-10-21T04:37:03.841+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:37:04.467+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 274143649547516 29 43196 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:37:04.467+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 162049908746825 28 43196 172.31.5.112:9000. HWT [28-28,28]. 2023-10-21T04:37:04.471+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 206082810619393 24 43196 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:04.475+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 120349894144442 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.475+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 143687065777286 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.479+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 212967951483349 28 43196 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:04.479+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 116493903186721 22 43196 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:04.480+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:37:04.480+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:37:04.481+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 78446412189949 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.483+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 263088038522959 18 43196 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:37:04.485+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 136568163328749 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.491+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 261951617336488 35 43196 172.31.5.112:9000. HWT [0-35,35]. 2023-10-21T04:37:04.495+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 127746572208200 19 43196 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:04.502+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 58733680850130 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.505+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 188740876924377 22 43196 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:04.505+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 173457131919400 12 43196 172.31.5.112:9000. HWT [0-12,12]. 2023-10-21T04:37:04.506+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 277338544220217 21 43196 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:04.517+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 206088069380492 13 43196 172.31.5.112:9000. HWT [0-13,13]. 2023-10-21T04:37:04.517+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 34510046292135 19 43196 172.31.5.112:9000. HWT [19-19,19]. 2023-10-21T04:37:04.517+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 250426459207023 28 43196 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:04.521+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 245247859472006 20 43196 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:37:04.521+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 274158831830257 20 43196 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:37:04.525+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 125696667341112 27 43196 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:37:04.525+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 35716737784090 24 43196 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:04.525+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 191500187207978 21 43196 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:04.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 106231614376481 21 43196 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:04.537+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 118920785117980 21 43196 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:37:04.537+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 137256376714252 22 43196 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:04.541+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 5906787184229 20 43196 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:37:04.542+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 146375746634523 17 43196 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:37:04.545+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 112482982607415 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.548+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 41101832875819 19 43196 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:04.551+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 241391494503904 28 43196 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:04.552+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 121450709505223 25 43196 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:04.555+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 216238759475962 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.558+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 127702263378100 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.558+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 168446351781198 25 43196 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:04.562+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 100008922352935 17 43196 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:37:04.562+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 169268048765249 19 43196 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:04.564+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 245900669851618 19 43196 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:04.565+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 149033089785397 28 43196 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:04.569+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 106297365959267 27 43196 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:37:04.570+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 88718752743571 25 43196 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:04.575+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 39045953044628 18 43196 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:37:04.575+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 160399698726237 28 43196 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:04.579+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 217237947818243 20 43196 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:37:04.583+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 110286285845649 34 43196 172.31.5.112:9000. HWT [34-34,34]. 2023-10-21T04:37:04.583+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 264001907139144 22 43196 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:04.583+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 63737850123332 19 43196 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:04.583+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 132470209691538 26 43196 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:37:04.583+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 162870112426804 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.585+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 266604622117081 26 43196 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:37:04.588+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 32274016651656 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.588+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 84581802233088 22 43196 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:37:04.590+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 225662991941187 26 43196 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:37:04.592+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 272523968790649 22 43196 172.31.5.112:9000. HWT [22-22,22]. 2023-10-21T04:37:04.593+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 241607113601772 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.597+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 197957220074784 24 43196 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:37:04.597+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 77598347829810 19 43196 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:37:04.599+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 97677619676504 31 43196 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:37:04.599+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 136038310030103 28 43196 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:37:04.602+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 72072509155168 23 43196 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:37:04.602+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 105415986286631 18 43196 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:37:04.607+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 27479155597282 29 43196 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:37:04.608+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 161870047388121 25 43196 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:37:04.615+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2 InstanceIds [3643393537124379309 4443661069773468873 7516828288031590951 3127033699511712809 17434304260397056806 8965923595543532942] 2023-10-21T04:37:04.615+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 43196 2023-10-21T04:37:04.615+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 43196 2023-10-21T04:37:04.615+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:37:04.615+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:37:04.615+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:37:04.615+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 43196 2023-10-21T04:37:04.615+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 43196 STREAM_ACTIVE 2023-10-21T04:37:05.383+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:37:05.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:05.392+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:05.397+05:30 [Info] switched currmeta from 18 -> 18 force true 2023-10-21T04:37:05.479+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:37:05.551+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 2 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:37:05.552+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 2 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:37:05.552+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 2 NodeUUIDs: [18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] 2023-10-21T04:37:05.552+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:37:05.552+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:37:05.555+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:37:05.556+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:37:05.616+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:37:05.616+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:37:05.827+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52430 2023-10-21T04:37:05.855+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52430 2023-10-21T04:37:06.456+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 2023-10-21T04:37:06.456+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 {Index inst : [partitions]}: map[7516828288031590951:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843222622515965, Old rollbacktime: 0 2023-10-21T04:37:06.456+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0] 17434304260397056806:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 {Index inst : [partitions]}: map[7516828288031590951:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843222622515965, Old rollbacktime: 0 2023-10-21T04:37:06.456+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0] 8965923595543532942:[3 4 5 6 7 8 1 2] 17434304260397056806:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 {Index inst : [partitions]}: map[7516828288031590951:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843222622515965, Old rollbacktime: 0 2023-10-21T04:37:06.456+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0] 8965923595543532942:[3 4 5 6 7 8 1 2] 17434304260397056806:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 {Index inst : [partitions]}: map[3643393537124379309:[0] 7516828288031590951:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843222622515965, Old rollbacktime: 0 2023-10-21T04:37:06.456+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3127033699511712809:[0] 8965923595543532942:[3 4 5 6 7 8 1 2] 17434304260397056806:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 {Index inst : [partitions]}: map[3643393537124379309:[0] 4443661069773468873:[0] 7516828288031590951:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843222622515965, Old rollbacktime: 0 2023-10-21T04:37:07.450+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39476 2023-10-21T04:37:07.451+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39478 2023-10-21T04:37:07.451+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39480 2023-10-21T04:37:16.101+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.105+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:37:16.225+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:51242" doAuth() ... 2023-10-21T04:37:16.226+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:51242" doReceive() ... 2023-10-21T04:37:16.714+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is enabled 2023-10-21T04:37:18.220+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is enabled 2023-10-21T04:37:20.205+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4443661069773468873 &{0} 2023-10-21T04:37:20.205+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4443661069773468873 2023-10-21T04:37:20.205+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:37:20.205+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:20.205+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4443661069773468873 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:37:20.206+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4443661069773468873] 2023-10-21T04:37:20.207+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4443661069773468873] 2023-10-21T04:37:20.207+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:20.207+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4443661069773468873] 2023-10-21T04:37:20.208+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [4718895791748289908 15531034489713578274]}] 1 SINGLE 0}] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:20.211+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4443661069773468873, PartitionId 0, IndexDefnId 12582561312368048736. 2023-10-21T04:37:20.211+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:20.211+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:20.211+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:20.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex] 2023-10-21T04:37:20.211+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:37:20.211+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex] 2023-10-21T04:37:20.211+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex] 2023-10-21T04:37:20.211+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex 2023-10-21T04:37:20.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex stopped 2023-10-21T04:37:20.211+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:20.213+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:20.213+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:20.213+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:20.209+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4443661069773468873 Defn: DefnId: 12582561312368048736 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] [4718895791748289908 15531034489713578274]}] 1 SINGLE 0}] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:20.213+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 8.347436ms 2023-10-21T04:37:20.218+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [4443661069773468873] 2023-10-21T04:37:20.219+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4443661069773468873 2023-10-21T04:37:20.219+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:20.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex] 2023-10-21T04:37:20.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex closed 2023-10-21T04:37:20.220+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:20.220+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:20.220+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:20.220+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:20.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex] 2023-10-21T04:37:20.220+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:37:20.222+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex] 2023-10-21T04:37:20.222+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex] 2023-10-21T04:37:20.222+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex 2023-10-21T04:37:20.222+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex stopped 2023-10-21T04:37:20.222+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:20.222+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:20.222+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:20.222+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:20.223+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:37:20.223+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 55822 2023-10-21T04:37:20.224+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 10.23759ms 2023-10-21T04:37:20.224+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 18.670889ms 2023-10-21T04:37:20.241+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:20.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex] 2023-10-21T04:37:20.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex closed 2023-10-21T04:37:20.241+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:20.241+05:30 [Info] Indexer::cleanupIndexData IndexInst 4443661069773468873 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:20.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex ... 2023-10-21T04:37:20.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 37 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex ... 2023-10-21T04:37:20.242+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 41.741257ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:20.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:20.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/docIndex sucessfully destroyed 2023-10-21T04:37:20.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex ... 2023-10-21T04:37:20.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 38 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex ... 2023-10-21T04:37:20.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:20.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_4443661069773468873_0.index destroyed 2023-10-21T04:37:20.262+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4443661069773468873, PartitionId 0, IndexDefnId 12582561312368048736. 2023-10-21T04:37:20.262+05:30 [Info] Indexer::cleanupIndexData IndexInst 4443661069773468873 Partition 0 Destroy Done 2023-10-21T04:37:20.295+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:20.296+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:20.299+05:30 [Info] switched currmeta from 18 -> 22 force true 2023-10-21T04:37:20.305+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:20.305+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:20.308+05:30 [Info] switched currmeta from 22 -> 22 force true 2023-10-21T04:37:20.418+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:37:20.422+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:20.422+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:20.423+05:30 [Info] switched currmeta from 22 -> 22 force true 2023-10-21T04:37:30.478+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 4286621850367430051 2023-10-21T04:37:30.478+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 399.67µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:30.528+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53064 2023-10-21T04:37:30.531+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53064 2023-10-21T04:37:30.547+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53064 2023-10-21T04:37:30.605+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 4286621850367430051, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:37:30.605+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 4286621850367430051, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:37:30.629+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 642374041935721654, indexDefn DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:37:30.630+05:30 [Info] Indexer::handleCreateIndex InstId: 642374041935721654 Defn: DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:37:30.632+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 642374041935721654 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:30.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:37:30.641+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data/recovery) 2023-10-21T04:37:30.642+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:30.643+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:30.650+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:37:30.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data/recovery) 2023-10-21T04:37:30.653+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:30.653+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:30.654+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:30.654+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:30.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex 2023-10-21T04:37:30.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274 2023-10-21T04:37:30.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274/data 2023-10-21T04:37:30.657+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex 2023-10-21T04:37:30.657+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex 2023-10-21T04:37:30.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex started 2023-10-21T04:37:30.657+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:30.657+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:30.657+05:30 [Info] switched currmeta from 22 -> 25 force true 2023-10-21T04:37:30.661+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:30.662+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:30.663+05:30 [Info] switched currmeta from 25 -> 25 force true 2023-10-21T04:37:30.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Assign plasmaId 55 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex 2023-10-21T04:37:30.682+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908 2023-10-21T04:37:30.683+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:30.683+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:30.687+05:30 [Info] switched currmeta from 25 -> 26 force true 2023-10-21T04:37:30.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908/data 2023-10-21T04:37:30.689+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex 2023-10-21T04:37:30.689+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex 2023-10-21T04:37:30.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex started 2023-10-21T04:37:30.690+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:30.691+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:30.691+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 642374041935721654 partitionId 0 WriterThreads 4 2023-10-21T04:37:30.691+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 642374041935721654 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index Index: 642374041935721654 Partition: 0 2023-10-21T04:37:30.693+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 642374041935721654 2023-10-21T04:37:30.693+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:30.694+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:37:30.694+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:30.694+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:30.694+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 64.060904ms 2023-10-21T04:37:30.694+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 64.139648ms 2023-10-21T04:37:30.694+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 642374041935721654, partnShardIdMap: map[0:[4718895791748289908 15531034489713578274]], indexDefn DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:37:30.710+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:30.710+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:30.714+05:30 [Info] switched currmeta from 26 -> 28 force true 2023-10-21T04:37:30.715+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [642374041935721654] &{0} 2023-10-21T04:37:30.715+05:30 [Info] Indexer::handleBuildIndex [642374041935721654] 2023-10-21T04:37:30.716+05:30 [Info] bulkUpdateRState: Index instance 642374041935721654 rstate moved to ACTIVE 2023-10-21T04:37:30.716+05:30 [Info] Indexer::handleBuildIndex Added Index: [642374041935721654] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:37:30.717+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:30.717+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:30.718+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:30.718+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:37:30.718+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:30.718+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:37:30.718+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:37:30.718+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:30.718+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:37:30.718+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:30.718+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:30.718+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:37:30.718+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:30.718+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:37:30.718+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:3] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 642374041935721654 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:37:30.718+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:37:30.719+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [ InstId: 642374041935721654 Defn: DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 41958 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:37:30.719+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:37:30.719+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:37:30.720+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 642374041935721654 Defn: DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:30.720+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 4.773324ms 2023-10-21T04:37:30.720+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 4.812339ms 2023-10-21T04:37:30.720+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [642374041935721654] 2023-10-21T04:37:30.732+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 4286621850367430051 2023-10-21T04:37:30.732+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 127.653927ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:30.734+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:37:30.736+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:37:30.744+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:30.744+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:30.749+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.911604ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:30.756+05:30 [Info] switched currmeta from 28 -> 30 force true 2023-10-21T04:37:30.783+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40748" doAuth() ... 2023-10-21T04:37:30.786+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40748" auth successful 2023-10-21T04:37:30.791+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:40748" +1 2023-10-21T04:37:30.802+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:30.803+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:30.813+05:30 [Info] switched currmeta from 30 -> 31 force true 2023-10-21T04:37:30.834+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:30.835+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:30.851+05:30 [Info] switched currmeta from 31 -> 31 force true 2023-10-21T04:37:30.915+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:39990 2023-10-21T04:37:30.929+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 8.69332ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:30.961+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 34.567µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:30.971+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39990. Reason = EOF. Cleaning Pipe. 2023-10-21T04:37:30.971+05:30 [Info] messageListener.start(): message channel closed. Remove peer da:b7:f5:6:2b:43:2f:33 as follower. 2023-10-21T04:37:31.184+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 1949031200982 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.188+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 245782287380001 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.190+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 52522903947059 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.191+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 93079113162334 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.191+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 205201940883916 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.194+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 244791485011286 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.197+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 177815184127237 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.208+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 54236768606397 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.208+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 167580527121081 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.214+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 160381752747972 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.214+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 134409829206048 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.223+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 123957315902602 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.223+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 142387448675642 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.223+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 90115353725987 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.226+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 161230548567164 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.227+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 85302001459467 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.227+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 58522600746999 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.227+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 213108110408291 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.227+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 163460438181870 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.237+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 147946970349859 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 150690956328758 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 151273673850546 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 258671969827141 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 53829314402679 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.246+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 19989312661772 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.246+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 114823206239184 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.246+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 68847799765163 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.247+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 232028101514735 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.247+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 25510109770064 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.250+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 124811039198764 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.251+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 33310272161722 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.256+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 207489375637709 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.256+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 107407811917992 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.256+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 8305763409275 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.259+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 100746255343609 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.261+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 228857796432725 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 47156288300713 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 203503670520010 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 117298435979365 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 236904976023311 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 158357249947806 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 104136033125373 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 124310544527292 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.274+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 246959094981553 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.274+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 196269910725961 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.279+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 65885584526743 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.281+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 79077905592013 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.281+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 172365063201867 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.293+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 246582249071123 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.293+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 276463747117997 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.293+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 212884822297038 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.297+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 213334533059126 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.298+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 142324128597367 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.298+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 254944616021293 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.302+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 55145461136105 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.304+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 219528837936411 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.305+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 194568213794435 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.310+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 80145536651832 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.313+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 25511316571199 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.314+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 279733370822238 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.314+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 106620692996001 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.316+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 223881084081370 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.320+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 36300629322478 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.321+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 25657368013413 0 41958 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:31.334+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_1:_default:c1 InstanceIds [642374041935721654] 2023-10-21T04:37:31.334+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 41958 2023-10-21T04:37:31.334+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:37:31.334+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:37:31.334+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs , fetchKVSeq false 2023-10-21T04:37:31.334+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:37:31.334+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:37:31.334+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:37:31.335+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:37:31.335+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:31.430+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:37:31.596+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:37:31.596+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:37:31.601+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40016 2023-10-21T04:37:31.601+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:37:31.625+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 19.677992ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:31.625+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:37:31.625+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:37:31.625+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:37:31.638+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:37:31.665+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:37:31.665+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:37:31.665+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:37:31.665+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:37:31.665+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:37:31.665+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:37:31.665+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:40016->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:37:31.665+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:36392->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:37:31.666+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40016. Reason = EOF. Cleaning Pipe. 2023-10-21T04:37:31.666+05:30 [Info] messageListener.start(): message channel closed. Remove peer 8:9c:61:b:8d:9:2d:7b as follower. 2023-10-21T04:37:31.690+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:37:31.722+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:31.729+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:31.731+05:30 [Info] switched currmeta from 31 -> 32 force true 2023-10-21T04:37:31.733+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 642374041935721654 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:37:31.733+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 642374041935721654 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.837µs SnapOpenDur 85.532µs 2023-10-21T04:37:31.733+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 642374041935721654 INDEX_STATE_CATCHUP 2023-10-21T04:37:31.733+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 642374041935721654 true 2023-10-21T04:37:31.733+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 642374041935721654 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 41958 BuildTS: [0=23,1=21,2=28,3=24,4=17,5=23,6=23,7=17,8=20,9=26,10=18,11=24,12=30,13=21,14=25,15=21,16=23,17=23,18=34,19=26,20=25,21=24,22=19,23=32,24=15,25=25,26=25,27=19,28=28,29=19,30=18,31=12,32=24,33=20,34=19,35=24,36=25,37=24,38=24,39=21,40=29,41=22,42=21,43=25,44=34,45=19,46=20,47=21,48=17,49=25,50=20,51=25,52=22,53=24,54=16,55=25,56=29,57=17,58=16,59=23,60=23,61=17,62=32,63=17] 2023-10-21T04:37:31.733+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 642374041935721654, PartitionId 0 SnapshotId 12 Creating recovery point ... 2023-10-21T04:37:31.737+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 41958 2023-10-21T04:37:31.738+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 41958 2023-10-21T04:37:31.738+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:31.738+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:31.738+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:31.739+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 642374041935721654 Defn: DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:31.739+05:30 [Info] send progress stats to clients 2023-10-21T04:37:31.739+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:37:31.739+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:37:31.739+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:37:31.739+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 642374041935721654 Defn: DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 41958 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:37:31.742+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:37:31.748+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_1 2023-10-21T04:37:31.748+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 41958 2023-10-21T04:37:31.748+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 41958 2023-10-21T04:37:31.748+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:37:31.748+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:37:31.750+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 642374041935721654 false 2023-10-21T04:37:31.750+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 642374041935721654 2023-10-21T04:37:31.750+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:37:31.750+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 41958 2023-10-21T04:37:31.753+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.866621ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:31.762+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:31.762+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:31.766+05:30 [Info] switched currmeta from 32 -> 33 force true 2023-10-21T04:37:31.770+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 642374041935721654, PartitionId 0 SnapshotId 12 Created recovery point (took 37.17831ms) 2023-10-21T04:37:32.125+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:37:32.322+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:32.322+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:32.334+05:30 [Info] switched currmeta from 33 -> 34 force true 2023-10-21T04:37:32.335+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:37:32.335+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:37:32.335+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 642374041935721654 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 41958, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 5334336579729819384 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 1c5cb4ff8d6 2 23 0 23} { 1 df89a5ee6e21 2 27 0 27} { 2 2fc4f1084733 2 28 0 28} { 3 baa14ccb29cc 2 24 0 24} { 4 54a7ab79de5e 2 18 0 18} { 5 dea2f5821556 2 23 0 23} { 6 9869e1ca52b9 2 23 0 23} { 7 a1b8d2e2b505 2 17 0 17} { 8 3153fb5398bd 2 22 0 22} { 9 7a3eba1fc820 2 26 0 26} { 10 91ddc966a7c4 2 18 0 18} { 11 3539dad42ff7 2 24 0 24} { 12 8180294c7d3a 2 30 0 30} { 13 51f59da5cc23 2 21 0 21} { 14 92a36993047c 2 25 0 25} { 15 4d94eb7ee10b 2 21 0 21} { 16 70bd0fc81c8a 2 30 0 30} { 17 c1d219383a63 2 23 0 23} { 18 94aa995a47ee 2 34 0 34} { 19 890d78ec3f36 2 26 0 26} { 20 89952597feb2 2 25 0 25} { 21 868e969d0123 2 24 0 24} { 22 eb42c2dff945 2 19 0 19} { 23 30f51d29d977 2 32 0 32} { 24 122e1fe1690c 2 15 0 15} { 25 686e5c6b7bd0 2 25 0 25} { 26 3e9de0a608ab 2 25 0 25} { 27 d307409195ef 2 19 0 19} { 28 17338907f150 2 28 0 28} { 29 78dd613dd7b 2 21 0 21} { 30 7183d5a7ce2c 2 18 0 18} { 31 bcb5e28790cd 2 12 0 12} { 32 1e4ba6f087ba 2 25 0 25} { 33 61afd479b8a8 2 20 0 20} { 34 5ba0d0c31ff9 2 19 0 19} { 35 6aaeab9bf865 2 24 0 24} { 36 d0251ba91355 2 25 0 25} { 37 b915e3fde8ca 2 24 0 24} { 38 2ae36dd026a9 2 24 0 24} { 39 90066ba1789e 2 21 0 21} { 40 5eb60f4343fd 2 29 0 29} { 41 d776bcf0ff0f 2 22 0 22} { 42 710f4dd92fbc 2 21 0 21} { 43 b281a64f1549 2 26 0 26} { 44 e09ba52163b1 2 34 0 34} { 45 fb713b70b3ad 2 25 0 25} { 46 3bec2edc8d97 2 20 0 20} { 47 9cc3de5d244b 2 21 0 21} { 48 47ebc278eecd 2 17 0 17} { 49 c19e1c3609ce 2 25 0 25} { 50 e043e75d2213 2 20 0 20} { 51 c206d10fb236 2 25 0 25} { 52 81716b20c577 2 22 0 22} { 53 e7deeb467d2d 2 24 0 24} { 54 32278d9f3ae9 2 16 0 16} { 55 c7a90a67351b 2 25 0 25} { 56 b0f571448e83 2 29 0 29} { 57 48e4563e1238 2 17 0 17} { 58 1733d0f6443f 2 16 0 16} { 59 60f890881fa1 2 23 0 23} { 60 fe6a80331e5e 2 23 0 23} { 61 cb9e607d14da 2 17 0 17} { 62 2103e61daaee 2 34 0 34} { 63 1755d24e6a65 2 17 0 17} 2023-10-21T04:37:32.335+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 642374041935721654 INDEX_STATE_ACTIVE 2023-10-21T04:37:32.335+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:37:32.335+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:37:32.335+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:37:32.335+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 0 New 0 2023-10-21T04:37:32.335+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 41958 2023-10-21T04:37:32.336+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:32.336+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:32.336+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:32.336+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 41958 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:32.336+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 642374041935721654 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:37:32.336+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:37:32.336+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:37:32.336+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:37:32.336+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 41958 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:32.336+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:37:32.336+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:16 1:14 2:21 3:17 4:10 5:16 6:16 7:10 8:13 9:19 10:11 11:17 12:23 13:14 14:18 15:14 16:16 17:16 18:27 19:19 20:18 21:17 22:12 23:25 24:8 25:18 26:18 27:12 28:21 29:12 30:11 31:5 32:17 33:13 34:12 35:17 36:18 37:17 38:17 39:14 40:22 41:15 42:14 43:18 44:27 45:12 46:13 47:14 48:10 49:18 50:13 51:18 52:15 53:17 54:9 55:18 56:22 57:10 58:9 59:16 60:16 61:10 62:25 63:10] 2023-10-21T04:37:32.337+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:23 1:21 2:28 3:24 4:17 5:23 6:23 7:17 8:20 9:26 10:18 11:24 12:30 13:21 14:25 15:21 16:23 17:23 18:34 19:26 20:25 21:24 22:19 23:32 24:15 25:25 26:25 27:19 28:28 29:19 30:18 31:12 32:24 33:20 34:19 35:24 36:25 37:24 38:24 39:21 40:29 41:22 42:21 43:25 44:34 45:19 46:20 47:21 48:17 49:25 50:20 51:25 52:22 53:24 54:16 55:25 56:29 57:17 58:16 59:23 60:23 61:17 62:32 63:17] 2023-10-21T04:37:32.337+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:37:32.337+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:40748: use of closed network connection 2023-10-21T04:37:32.337+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:40748" exit: read tcp4 127.0.0.1:9121->127.0.0.1:40748: use of closed network connection 2023-10-21T04:37:32.337+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:40748" closed ! 2023-10-21T04:37:32.338+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:37:32.338+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:40748" already gone 2023-10-21T04:37:32.338+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:32.338+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:32.338+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:32.338+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:32.338+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:37:32.338+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 642374041935721654 Defn: DefnId: 4286621850367430051 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:32.339+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 41958 2023-10-21T04:37:32.339+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:37:32.351+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.984113ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:32.366+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:37:32.366+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:37:32.366+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:37:32.366+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 41958 2023-10-21T04:37:32.366+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 41958 2023-10-21T04:37:32.366+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:37:32.367+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:32.367+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:32.370+05:30 [Info] switched currmeta from 34 -> 35 force true 2023-10-21T04:37:32.819+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1492646773239158168:[0]]. New rollbacktime: 1697843078734407204, Old rollbacktime: 0 2023-10-21T04:37:32.900+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:37:32.958+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[642374041935721654:[0]]. New rollbacktime: 1697843222622515965, Old rollbacktime: 0 2023-10-21T04:37:35.451+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:37:35.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:35.455+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:35.458+05:30 [Info] switched currmeta from 35 -> 35 force true 2023-10-21T04:37:35.555+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. err: operation_canceled 2023-10-21T04:37:35.555+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:37:35.555+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:37:35.558+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:37:35.558+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:37:35.558+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:37:35.559+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:37:43.196+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3127033699511712809 &{0} 2023-10-21T04:37:43.197+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3127033699511712809 2023-10-21T04:37:43.197+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:37:43.197+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:43.197+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3127033699511712809 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:37:43.198+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3127033699511712809] 2023-10-21T04:37:43.199+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3127033699511712809] 2023-10-21T04:37:43.199+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:43.202+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3127033699511712809] 2023-10-21T04:37:43.203+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] [17406544375239214575 10170029854712240877]}] 1 SINGLE 0}] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:43.203+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 3127033699511712809 Defn: DefnId: 14549775068266281786 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] [17406544375239214575 10170029854712240877]}] 1 SINGLE 0}] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:43.205+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 8.1605ms 2023-10-21T04:37:43.206+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0, IndexDefnId 14549775068266281786. 2023-10-21T04:37:43.206+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:43.206+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:43.206+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:43.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex] 2023-10-21T04:37:43.206+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:37:43.206+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex] 2023-10-21T04:37:43.206+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex] 2023-10-21T04:37:43.206+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex 2023-10-21T04:37:43.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex stopped 2023-10-21T04:37:43.207+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:43.207+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:43.207+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [3127033699511712809] 2023-10-21T04:37:43.207+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3127033699511712809 2023-10-21T04:37:43.208+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:43.208+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:43.212+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:37:43.212+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 43196 2023-10-21T04:37:43.212+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 6.370149ms 2023-10-21T04:37:43.212+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 15.438437ms 2023-10-21T04:37:43.214+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:43.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex] 2023-10-21T04:37:43.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex closed 2023-10-21T04:37:43.216+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:43.216+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:37:43.216+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:37:43.216+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:37:43.216+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex] 2023-10-21T04:37:43.216+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:37:43.216+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex] 2023-10-21T04:37:43.219+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex] 2023-10-21T04:37:43.219+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex 2023-10-21T04:37:43.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex stopped 2023-10-21T04:37:43.219+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:37:43.219+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:37:43.219+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:37:43.219+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:37:43.231+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:37:43.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex] 2023-10-21T04:37:43.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex closed 2023-10-21T04:37:43.232+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:37:43.232+05:30 [Info] Indexer::cleanupIndexData IndexInst 3127033699511712809 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:37:43.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex ... 2023-10-21T04:37:43.233+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 45 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex ... 2023-10-21T04:37:43.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:43.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/docIndex sucessfully destroyed 2023-10-21T04:37:43.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex ... 2023-10-21T04:37:43.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex ... 2023-10-21T04:37:43.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:43.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_3127033699511712809_0.index destroyed 2023-10-21T04:37:43.244+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3127033699511712809, PartitionId 0, IndexDefnId 14549775068266281786. 2023-10-21T04:37:43.244+05:30 [Info] Indexer::cleanupIndexData IndexInst 3127033699511712809 Partition 0 Destroy Done 2023-10-21T04:37:43.247+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 61.588495ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:43.256+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:43.257+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:43.268+05:30 [Info] switched currmeta from 35 -> 37 force true 2023-10-21T04:37:43.277+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:43.277+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:43.281+05:30 [Info] switched currmeta from 37 -> 38 force true 2023-10-21T04:37:43.311+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:43.311+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:43.317+05:30 [Info] switched currmeta from 38 -> 39 force true 2023-10-21T04:37:43.324+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:43.324+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:43.326+05:30 [Info] switched currmeta from 39 -> 39 force true 2023-10-21T04:37:45.481+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:37:45.486+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:45.486+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:45.488+05:30 [Info] switched currmeta from 39 -> 39 force true 2023-10-21T04:37:53.512+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 9665410041638025825 2023-10-21T04:37:53.512+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 157.156µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:53.553+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53064 2023-10-21T04:37:53.558+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53064 2023-10-21T04:37:53.572+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:53064 2023-10-21T04:37:53.626+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 9665410041638025825, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:37:53.626+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 9665410041638025825, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:37:53.637+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 13749997730254346133, indexDefn DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:37:53.638+05:30 [Info] Indexer::handleCreateIndex InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:37:53.638+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 13749997730254346133 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:37:53.650+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:53.650+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:53.655+05:30 [Info] switched currmeta from 39 -> 41 force true 2023-10-21T04:37:53.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:37:53.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data/recovery) 2023-10-21T04:37:53.658+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:53.658+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:53.667+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:37:53.669+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:53.671+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:53.670+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data/recovery) 2023-10-21T04:37:53.672+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:37:53.672+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:37:53.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Assign plasmaId 54 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex 2023-10-21T04:37:53.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877 2023-10-21T04:37:53.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877/data 2023-10-21T04:37:53.677+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex 2023-10-21T04:37:53.677+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex 2023-10-21T04:37:53.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex started 2023-10-21T04:37:53.680+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:53.680+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:53.682+05:30 [Info] switched currmeta from 41 -> 42 force true 2023-10-21T04:37:53.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Assign plasmaId 55 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex 2023-10-21T04:37:53.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575 2023-10-21T04:37:53.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575/data 2023-10-21T04:37:53.693+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex 2023-10-21T04:37:53.693+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex 2023-10-21T04:37:53.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex started 2023-10-21T04:37:53.693+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 13749997730254346133 partitionId 0 WriterThreads 4 2023-10-21T04:37:53.693+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 13749997730254346133 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index Index: 13749997730254346133 Partition: 0 2023-10-21T04:37:53.693+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:37:53.693+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:37:53.693+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 13749997730254346133 2023-10-21T04:37:53.693+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:53.693+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:37:53.693+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:53.694+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:53.694+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 56.102276ms 2023-10-21T04:37:53.694+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 56.169755ms 2023-10-21T04:37:53.694+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 13749997730254346133, partnShardIdMap: map[0:[17406544375239214575 10170029854712240877]], indexDefn DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:37:53.719+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:53.719+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:53.726+05:30 [Info] switched currmeta from 42 -> 45 force true 2023-10-21T04:37:53.766+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [13749997730254346133] &{0} 2023-10-21T04:37:53.766+05:30 [Info] Indexer::handleBuildIndex [13749997730254346133] 2023-10-21T04:37:53.767+05:30 [Info] bulkUpdateRState: Index instance 13749997730254346133 rstate moved to ACTIVE 2023-10-21T04:37:53.767+05:30 [Info] Indexer::handleBuildIndex Added Index: [13749997730254346133] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:37:53.767+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:53.768+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:53.768+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:53.768+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:37:53.769+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:53.769+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:53.769+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:53.769+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:37:53.769+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:37:53.770+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:37:53.770+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:53.770+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:37:53.770+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:53.771+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:53.771+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:37:53.772+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:37:53.776+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:53.776+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:53.777+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:37:53.777+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:37:53.777+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 13749997730254346133 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:37:53.777+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:37:53.774+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:53.778+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:53.778+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [ InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 29203 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:37:53.779+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:37:53.779+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:37:53.780+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:53.780+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 14.231157ms 2023-10-21T04:37:53.780+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 14.276672ms 2023-10-21T04:37:53.780+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [13749997730254346133] 2023-10-21T04:37:53.780+05:30 [Info] switched currmeta from 45 -> 47 force true 2023-10-21T04:37:53.785+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 9665410041638025825 2023-10-21T04:37:53.785+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 158.604351ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:53.787+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:37:53.791+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.506812ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:53.801+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:37:53.829+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:53.829+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:53.833+05:30 [Info] switched currmeta from 47 -> 48 force true 2023-10-21T04:37:53.837+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:41244" doAuth() ... 2023-10-21T04:37:53.837+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:41244" auth successful 2023-10-21T04:37:53.838+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:41244" +1 2023-10-21T04:37:53.844+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:53.844+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:53.846+05:30 [Info] switched currmeta from 48 -> 48 force true 2023-10-21T04:37:53.914+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40484 2023-10-21T04:37:53.944+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.169654ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:53.986+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 60.654µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:53.995+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40484. Reason = EOF. Cleaning Pipe. 2023-10-21T04:37:53.995+05:30 [Info] messageListener.start(): message channel closed. Remove peer 58:88:19:99:10:b:3:4e as follower. 2023-10-21T04:37:54.225+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 274143649547516 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.225+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 162049908746825 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.228+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 206082810619393 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.231+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 120349894144442 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.235+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 212967951483349 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.251+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 116493903186721 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.251+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 143687065777286 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.258+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 136568163328749 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.260+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 263088038522959 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.275+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 261951617336488 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.275+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 78446412189949 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 188740876924377 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 245247859472006 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 58733680850130 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.309+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 206088069380492 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.309+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 173457131919400 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.309+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 127746572208200 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.309+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 277338544220217 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.321+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 125696667341112 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.323+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 34510046292135 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.324+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 250426459207023 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.325+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 274158831830257 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.328+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 191500187207978 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.329+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 118920785117980 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.348+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 146375746634523 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.348+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 5906787184229 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.348+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 35716737784090 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.348+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 137256376714252 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.348+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 106231614376481 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.349+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 241391494503904 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.349+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 216238759475962 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.349+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 112482982607415 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 168446351781198 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 41101832875819 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 121450709505223 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.350+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 127702263378100 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.351+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 100008922352935 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.356+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 169268048765249 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.364+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 106297365959267 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.364+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 245900669851618 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 39045953044628 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 149033089785397 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.367+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 88718752743571 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.369+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 160399698726237 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.374+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 132470209691538 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.377+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 162870112426804 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.379+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 110286285845649 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.379+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 217237947818243 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.380+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 264001907139144 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.389+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 266604622117081 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.389+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 63737850123332 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.390+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 84581802233088 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.390+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 32274016651656 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.393+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 225662991941187 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.394+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 241607113601772 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.395+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 272523968790649 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.395+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 77598347829810 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.396+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 136038310030103 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.396+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 197957220074784 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.397+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 97677619676504 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.398+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b bucket_%2:_default:c1 InstanceIds [13749997730254346133] 2023-10-21T04:37:54.398+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 29203 2023-10-21T04:37:54.399+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 27479155597282 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.399+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 105415986286631 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.400+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:37:54.400+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:37:54.401+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:37:54.401+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:37:54.401+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:54.402+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:37:54.402+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 72072509155168 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.402+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 161870047388121 0 29203 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:37:54.446+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:37:54.598+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:37:54.598+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:37:54.610+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:40530 2023-10-21T04:37:54.611+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:37:54.650+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 34.924465ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:54.650+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:37:54.650+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:37:54.650+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:37:54.670+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:37:54.730+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:37:54.734+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:37:54.735+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:37:54.737+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:37:54.739+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:37:54.740+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:37:54.740+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:40530->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:37:54.741+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:40530. Reason = EOF. Cleaning Pipe. 2023-10-21T04:37:54.743+05:30 [Info] messageListener.start(): message channel closed. Remove peer da:43:c2:48:bc:60:27:d4 as follower. 2023-10-21T04:37:54.741+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:36908->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:37:54.988+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:37:54.999+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13749997730254346133 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:37:55.000+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13749997730254346133 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.91µs SnapOpenDur 245.73µs 2023-10-21T04:37:55.000+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 13749997730254346133 INDEX_STATE_CATCHUP 2023-10-21T04:37:55.000+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 13749997730254346133 true 2023-10-21T04:37:55.007+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 13749997730254346133 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 29203 BuildTS: [0=27,1=22,2=24,3=23,4=23,5=28,6=22,7=23,8=18,9=23,10=35,11=19,12=23,13=22,14=12,15=21,16=15,17=13,18=14,19=28,20=20,21=27,22=24,23=21,24=21,25=21,26=22,27=20,28=17,29=23,30=19,31=28,32=25,33=23,34=25,35=23,36=17,37=19,38=19,39=28,40=27,41=25,42=18,43=26,44=20,45=28,46=23,47=26,48=22,49=19,50=26,51=23,52=22,53=26,54=23,55=20,56=19,57=24,58=28,59=31,60=23,61=18,62=29,63=25] 2023-10-21T04:37:55.007+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13749997730254346133, PartitionId 0 SnapshotId 13 Creating recovery point ... 2023-10-21T04:37:55.010+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 29203 2023-10-21T04:37:55.010+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 29203 2023-10-21T04:37:55.010+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:55.010+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:55.010+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:55.010+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:55.011+05:30 [Info] send progress stats to clients 2023-10-21T04:37:55.011+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:37:55.011+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:37:55.011+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:4] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:37:55.011+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 29203 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:37:55.017+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:37:55.023+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.051999ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:55.029+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:55.029+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:55.034+05:30 [Info] switched currmeta from 48 -> 50 force true 2023-10-21T04:37:55.039+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:55.039+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:55.041+05:30 [Info] switched currmeta from 50 -> 50 force true 2023-10-21T04:37:55.044+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId bucket_%2 2023-10-21T04:37:55.044+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 29203 2023-10-21T04:37:55.044+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 29203 2023-10-21T04:37:55.045+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:37:55.045+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:37:55.045+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 13749997730254346133 false 2023-10-21T04:37:55.045+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 13749997730254346133 2023-10-21T04:37:55.045+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:37:55.045+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 29203 2023-10-21T04:37:55.064+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13749997730254346133, PartitionId 0 SnapshotId 13 Created recovery point (took 57.41502ms) 2023-10-21T04:37:55.151+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:37:55.396+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:55.396+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:55.398+05:30 [Info] switched currmeta from 50 -> 51 force true 2023-10-21T04:37:55.402+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:37:55.402+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:37:55.402+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 13749997730254346133 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 29203, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11862654290959748966 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 f9550ad944fc 2 29 0 29} { 1 93622f3f2a49 2 28 0 28} { 2 bb6e64b8a601 2 24 0 24} { 3 6d7524b071ba 2 23 0 23} { 4 82aec083b486 2 23 0 23} { 5 c1b17718c1d5 2 28 0 28} { 6 69f359b7cb21 2 22 0 22} { 7 4758ba8850fd 2 23 0 23} { 8 ef46f514b04f 2 18 0 18} { 9 7c3540dce2ed 2 23 0 23} { 10 ee3e5d1984a8 2 35 0 35} { 11 742f510ec848 2 19 0 19} { 12 356b002ef4d2 2 23 0 23} { 13 aba8a8ec75d9 2 22 0 22} { 14 9dc222bb5828 2 12 0 12} { 15 fc3ce9692839 2 21 0 21} { 16 df0d378bb686 2 20 0 20} { 17 bb6f9e2af98c 2 13 0 13} { 18 1f62ff0ce8a7 2 19 0 19} { 19 e3c2f424b56f 2 28 0 28} { 20 f95893c884f1 2 20 0 20} { 21 72520934d138 2 27 0 27} { 22 207bf377d91a 2 24 0 24} { 23 ae2b1ca2092a 2 21 0 21} { 24 609df9a38a21 2 21 0 21} { 25 6c286727431c 2 21 0 21} { 26 7cd57d93c40c 2 22 0 22} { 27 55f47f3ae65 2 20 0 20} { 28 8520c261c31b 2 17 0 17} { 29 664d7c38ba37 2 23 0 23} { 30 2561c425ef2b 2 19 0 19} { 31 db8b56481de0 2 28 0 28} { 32 6e757265d4c7 2 25 0 25} { 33 c4ab0271fafa 2 23 0 23} { 34 993378f62d4e 2 25 0 25} { 35 7425000bb0b4 2 23 0 23} { 36 5af5244a8d27 2 17 0 17} { 37 99f2c9eb2141 2 19 0 19} { 38 dfa5361397e2 2 19 0 19} { 39 878b78605e35 2 28 0 28} { 40 60ad48bd1263 2 27 0 27} { 41 50b071be0093 2 25 0 25} { 42 238318294094 2 18 0 18} { 43 91e1f7108d5d 2 28 0 28} { 44 c593a6b62103 2 20 0 20} { 45 644e06e43491 2 34 0 34} { 46 942127321b34 2 23 0 23} { 47 787b1fcabf92 2 26 0 26} { 48 f01bbbe52248 2 22 0 22} { 49 39f81fea5444 2 19 0 19} { 50 f279b9cbf8d9 2 26 0 26} { 51 1d5a614c4988 2 23 0 23} { 52 4ced3c46cd00 2 22 0 22} { 53 cd3d42793e43 2 26 0 26} { 54 dbbd8a2e72ec 2 23 0 23} { 55 f7dbee5a4079 2 22 0 22} { 56 469345f52232 2 19 0 19} { 57 b40a81c88520 2 24 0 24} { 58 7bb9e3252b17 2 28 0 28} { 59 58d657d2c558 2 31 0 31} { 60 418cb04e6760 2 23 0 23} { 61 5fe0126a9027 2 18 0 18} { 62 18fdfd4edfe2 2 29 0 29} { 63 93384eaca1d9 2 25 0 25} 2023-10-21T04:37:55.402+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 13749997730254346133 INDEX_STATE_ACTIVE 2023-10-21T04:37:55.402+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:37:55.402+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:37:55.402+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:37:55.402+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 4 CurrMaint 0 New 0 2023-10-21T04:37:55.402+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 29203 2023-10-21T04:37:55.402+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:55.403+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:37:55.403+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:37:55.403+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 29203 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:55.403+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 13749997730254346133 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:37:55.403+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:37:55.403+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:37:55.403+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:37:55.403+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 29203 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:37:55.404+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:37:55.404+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:20 1:15 2:17 3:16 4:16 5:21 6:15 7:16 8:11 9:16 10:28 11:12 12:16 13:15 14:5 15:14 16:8 17:6 18:7 19:21 20:13 21:20 22:17 23:14 24:14 25:14 26:15 27:13 28:10 29:16 30:12 31:21 32:18 33:16 34:18 35:16 36:10 37:12 38:12 39:21 40:20 41:18 42:11 43:19 44:13 45:21 46:16 47:19 48:15 49:12 50:19 51:16 52:15 53:19 54:16 55:13 56:12 57:17 58:21 59:24 60:16 61:11 62:22 63:18] 2023-10-21T04:37:55.404+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:27 1:22 2:24 3:23 4:23 5:28 6:22 7:23 8:18 9:23 10:35 11:19 12:23 13:22 14:12 15:21 16:15 17:13 18:14 19:28 20:20 21:27 22:24 23:21 24:21 25:21 26:22 27:20 28:17 29:23 30:19 31:28 32:25 33:23 34:25 35:23 36:17 37:19 38:19 39:28 40:27 41:25 42:18 43:26 44:20 45:28 46:23 47:26 48:22 49:19 50:26 51:23 52:22 53:26 54:23 55:20 56:19 57:24 58:28 59:31 60:23 61:18 62:29 63:25] 2023-10-21T04:37:55.404+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:37:55.404+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:41244: use of closed network connection 2023-10-21T04:37:55.404+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:41244" exit: read tcp4 127.0.0.1:9121->127.0.0.1:41244: use of closed network connection 2023-10-21T04:37:55.404+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:41244" closed ! 2023-10-21T04:37:55.404+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:37:55.405+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:55.405+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:55.405+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:37:55.405+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:37:55.405+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:37:55.405+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:37:55.406+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 29203 2023-10-21T04:37:55.407+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:37:55.421+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.317535ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:37:55.434+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:55.434+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:55.436+05:30 [Info] switched currmeta from 51 -> 52 force true 2023-10-21T04:37:55.441+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:37:55.441+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:37:55.441+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:37:55.441+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 29203 2023-10-21T04:37:55.441+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 29203 2023-10-21T04:37:55.442+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:37:55.537+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:37:55.540+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:37:55.540+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:37:55.542+05:30 [Info] switched currmeta from 52 -> 52 force true 2023-10-21T04:37:55.957+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13749997730254346133:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 2023-10-21T04:37:55.957+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13749997730254346133:[0]]. New rollbacktime: 1697843222668465492, Old rollbacktime: 0 2023-10-21T04:37:56.403+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:37:56.819+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4604997787837021504:[0]]. New rollbacktime: 1697843078714796483, Old rollbacktime: 0 2023-10-21T04:38:02.456+05:30 [Info] janitor: running cleanup. 2023-10-21T04:38:02.456+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[12833670934079048728 573521267164877540 12582561312368048736 11365633301282701856 11619165464560280276 18026660730857541767 18323460706027642575 6985049432225870291 8378417662949994532 12824340686309573428 14741949230702299196 3674078063353481907 4138047918192025493 4790214984019693894 14292583894719865678 17797267771846799851 15968368593314043432 5465835793930010953 8857269605176042330 12027054384338714804 13268712595133993155 12338036531800130039 4535197604682045916 12599498416578472779 13856704970496601958 14889401051926000176 15431181238515168303 16162012140118277065 2048006213723315482 12195715103909237684 1252359852432998131 7147407900589480727 7392796230534034390 7525085113233084253 249887442666643345 4860549750398470299 18046371353728341033 8669649359784409226 14549775068266281786 15945117906390072636 17656144409930763141 8504686614261377811 12140458060295645238 4532813151040661705]} 2023-10-21T04:38:02.456+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:38:02.660+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_1 Forcing Overdue Commit 2023-10-21T04:38:02.660+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 4 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:38:02.660+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 4 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.77µs SnapOpenDur 90.14µs 2023-10-21T04:38:02.660+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 5 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:38:02.660+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 5 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.495µs SnapOpenDur 43.77µs 2023-10-21T04:38:02.660+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 6 Creating New Snapshot SnapshotInfo: count:122 committed:true 2023-10-21T04:38:02.660+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 6 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:122 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.799µs SnapOpenDur 48.524µs 2023-10-21T04:38:02.660+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 7 Creating New Snapshot SnapshotInfo: count:136 committed:true 2023-10-21T04:38:02.660+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 7 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:136 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.319µs SnapOpenDur 42.332µs 2023-10-21T04:38:02.660+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 8 Creating New Snapshot SnapshotInfo: count:106 committed:true 2023-10-21T04:38:02.660+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 8 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:106 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.528µs SnapOpenDur 42.955µs 2023-10-21T04:38:02.661+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 1 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:38:02.661+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 1 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.228µs SnapOpenDur 50.299µs 2023-10-21T04:38:02.661+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 2 Creating New Snapshot SnapshotInfo: count:136 committed:true 2023-10-21T04:38:02.661+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 2 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:136 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.018µs SnapOpenDur 48.198µs 2023-10-21T04:38:02.661+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 7516828288031590951 PartitionId 3 Creating New Snapshot SnapshotInfo: count:125 committed:true 2023-10-21T04:38:02.661+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 7516828288031590951 PartitionId: 3 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:125 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.295µs SnapOpenDur 43.333µs 2023-10-21T04:38:02.661+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 3643393537124379309 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:38:02.661+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 3643393537124379309 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:6 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.56µs SnapOpenDur 59.803µs 2023-10-21T04:38:02.661+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 642374041935721654 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:38:02.661+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 642374041935721654 PartitionId: 0 SliceId: 0 Crc64: 5334336579729819384 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.335µs SnapOpenDur 64.239µs 2023-10-21T04:38:02.661+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 642374041935721654, PartitionId 0 SnapshotId 13 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.662+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3643393537124379309, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:38:02.669+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 13749997730254346133 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 13749997730254346133 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.47µs SnapOpenDur 66.207µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17434304260397056806 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17434304260397056806 PartitionId: 0 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:6 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.498µs SnapOpenDur 58.543µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 3 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 3 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.222µs SnapOpenDur 61.828µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 4 Creating New Snapshot SnapshotInfo: count:118 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 4 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:118 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.319µs SnapOpenDur 51.821µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 5 Creating New Snapshot SnapshotInfo: count:138 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 5 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:138 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 6.42µs SnapOpenDur 42.306µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 6 Creating New Snapshot SnapshotInfo: count:127 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 6 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:127 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.177µs SnapOpenDur 45.569µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 7 Creating New Snapshot SnapshotInfo: count:131 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 7 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:131 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.839µs SnapOpenDur 46.677µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 8 Creating New Snapshot SnapshotInfo: count:112 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 8 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:112 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.204µs SnapOpenDur 55.905µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 1 Creating New Snapshot SnapshotInfo: count:128 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 1 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:128 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.159µs SnapOpenDur 34.787µs 2023-10-21T04:38:02.670+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 8965923595543532942 PartitionId 2 Creating New Snapshot SnapshotInfo: count:123 committed:true 2023-10-21T04:38:02.670+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 8965923595543532942 PartitionId: 2 SliceId: 0 Crc64: 11862654290959748966 (SnapshotInfo: count:123 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.165µs SnapOpenDur 34.831µs 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13749997730254346133, PartitionId 0 SnapshotId 14 Creating recovery point ... 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.671+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.672+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.672+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:38:02.678+05:30 [Info] memstats {"Alloc":126180744, "TotalAlloc":159634976, "Sys":170036520, "Lookups":0, "Mallocs":492123,"Frees":351103, "HeapAlloc":126180744, "HeapSys":150994944, "HeapIdle":15392768, "HeapInuse":135602176,"HeapReleased":4063232, "HeapObjects":141020,"MSpanInuse":573600, "MSpanSys": 783360, "StackInuse": 4194304,"GCSys":11448312, "LastGC":1697843223048762344,"PauseTotalNs":1557686, "PauseNs":[441799, 215798, 42897, 382948, 273857, 200387], "NumGC":6} 2023-10-21T04:38:02.720+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 5 SnapshotId 2 Created recovery point (took 57.83202ms) 2023-10-21T04:38:02.721+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:38:02.722+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:38:02.796+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3643393537124379309, PartitionId 0 SnapshotId 3 Created recovery point (took 133.84011ms) 2023-10-21T04:38:02.797+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 3643393537124379309, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:38:02.827+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 SnapshotId 3 Created recovery point (took 155.831741ms) 2023-10-21T04:38:02.829+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:38:02.872+05:30 [Info] connected with 2 indexers 2023-10-21T04:38:02.872+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:38:02.893+05:30 [Info] num concurrent scans {0} 2023-10-21T04:38:02.897+05:30 [Info] average scan response {0 ms} 2023-10-21T04:38:02.902+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:38:02.915+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:38:02.936+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4 SnapshotId 2 Created recovery point (took 274.488413ms) 2023-10-21T04:38:02.939+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 4. 2023-10-21T04:38:02.940+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:38:02.942+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 4. 2023-10-21T04:38:02.942+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:38:02.946+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:38:02.949+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 2 SnapshotId 2 Created recovery point (took 287.878241ms) 2023-10-21T04:38:02.952+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:38:02.952+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:38:02.981+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 6 SnapshotId 2 Created recovery point (took 319.472936ms) 2023-10-21T04:38:02.984+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:38:02.985+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:122 committed:false. num RPs 3. 2023-10-21T04:38:03.001+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 7 SnapshotId 2 Created recovery point (took 339.352932ms) 2023-10-21T04:38:03.002+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:38:03.003+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:136 committed:false. num RPs 3. 2023-10-21T04:38:03.021+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 8 SnapshotId 2 Created recovery point (took 359.210181ms) 2023-10-21T04:38:03.022+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:38:03.023+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:106 committed:false. num RPs 3. 2023-10-21T04:38:03.027+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 259] aggregate storage info: map[bucket_%2:map[kv:2402496 index:15515486] bucket_1:map[kv:2468032 index:16388208]] 2023-10-21T04:38:03.031+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 1 SnapshotId 2 Created recovery point (took 369.210404ms) 2023-10-21T04:38:03.033+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:38:03.033+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:38:03.038+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 642374041935721654, PartitionId 0 SnapshotId 13 Created recovery point (took 376.263041ms) 2023-10-21T04:38:03.048+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3 SnapshotId 2 Created recovery point (took 386.116721ms) 2023-10-21T04:38:03.050+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 4. 2023-10-21T04:38:03.050+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:38:03.050+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 4. 2023-10-21T04:38:03.051+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:125 committed:false. num RPs 3. 2023-10-21T04:38:03.066+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 13749997730254346133, PartitionId 0 SnapshotId 14 Created recovery point (took 395.015997ms) 2023-10-21T04:38:03.079+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 SnapshotId 2 Created recovery point (took 408.089152ms) 2023-10-21T04:38:03.080+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:38:03.081+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:38:03.105+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 SnapshotId 2 Created recovery point (took 433.783463ms) 2023-10-21T04:38:03.106+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:38:03.106+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:118 committed:false. num RPs 3. 2023-10-21T04:38:03.118+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 SnapshotId 2 Created recovery point (took 447.336422ms) 2023-10-21T04:38:03.119+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:38:03.120+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:138 committed:false. num RPs 3. 2023-10-21T04:38:03.141+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 SnapshotId 2 Created recovery point (took 469.284148ms) 2023-10-21T04:38:03.142+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 4. 2023-10-21T04:38:03.142+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:38:03.142+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 4. 2023-10-21T04:38:03.143+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:127 committed:false. num RPs 3. 2023-10-21T04:38:03.148+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 SnapshotId 2 Created recovery point (took 476.553654ms) 2023-10-21T04:38:03.149+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:38:03.150+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:131 committed:false. num RPs 3. 2023-10-21T04:38:03.162+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 SnapshotId 2 Created recovery point (took 490.677987ms) 2023-10-21T04:38:03.163+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:38:03.164+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:112 committed:false. num RPs 3. 2023-10-21T04:38:03.181+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 SnapshotId 2 Created recovery point (took 509.591075ms) 2023-10-21T04:38:03.183+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:38:03.186+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:128 committed:false. num RPs 3. 2023-10-21T04:38:03.191+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 SnapshotId 2 Created recovery point (took 518.939482ms) 2023-10-21T04:38:03.192+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:38:03.192+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:123 committed:false. num RPs 3. 2023-10-21T04:38:03.951+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 8965923595543532942 &{0} 2023-10-21T04:38:03.955+05:30 [Info] Indexer::handleDropIndex - IndexInstId 8965923595543532942 2023-10-21T04:38:03.955+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:38:03.956+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:03.960+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8965923595543532942 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:38:03.961+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [8965923595543532942] 2023-10-21T04:38:03.961+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [8965923595543532942] 2023-10-21T04:38:03.961+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:03.961+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [8965923595543532942] 2023-10-21T04:38:03.961+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:38:03.962+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:38:03.963+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:38:03.963+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:38:03.963+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex 2023-10-21T04:38:03.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex stopped 2023-10-21T04:38:03.963+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.964+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.964+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.964+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.965+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:38:03.965+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:38:03.965+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex 2023-10-21T04:38:03.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex stopped 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.965+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:38:03.965+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:38:03.965+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex 2023-10-21T04:38:03.965+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex stopped 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.965+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.966+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:38:03.966+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:38:03.966+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex 2023-10-21T04:38:03.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex stopped 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.966+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:38:03.966+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:38:03.966+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex 2023-10-21T04:38:03.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex stopped 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.966+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.966+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] [17406544375239214575 10170029854712240877]} 2:{2 1 [:9123] [17406544375239214575 10170029854712240877]} 3:{3 1 [:9123] [17406544375239214575 10170029854712240877]} 4:{4 1 [:9123] [17406544375239214575 10170029854712240877]} 5:{5 1 [:9123] [17406544375239214575 10170029854712240877]} 6:{6 1 [:9123] [17406544375239214575 10170029854712240877]} 7:{7 1 [:9123] [17406544375239214575 10170029854712240877]} 8:{8 1 [:9123] [17406544375239214575 10170029854712240877]}] 8 KEY 0}] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:03.967+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 8965923595543532942 Defn: DefnId: 8910459001587546151 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] [17406544375239214575 10170029854712240877]} 2:{2 1 [:9123] [17406544375239214575 10170029854712240877]} 3:{3 1 [:9123] [17406544375239214575 10170029854712240877]} 4:{4 1 [:9123] [17406544375239214575 10170029854712240877]} 5:{5 1 [:9123] [17406544375239214575 10170029854712240877]} 6:{6 1 [:9123] [17406544375239214575 10170029854712240877]} 7:{7 1 [:9123] [17406544375239214575 10170029854712240877]} 8:{8 1 [:9123] [17406544375239214575 10170029854712240877]}] 8 KEY 0}] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:03.967+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 12.321701ms 2023-10-21T04:38:03.967+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.967+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.967+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.967+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:38:03.967+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:38:03.967+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:38:03.967+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:38:03.967+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex 2023-10-21T04:38:03.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex stopped 2023-10-21T04:38:03.968+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.968+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.968+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.968+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.968+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.968+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.968+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.968+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:38:03.974+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:38:03.975+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:38:03.975+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:38:03.976+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex 2023-10-21T04:38:03.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex stopped 2023-10-21T04:38:03.976+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 8965923595543532942 2023-10-21T04:38:03.980+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4, IndexDefnId 8910459001587546151. 2023-10-21T04:38:03.980+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.980+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:38:03.981+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:38:03.981+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex 2023-10-21T04:38:03.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex stopped 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:03.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex] 2023-10-21T04:38:03.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex closed 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:38:03.981+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.981+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex stopped 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.981+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.982+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [8965923595543532942] 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex] 2023-10-21T04:38:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex closed 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:38:03.988+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:38:03.988+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex 2023-10-21T04:38:03.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex stopped 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:03.988+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:03.992+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:38:03.994+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 43196 2023-10-21T04:38:03.994+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 26.60914ms 2023-10-21T04:38:03.994+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 39.252694ms 2023-10-21T04:38:03.992+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:03.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex] 2023-10-21T04:38:03.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex closed 2023-10-21T04:38:03.995+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:03.995+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:03.995+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:03.996+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:03.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:38:03.996+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:38:03.996+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:38:03.996+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:38:03.996+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex 2023-10-21T04:38:03.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex stopped 2023-10-21T04:38:03.997+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:03.997+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:03.997+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:03.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex] 2023-10-21T04:38:03.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex closed 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:38:04.000+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:38:04.000+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex 2023-10-21T04:38:04.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex stopped 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex] 2023-10-21T04:38:04.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex closed 2023-10-21T04:38:04.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.000+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex ... 2023-10-21T04:38:04.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex ... 2023-10-21T04:38:04.001+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.001+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.002+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex] 2023-10-21T04:38:04.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex closed 2023-10-21T04:38:04.005+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.005+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.005+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.005+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:38:04.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.005+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex] 2023-10-21T04:38:04.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex closed 2023-10-21T04:38:04.007+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.007+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.010+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.005+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.008+05:30 [Info] switched currmeta from 52 -> 53 force true 2023-10-21T04:38:03.997+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.009+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 68.25892ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:04.010+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.011+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.011+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:38:04.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.013+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.022+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex] 2023-10-21T04:38:04.049+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:38:04.050+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:38:04.050+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:38:04.050+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex 2023-10-21T04:38:04.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex stopped 2023-10-21T04:38:04.051+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.051+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.050+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.050+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex closed 2023-10-21T04:38:04.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex] 2023-10-21T04:38:04.051+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.053+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.060+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex] 2023-10-21T04:38:04.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex closed 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:38:04.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex ... 2023-10-21T04:38:04.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex ... 2023-10-21T04:38:04.063+05:30 [Info] switched currmeta from 53 -> 55 force true 2023-10-21T04:38:04.065+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_2.index destroyed 2023-10-21T04:38:04.066+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 2, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.066+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 2 Destroy Done 2023-10-21T04:38:04.068+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.068+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex closed 2023-10-21T04:38:04.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex] 2023-10-21T04:38:04.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex closed 2023-10-21T04:38:04.069+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.069+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:38:04.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex ... 2023-10-21T04:38:04.071+05:30 [Info] switched currmeta from 55 -> 55 force true 2023-10-21T04:38:04.073+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.077+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:38:04.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex 2023-10-21T04:38:04.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:38:04.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex closed 2023-10-21T04:38:04.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:38:04.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex] 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex closed 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex ... 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex ... 2023-10-21T04:38:04.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex stopped 2023-10-21T04:38:04.079+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.079+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.079+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.079+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.080+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.080+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex stopped 2023-10-21T04:38:04.080+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.080+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.080+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.080+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.081+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.081+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:38:04.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex ... 2023-10-21T04:38:04.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex stopped 2023-10-21T04:38:04.081+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.081+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.081+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.081+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex ... 2023-10-21T04:38:04.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex ... 2023-10-21T04:38:04.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_7.index destroyed 2023-10-21T04:38:04.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex ... 2023-10-21T04:38:04.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex ... 2023-10-21T04:38:04.095+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.097+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 7, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.097+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 7 Destroy Done 2023-10-21T04:38:04.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex] 2023-10-21T04:38:04.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex closed 2023-10-21T04:38:04.108+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.108+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex ... 2023-10-21T04:38:04.109+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex ... 2023-10-21T04:38:04.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_3.index destroyed 2023-10-21T04:38:04.110+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 3, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.110+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 3 Destroy Done 2023-10-21T04:38:04.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex ... 2023-10-21T04:38:04.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex ... 2023-10-21T04:38:04.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex ... 2023-10-21T04:38:04.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex ... 2023-10-21T04:38:04.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_5.index destroyed 2023-10-21T04:38:04.118+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 5, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.118+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 5 Destroy Done 2023-10-21T04:38:04.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex ... 2023-10-21T04:38:04.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex ... 2023-10-21T04:38:04.121+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex] 2023-10-21T04:38:04.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex closed 2023-10-21T04:38:04.126+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.126+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 8 Close Done 2023-10-21T04:38:04.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_4.index destroyed 2023-10-21T04:38:04.127+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 4, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.127+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 4 Destroy Done 2023-10-21T04:38:04.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex ... 2023-10-21T04:38:04.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.128+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex ... 2023-10-21T04:38:04.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex ... 2023-10-21T04:38:04.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_1.index destroyed 2023-10-21T04:38:04.131+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 1, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.131+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 1 Destroy Done 2023-10-21T04:38:04.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex ... 2023-10-21T04:38:04.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex ... 2023-10-21T04:38:04.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex] 2023-10-21T04:38:04.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex closed 2023-10-21T04:38:04.135+05:30 [Info] bucket_%2/idx_partitioned/Backstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.135+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex ... 2023-10-21T04:38:04.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex ... 2023-10-21T04:38:04.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_8.index destroyed 2023-10-21T04:38:04.137+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 8, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.137+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 8 Destroy Done 2023-10-21T04:38:04.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex ... 2023-10-21T04:38:04.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex ... 2023-10-21T04:38:04.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_8965923595543532942_6.index destroyed 2023-10-21T04:38:04.142+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8965923595543532942, PartitionId 6, IndexDefnId 8910459001587546151. 2023-10-21T04:38:04.142+05:30 [Info] Indexer::cleanupIndexData IndexInst 8965923595543532942 Partition 6 Destroy Done 2023-10-21T04:38:04.151+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.151+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.186+05:30 [Info] switched currmeta from 55 -> 56 force true 2023-10-21T04:38:04.208+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.208+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.231+05:30 [Info] switched currmeta from 56 -> 56 force true 2023-10-21T04:38:04.316+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 13749997730254346133 &{0} 2023-10-21T04:38:04.316+05:30 [Info] Indexer::handleDropIndex - IndexInstId 13749997730254346133 2023-10-21T04:38:04.316+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:38:04.316+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:04.316+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 13749997730254346133 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:38:04.316+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [13749997730254346133] 2023-10-21T04:38:04.316+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [13749997730254346133] 2023-10-21T04:38:04.316+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:04.317+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [13749997730254346133] 2023-10-21T04:38:04.317+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:04.317+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 13749997730254346133 Defn: DefnId: 9665410041638025825 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:04.318+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.004349ms 2023-10-21T04:38:04.318+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 13749997730254346133, PartitionId 0, IndexDefnId 9665410041638025825. 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex] 2023-10-21T04:38:04.318+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex] 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex] 2023-10-21T04:38:04.318+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex 2023-10-21T04:38:04.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex stopped 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.318+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 13749997730254346133 2023-10-21T04:38:04.319+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [13749997730254346133] 2023-10-21T04:38:04.322+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex] 2023-10-21T04:38:04.322+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex closed 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Mainstore#13749997730254346133:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex] 2023-10-21T04:38:04.323+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex] 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex] 2023-10-21T04:38:04.323+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex 2023-10-21T04:38:04.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex stopped 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.323+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.323+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:38:04.323+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 43196 2023-10-21T04:38:04.323+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 5.759427ms 2023-10-21T04:38:04.323+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 7.83105ms 2023-10-21T04:38:04.326+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex] 2023-10-21T04:38:04.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex closed 2023-10-21T04:38:04.327+05:30 [Info] bucket_%2/idx_secondary/Backstore#13749997730254346133:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.327+05:30 [Info] Indexer::cleanupIndexData IndexInst 13749997730254346133 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex ... 2023-10-21T04:38:04.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex ... 2023-10-21T04:38:04.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex ... 2023-10-21T04:38:04.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 55 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex ... 2023-10-21T04:38:04.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_13749997730254346133_0.index destroyed 2023-10-21T04:38:04.336+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 13749997730254346133, PartitionId 0, IndexDefnId 9665410041638025825. 2023-10-21T04:38:04.336+05:30 [Info] Indexer::cleanupIndexData IndexInst 13749997730254346133 Partition 0 Destroy Done 2023-10-21T04:38:04.337+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 27.579863ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:04.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.370+05:30 [Info] switched currmeta from 56 -> 60 force true 2023-10-21T04:38:04.380+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.380+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.382+05:30 [Info] switched currmeta from 60 -> 60 force true 2023-10-21T04:38:04.447+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3643393537124379309 &{0} 2023-10-21T04:38:04.447+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3643393537124379309 2023-10-21T04:38:04.447+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 2 ReplicaId: 0 2023-10-21T04:38:04.447+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:04.447+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 3643393537124379309 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 2 ReplicaId: 0 2023-10-21T04:38:04.448+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3643393537124379309] 2023-10-21T04:38:04.448+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3643393537124379309] 2023-10-21T04:38:04.448+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:04.448+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3643393537124379309] 2023-10-21T04:38:04.448+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] [4718895791748289908]}] 1 SINGLE 0}] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:04.448+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 3643393537124379309 Defn: DefnId: 11276338709617092948 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[0:{0 2 [:9123] [4718895791748289908]}] 1 SINGLE 0}] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:04.448+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.227693ms 2023-10-21T04:38:04.448+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3643393537124379309, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:38:04.448+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.448+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.448+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex] 2023-10-21T04:38:04.449+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:38:04.449+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex] 2023-10-21T04:38:04.449+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex] 2023-10-21T04:38:04.449+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex 2023-10-21T04:38:04.449+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex stopped 2023-10-21T04:38:04.449+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.449+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.449+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.449+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.449+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3643393537124379309 2023-10-21T04:38:04.449+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [3643393537124379309] 2023-10-21T04:38:04.452+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:38:04.452+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 55822 2023-10-21T04:38:04.452+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 3.828637ms 2023-10-21T04:38:04.452+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 5.163832ms 2023-10-21T04:38:04.458+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex] 2023-10-21T04:38:04.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex closed 2023-10-21T04:38:04.459+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.459+05:30 [Info] Indexer::cleanupIndexData IndexInst 3643393537124379309 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex ... 2023-10-21T04:38:04.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex ... 2023-10-21T04:38:04.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_3643393537124379309_0.index destroyed 2023-10-21T04:38:04.465+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3643393537124379309, PartitionId 0, IndexDefnId 11276338709617092948. 2023-10-21T04:38:04.465+05:30 [Info] Indexer::cleanupIndexData IndexInst 3643393537124379309 Partition 0 Destroy Done 2023-10-21T04:38:04.500+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 59.737341ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:04.509+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.509+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.519+05:30 [Info] switched currmeta from 60 -> 63 force true 2023-10-21T04:38:04.546+05:30 [Info] Indexer::ReadMemstats Time Taken 138.828µs 2023-10-21T04:38:04.561+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.561+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.571+05:30 [Info] switched currmeta from 63 -> 64 force true 2023-10-21T04:38:04.604+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.604+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.605+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 7516828288031590951 &{0} 2023-10-21T04:38:04.605+05:30 [Info] Indexer::handleDropIndex - IndexInstId 7516828288031590951 2023-10-21T04:38:04.606+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 2 ReplicaId: 0 2023-10-21T04:38:04.606+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:04.606+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7516828288031590951 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 2 ReplicaId: 0 2023-10-21T04:38:04.606+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [7516828288031590951] 2023-10-21T04:38:04.606+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [7516828288031590951] 2023-10-21T04:38:04.606+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:04.606+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [7516828288031590951] 2023-10-21T04:38:04.606+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.606+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.606+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.606+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex] 2023-10-21T04:38:04.606+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:38:04.606+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex] 2023-10-21T04:38:04.607+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex] 2023-10-21T04:38:04.607+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex 2023-10-21T04:38:04.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex stopped 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.607+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex] 2023-10-21T04:38:04.607+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:38:04.607+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex] 2023-10-21T04:38:04.607+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex] 2023-10-21T04:38:04.607+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex 2023-10-21T04:38:04.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex stopped 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.607+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.607+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.607+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex] 2023-10-21T04:38:04.607+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex stopped 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.607+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex] 2023-10-21T04:38:04.607+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] [4718895791748289908 15531034489713578274]} 2:{2 2 [:9123] [4718895791748289908 15531034489713578274]} 3:{3 2 [:9123] [4718895791748289908 15531034489713578274]} 4:{4 2 [:9123] [4718895791748289908 15531034489713578274]} 5:{5 2 [:9123] [4718895791748289908 15531034489713578274]} 6:{6 2 [:9123] [4718895791748289908 15531034489713578274]} 7:{7 2 [:9123] [4718895791748289908 15531034489713578274]} 8:{8 2 [:9123] [4718895791748289908 15531034489713578274]}] 8 KEY 0}] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex stopped 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.607+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 7516828288031590951 Defn: DefnId: 16961575949607441659 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emailid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] [4718895791748289908 15531034489713578274]} 2:{2 2 [:9123] [4718895791748289908 15531034489713578274]} 3:{3 2 [:9123] [4718895791748289908 15531034489713578274]} 4:{4 2 [:9123] [4718895791748289908 15531034489713578274]} 5:{5 2 [:9123] [4718895791748289908 15531034489713578274]} 6:{6 2 [:9123] [4718895791748289908 15531034489713578274]} 7:{7 2 [:9123] [4718895791748289908 15531034489713578274]} 8:{8 2 [:9123] [4718895791748289908 15531034489713578274]}] 8 KEY 0}] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.658558ms 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.608+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex stopped 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex] 2023-10-21T04:38:04.608+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex 2023-10-21T04:38:04.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex stopped 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.608+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7516828288031590951, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex] 2023-10-21T04:38:04.609+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:38:04.609+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex] 2023-10-21T04:38:04.609+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex] 2023-10-21T04:38:04.609+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex 2023-10-21T04:38:04.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex stopped 2023-10-21T04:38:04.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.609+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.608+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.609+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 7516828288031590951 2023-10-21T04:38:04.612+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:38:04.612+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex] 2023-10-21T04:38:04.613+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex] 2023-10-21T04:38:04.613+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex 2023-10-21T04:38:04.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex stopped 2023-10-21T04:38:04.613+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.613+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.613+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.613+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.613+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId Instances [7516828288031590951] 2023-10-21T04:38:04.618+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceId 2023-10-21T04:38:04.618+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 55822 2023-10-21T04:38:04.618+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 10.217944ms 2023-10-21T04:38:04.618+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 12.939549ms 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex] 2023-10-21T04:38:04.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex closed 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex] 2023-10-21T04:38:04.623+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:38:04.623+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex] 2023-10-21T04:38:04.623+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex] 2023-10-21T04:38:04.623+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex 2023-10-21T04:38:04.623+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex stopped 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.623+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.634+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex] 2023-10-21T04:38:04.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex closed 2023-10-21T04:38:04.634+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.634+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.635+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.635+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex] 2023-10-21T04:38:04.635+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:38:04.635+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex] 2023-10-21T04:38:04.635+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex] 2023-10-21T04:38:04.635+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex 2023-10-21T04:38:04.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex stopped 2023-10-21T04:38:04.635+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.635+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.635+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.635+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.639+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex] 2023-10-21T04:38:04.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex closed 2023-10-21T04:38:04.640+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.640+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex ... 2023-10-21T04:38:04.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex ... 2023-10-21T04:38:04.640+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex] 2023-10-21T04:38:04.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex closed 2023-10-21T04:38:04.640+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.640+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.640+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.640+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex] 2023-10-21T04:38:04.641+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.641+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 43.448573ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:04.645+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex] 2023-10-21T04:38:04.645+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex closed 2023-10-21T04:38:04.645+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.645+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.646+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.646+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex] 2023-10-21T04:38:04.655+05:30 [Info] switched currmeta from 64 -> 66 force true 2023-10-21T04:38:04.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex ... 2023-10-21T04:38:04.656+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex ... 2023-10-21T04:38:04.656+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:38:04.657+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex] 2023-10-21T04:38:04.657+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex] 2023-10-21T04:38:04.657+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex 2023-10-21T04:38:04.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex stopped 2023-10-21T04:38:04.657+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.657+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.657+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.657+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex] 2023-10-21T04:38:04.658+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex closed 2023-10-21T04:38:04.659+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.659+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex ... 2023-10-21T04:38:04.660+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex ... 2023-10-21T04:38:04.680+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.687+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.691+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:38:04.691+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex] 2023-10-21T04:38:04.691+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex] 2023-10-21T04:38:04.691+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex 2023-10-21T04:38:04.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex] 2023-10-21T04:38:04.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex closed 2023-10-21T04:38:04.691+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.691+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex ... 2023-10-21T04:38:04.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex ... 2023-10-21T04:38:04.700+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex stopped 2023-10-21T04:38:04.701+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.701+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.701+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.702+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_7.index destroyed 2023-10-21T04:38:04.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex] 2023-10-21T04:38:04.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex closed 2023-10-21T04:38:04.703+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 7, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.703+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 7 Destroy Done 2023-10-21T04:38:04.703+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.703+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.703+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.703+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex] 2023-10-21T04:38:04.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex ... 2023-10-21T04:38:04.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex ... 2023-10-21T04:38:04.706+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.725+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index/mainIndex sucessfully destroyed 2023-10-21T04:38:04.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex] 2023-10-21T04:38:04.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex closed 2023-10-21T04:38:04.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex] 2023-10-21T04:38:04.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex closed 2023-10-21T04:38:04.729+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.729+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.729+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.729+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_8.index destroyed 2023-10-21T04:38:04.729+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 8, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.729+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 8 Destroy Done 2023-10-21T04:38:04.729+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.730+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.730+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.730+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex] 2023-10-21T04:38:04.730+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.731+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex] 2023-10-21T04:38:04.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex closed 2023-10-21T04:38:04.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex ... 2023-10-21T04:38:04.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex ... 2023-10-21T04:38:04.731+05:30 [Info] bucket_1/#primary/Mainstore#3643393537124379309:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.732+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:38:04.734+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex] 2023-10-21T04:38:04.734+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex] 2023-10-21T04:38:04.734+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex 2023-10-21T04:38:04.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex stopped 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex] 2023-10-21T04:38:04.734+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:38:04.734+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex] 2023-10-21T04:38:04.734+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex] 2023-10-21T04:38:04.734+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex 2023-10-21T04:38:04.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex stopped 2023-10-21T04:38:04.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex] 2023-10-21T04:38:04.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex closed 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.734+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.734+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex ... 2023-10-21T04:38:04.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex ... 2023-10-21T04:38:04.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.757+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:38:04.757+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex] 2023-10-21T04:38:04.757+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex] 2023-10-21T04:38:04.757+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex 2023-10-21T04:38:04.757+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:38:04.757+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex] 2023-10-21T04:38:04.757+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex] 2023-10-21T04:38:04.757+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex 2023-10-21T04:38:04.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex stopped 2023-10-21T04:38:04.757+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.757+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.757+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.757+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex stopped 2023-10-21T04:38:04.758+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.759+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.759+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.759+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.765+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.765+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.765+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex] 2023-10-21T04:38:04.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex closed 2023-10-21T04:38:04.766+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.766+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex ... 2023-10-21T04:38:04.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex ... 2023-10-21T04:38:04.781+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_1.index destroyed 2023-10-21T04:38:04.782+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 1, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.782+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 1 Destroy Done 2023-10-21T04:38:04.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex ... 2023-10-21T04:38:04.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex ... 2023-10-21T04:38:04.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_6.index destroyed 2023-10-21T04:38:04.790+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 6, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.790+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 6 Destroy Done 2023-10-21T04:38:04.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex ... 2023-10-21T04:38:04.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex ... 2023-10-21T04:38:04.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex] 2023-10-21T04:38:04.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex closed 2023-10-21T04:38:04.815+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.815+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.816+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex] 2023-10-21T04:38:04.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex closed 2023-10-21T04:38:04.816+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.816+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex ... 2023-10-21T04:38:04.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex ... 2023-10-21T04:38:04.821+05:30 [Info] switched currmeta from 66 -> 68 force true 2023-10-21T04:38:04.825+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 8732144615635911668 &{0} 2023-10-21T04:38:04.825+05:30 [Info] Indexer::handleDropIndex - IndexInstId 8732144615635911668 2023-10-21T04:38:04.826+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [8732144615635911668] 2023-10-21T04:38:04.826+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [8732144615635911668] 2023-10-21T04:38:04.826+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:04.826+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [8732144615635911668] 2023-10-21T04:38:04.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex ... 2023-10-21T04:38:04.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex ... 2023-10-21T04:38:04.828+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:04.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_5.index destroyed 2023-10-21T04:38:04.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex ... 2023-10-21T04:38:04.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.832+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.830+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 5, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.832+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 5 Destroy Done 2023-10-21T04:38:04.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex] 2023-10-21T04:38:04.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex closed 2023-10-21T04:38:04.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex ... 2023-10-21T04:38:04.834+05:30 [Info] bucket_1/idx_secondary/Backstore#4443661069773468873:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:04.834+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:04.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex ... 2023-10-21T04:38:04.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex ... 2023-10-21T04:38:04.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.838+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_2.index destroyed 2023-10-21T04:38:04.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex ... 2023-10-21T04:38:04.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex ... 2023-10-21T04:38:04.854+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 2, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.856+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 2 Destroy Done 2023-10-21T04:38:04.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/docIndex sucessfully destroyed 2023-10-21T04:38:04.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_3.index destroyed 2023-10-21T04:38:04.878+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 3, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.879+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 3 Destroy Done 2023-10-21T04:38:04.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex ... 2023-10-21T04:38:04.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex ... 2023-10-21T04:38:04.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:04.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_7516828288031590951_4.index destroyed 2023-10-21T04:38:04.890+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7516828288031590951, PartitionId 4, IndexDefnId 16961575949607441659. 2023-10-21T04:38:04.890+05:30 [Info] Indexer::cleanupIndexData IndexInst 7516828288031590951 Partition 4 Destroy Done 2023-10-21T04:38:04.891+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 3, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.891+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.891+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.891+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:38:04.892+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:38:04.892+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:38:04.892+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:38:04.892+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex 2023-10-21T04:38:04.897+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex stopped 2023-10-21T04:38:04.895+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 5, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.895+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 6, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.895+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 7, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.896+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 8732144615635911668 Defn: DefnId: 10606939961365971246 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] [17406544375239214575 10170029854712240877]} 2:{2 1 [:9123] [17406544375239214575 10170029854712240877]} 3:{3 1 [:9123] [17406544375239214575 10170029854712240877]} 4:{4 1 [:9123] [17406544375239214575 10170029854712240877]} 5:{5 1 [:9123] [17406544375239214575 10170029854712240877]} 6:{6 1 [:9123] [17406544375239214575 10170029854712240877]} 7:{7 1 [:9123] [17406544375239214575 10170029854712240877]} 8:{8 1 [:9123] [17406544375239214575 10170029854712240877]}] 8 KEY 0} 2023-10-21T04:38:04.896+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 8, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.896+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 1, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.896+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 2, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.896+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8732144615635911668, PartitionId 4, IndexDefnId 10606939961365971246. 2023-10-21T04:38:04.896+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.897+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.898+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.898+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.898+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.898+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 73.252443ms 2023-10-21T04:38:04.898+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 73.320247ms 2023-10-21T04:38:04.898+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 8732144615635911668 2023-10-21T04:38:04.899+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.899+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.902+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.902+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:04.905+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:38:04.909+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex 2023-10-21T04:38:04.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex stopped 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex 2023-10-21T04:38:04.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex stopped 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:38:04.910+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex 2023-10-21T04:38:04.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex stopped 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.910+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:38:04.915+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:38:04.915+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex 2023-10-21T04:38:04.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex stopped 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:38:04.915+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:38:04.915+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex 2023-10-21T04:38:04.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex stopped 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:38:04.915+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:38:04.915+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:38:04.916+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex 2023-10-21T04:38:04.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex stopped 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:04.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:38:04.916+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:38:04.916+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex 2023-10-21T04:38:04.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex stopped 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:04.916+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:04.952+05:30 [Info] switched currmeta from 68 -> 69 force true 2023-10-21T04:38:04.979+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:04.979+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:04.989+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 212.301504ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:04.995+05:30 [Info] switched currmeta from 69 -> 69 force true 2023-10-21T04:38:05.054+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.054+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex] 2023-10-21T04:38:05.054+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex closed 2023-10-21T04:38:05.061+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.061+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.062+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.062+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.062+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.072+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:38:05.073+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:38:05.075+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:38:05.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.067+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.080+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:38:05.088+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex 2023-10-21T04:38:05.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex stopped 2023-10-21T04:38:05.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex] 2023-10-21T04:38:05.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex closed 2023-10-21T04:38:05.090+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.090+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.091+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex] 2023-10-21T04:38:05.091+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex closed 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:38:05.092+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.092+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex stopped 2023-10-21T04:38:05.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.092+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:38:05.092+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex 2023-10-21T04:38:05.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex stopped 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.092+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex] 2023-10-21T04:38:05.092+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex closed 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:38:05.093+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:38:05.093+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex 2023-10-21T04:38:05.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex stopped 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex] 2023-10-21T04:38:05.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex closed 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:38:05.094+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:38:05.094+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex 2023-10-21T04:38:05.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex stopped 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex] 2023-10-21T04:38:05.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex closed 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:38:05.095+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:38:05.095+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex 2023-10-21T04:38:05.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex stopped 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.095+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex] 2023-10-21T04:38:05.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex closed 2023-10-21T04:38:05.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:38:05.097+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:38:05.098+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:38:05.098+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:38:05.098+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex 2023-10-21T04:38:05.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex stopped 2023-10-21T04:38:05.098+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.098+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.098+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.098+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.107+05:30 [Info] switched currmeta from 69 -> 71 force true 2023-10-21T04:38:05.146+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex] 2023-10-21T04:38:05.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex closed 2023-10-21T04:38:05.151+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8732144615635911668:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.151+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.151+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.152+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.154+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.154+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:38:05.155+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:38:05.155+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:38:05.162+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:38:05.163+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex 2023-10-21T04:38:05.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex stopped 2023-10-21T04:38:05.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.164+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.164+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.164+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.166+05:30 [Info] switched currmeta from 71 -> 71 force true 2023-10-21T04:38:05.204+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex] 2023-10-21T04:38:05.204+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex closed 2023-10-21T04:38:05.204+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.204+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:38:05.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex ... 2023-10-21T04:38:05.231+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:38:05.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex ... 2023-10-21T04:38:05.238+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex] 2023-10-21T04:38:05.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex closed 2023-10-21T04:38:05.242+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.242+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:38:05.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex ... 2023-10-21T04:38:05.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.248+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_3.index destroyed 2023-10-21T04:38:05.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:38:05.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex ... 2023-10-21T04:38:05.251+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 3, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.251+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 3 Destroy Done 2023-10-21T04:38:05.254+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex] 2023-10-21T04:38:05.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex closed 2023-10-21T04:38:05.258+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.258+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:38:05.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex ... 2023-10-21T04:38:05.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_5.index destroyed 2023-10-21T04:38:05.261+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 5, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.261+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 5 Destroy Done 2023-10-21T04:38:05.262+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.266+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.266+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:38:05.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex ... 2023-10-21T04:38:05.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex] 2023-10-21T04:38:05.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex closed 2023-10-21T04:38:05.269+05:30 [Info] switched currmeta from 71 -> 72 force true 2023-10-21T04:38:05.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex] 2023-10-21T04:38:05.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex closed 2023-10-21T04:38:05.275+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.275+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:38:05.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex ... 2023-10-21T04:38:05.271+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 17434304260397056806 &{0} 2023-10-21T04:38:05.277+05:30 [Info] Indexer::handleDropIndex - IndexInstId 17434304260397056806 2023-10-21T04:38:05.277+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.277+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:38:05.277+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:05.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.280+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.281+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17434304260397056806 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:38:05.281+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [17434304260397056806] 2023-10-21T04:38:05.283+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [17434304260397056806] 2023-10-21T04:38:05.283+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:05.283+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [17434304260397056806] 2023-10-21T04:38:05.284+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:38:05.284+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:38:05.284+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:05.284+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc004dda3c0 }] 2023-10-21T04:38:05.284+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 43196 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:05.284+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:38:05.284+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:38:05.284+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:38:05.284+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:38:05.284+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:38:05.285+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 8.011041ms 2023-10-21T04:38:05.283+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.285+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0, IndexDefnId 3381524789655566900. 2023-10-21T04:38:05.285+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.285+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.285+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:38:05.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex] 2023-10-21T04:38:05.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex closed 2023-10-21T04:38:05.286+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.286+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:38:05.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex ... 2023-10-21T04:38:05.288+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2] 2023-10-21T04:38:05.288+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 17434304260397056806 2023-10-21T04:38:05.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.289+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_2.index destroyed 2023-10-21T04:38:05.290+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 2, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.291+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 2 Destroy Done 2023-10-21T04:38:05.290+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:38:05.291+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:38:05.292+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 0 274143649547516 29 2023-10-21T04:38:05.293+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.295+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 1 162049908746825 29 2023-10-21T04:38:05.295+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.295+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 2 206082810619393 24 2023-10-21T04:38:05.295+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 7 78446412189949 23 2023-10-21T04:38:05.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.301+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_%2] 2023-10-21T04:38:05.303+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 5 212967951483349 28 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 6 116493903186721 22 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 8 263088038522959 18 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 9 136568163328749 23 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 10 261951617336488 35 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 11 127746572208200 19 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 3 120349894144442 23 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 4 143687065777286 23 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 12 58733680850130 23 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 15 277338544220217 21 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 16 245247859472006 21 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 17 206088069380492 13 2023-10-21T04:38:05.311+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.311+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 18 34510046292135 19 2023-10-21T04:38:05.312+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.312+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 19 250426459207023 29 2023-10-21T04:38:05.312+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.312+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 13 188740876924377 22 2023-10-21T04:38:05.312+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.312+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 14 173457131919400 12 2023-10-21T04:38:05.312+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.313+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 20 274158831830257 20 2023-10-21T04:38:05.313+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.313+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 37 169268048765249 19 2023-10-21T04:38:05.313+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.313+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 35 127702263378100 23 2023-10-21T04:38:05.313+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.314+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 40 106297365959267 27 2023-10-21T04:38:05.314+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.314+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 46 162870112426804 23 2023-10-21T04:38:05.314+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.315+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 48 264001907139144 22 2023-10-21T04:38:05.315+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex] 2023-10-21T04:38:05.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex closed 2023-10-21T04:38:05.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex] 2023-10-21T04:38:05.314+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 43196 2023-10-21T04:38:05.316+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 31.427219ms 2023-10-21T04:38:05.316+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 39.722126ms 2023-10-21T04:38:05.315+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 26 137256376714252 22 2023-10-21T04:38:05.317+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex closed 2023-10-21T04:38:05.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:38:05.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex ... 2023-10-21T04:38:05.316+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': swap is disabled 2023-10-21T04:38:05.319+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.319+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 1 Close Done 2023-10-21T04:38:05.317+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 58 136038310030103 28 2023-10-21T04:38:05.319+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8732144615635911668:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.321+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.321+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.322+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 34 168446351781198 25 2023-10-21T04:38:05.323+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.323+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.324+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.324+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 52 84581802233088 22 2023-10-21T04:38:05.325+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.325+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 59 97677619676504 31 2023-10-21T04:38:05.325+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.325+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 27 5906787184229 20 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 50 266604622117081 26 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 51 32274016651656 23 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 38 245900669851618 19 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 39 149033089785397 28 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 62 27479155597282 29 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 30 41101832875819 19 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 63 161870047388121 25 2023-10-21T04:38:05.326+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.326+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 31 241391494503904 28 2023-10-21T04:38:05.327+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.327+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 23 191500187207978 21 2023-10-21T04:38:05.327+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.327+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 55 272523968790649 22 2023-10-21T04:38:05.327+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.327+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 47 132470209691538 26 2023-10-21T04:38:05.327+05:30 [Info] switched currmeta from 72 -> 73 force true 2023-10-21T04:38:05.327+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.327+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 42 39045953044628 18 2023-10-21T04:38:05.327+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.327+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 45 110286285845649 35 2023-10-21T04:38:05.327+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.328+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 49 63737850123332 19 2023-10-21T04:38:05.328+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.328+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 54 241607113601772 23 2023-10-21T04:38:05.328+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.328+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 22 35716737784090 24 2023-10-21T04:38:05.331+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.332+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 56 77598347829810 19 2023-10-21T04:38:05.333+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.333+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 24 106231614376481 21 2023-10-21T04:38:05.334+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.334+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:38:05.335+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex ... 2023-10-21T04:38:05.335+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 21 125696667341112 27 2023-10-21T04:38:05.336+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.344+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 53 225662991941187 26 2023-10-21T04:38:05.345+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.348+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.349+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.350+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 32 121450709505223 25 2023-10-21T04:38:05.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:38:05.350+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex ... 2023-10-21T04:38:05.351+05:30 [Info] switched currmeta from 73 -> 73 force true 2023-10-21T04:38:05.350+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 43 160399698726237 29 2023-10-21T04:38:05.351+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 60 72072509155168 23 2023-10-21T04:38:05.351+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 28 146375746634523 17 2023-10-21T04:38:05.351+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 33 216238759475962 23 2023-10-21T04:38:05.351+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 41 88718752743571 25 2023-10-21T04:38:05.351+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 29 112482982607415 23 2023-10-21T04:38:05.351+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 61 105415986286631 18 2023-10-21T04:38:05.351+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.351+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 25 118920785117980 21 2023-10-21T04:38:05.352+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.352+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 57 197957220074784 24 2023-10-21T04:38:05.352+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.352+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 44 217237947818243 20 2023-10-21T04:38:05.352+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.352+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_%2 36 100008922352935 17 2023-10-21T04:38:05.352+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_%2 Stream MAINT_STREAM. Ignored. 2023-10-21T04:38:05.354+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 128.580826ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:05.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10170029854712240877(10170029854712240877)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.381+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.381+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.391+05:30 [Info] bucket_%2_b9538b29d7e025c819e734b646db039c: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:38:05.391+05:30 [Info] controller 'bucket_%2_b9538b29d7e025c819e734b646db039c/bucket_%2_b9538b29d7e025c819e734b646db039c': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex 2023-10-21T04:38:05.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex stopped 2023-10-21T04:38:05.392+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.392+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:38:05.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex ... 2023-10-21T04:38:05.392+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.393+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.397+05:30 [Info] switched currmeta from 73 -> 76 force true 2023-10-21T04:38:05.400+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index/mainIndex sucessfully destroyed 2023-10-21T04:38:05.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex] 2023-10-21T04:38:05.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex closed 2023-10-21T04:38:05.405+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#8965923595543532942:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.405+05:30 [Info] Indexer::cleanupIndexData IndexInst 17434304260397056806 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... 2023-10-21T04:38:05.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_7.index destroyed 2023-10-21T04:38:05.408+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 7, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.408+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 7 Destroy Done 2023-10-21T04:38:05.408+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.409+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.411+05:30 [Info] switched currmeta from 76 -> 76 force true 2023-10-21T04:38:05.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_6.index destroyed 2023-10-21T04:38:05.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:38:05.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex ... 2023-10-21T04:38:05.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_8.index destroyed 2023-10-21T04:38:05.417+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 8, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.417+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 8 Destroy Done 2023-10-21T04:38:05.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:38:05.417+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex ... 2023-10-21T04:38:05.415+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 6, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.418+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 6 Destroy Done 2023-10-21T04:38:05.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_4.index destroyed 2023-10-21T04:38:05.421+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 4, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.421+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 4 Destroy Done 2023-10-21T04:38:05.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:38:05.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex ... 2023-10-21T04:38:05.427+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8732144615635911668_1.index destroyed 2023-10-21T04:38:05.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex ... 2023-10-21T04:38:05.428+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex ... 2023-10-21T04:38:05.430+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8732144615635911668, PartitionId 1, IndexDefnId 10606939961365971246. 2023-10-21T04:38:05.430+05:30 [Info] Indexer::cleanupIndexData IndexInst 8732144615635911668 Partition 1 Destroy Done 2023-10-21T04:38:05.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17406544375239214575(17406544375239214575)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17434304260397056806_0.index destroyed 2023-10-21T04:38:05.439+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17434304260397056806, PartitionId 0, IndexDefnId 3381524789655566900. 2023-10-21T04:38:05.439+05:30 [Info] Indexer::cleanupIndexData IndexInst 17434304260397056806 Partition 0 Destroy Done 2023-10-21T04:38:05.467+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 642374041935721654 &{0} 2023-10-21T04:38:05.468+05:30 [Info] Indexer::handleDropIndex - IndexInstId 642374041935721654 2023-10-21T04:38:05.468+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:38:05.468+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:05.469+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 642374041935721654 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:38:05.469+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [642374041935721654] 2023-10-21T04:38:05.470+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [642374041935721654] 2023-10-21T04:38:05.471+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:05.472+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [642374041935721654] 2023-10-21T04:38:05.472+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:38:05.472+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:38:05.472+05:30 [Info] plasmaSlice::Close Soft Closing Slice Id 0, IndexInstId 642374041935721654, PartitionId 0, IndexDefnId 4286621850367430051 2023-10-21T04:38:05.472+05:30 [Info] Indexer::cleanupIndexData IndexInst 642374041935721654 Partition 0 Close Done 2023-10-21T04:38:05.472+05:30 [Info] plasmaSlice::Destroy Soft deleted Slice Id 0, IndexInstId 642374041935721654, PartitionId 0 IndexDefnId 4286621850367430051 RefCount 1 NumOpenSnapshots 0 2023-10-21T04:38:05.472+05:30 [Info] Indexer::cleanupIndexData IndexInst 642374041935721654 Partition 0 Destroy Done 2023-10-21T04:38:05.472+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:05.473+05:30 [Info] MutationStreamReader:Shutdown StreamReader MAINT_STREAM 2023-10-21T04:38:05.473+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_1 total received key-versions: map[0:0 1:1 2:0 3:0 4:1 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:1 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:1 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:1 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:38:05.473+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_1 latest sequence numbers: map[0:0 1:28 2:0 3:0 4:19 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:31 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:26 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:26 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:38:05.473+05:30 [Error] DATP[->dataport ":9123"] Accept() Error: accept tcp4 0.0.0.0:9123: use of closed network connection 2023-10-21T04:38:05.474+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9123->127.0.0.1:35652: use of closed network connection 2023-10-21T04:38:05.474+05:30 [Error] DATP[->dataport ":9123"] worker "127.0.0.1:35652" exit: read tcp4 127.0.0.1:9123->127.0.0.1:35652: use of closed network connection 2023-10-21T04:38:05.474+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:35652" closed ! 2023-10-21T04:38:05.474+05:30 [Info] DATP[->dataport ":9123"] ... stopped 2023-10-21T04:38:05.474+05:30 [Error] DATP[->dataport ":9123"] fatal remote "127.0.0.1:35652" already gone 2023-10-21T04:38:05.474+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 55822 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:38:05.474+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:38:05.474+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:38:05.474+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:38:05.474+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 6.427008ms 2023-10-21T04:38:05.474+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 642374041935721654 2023-10-21T04:38:05.475+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1] 2023-10-21T04:38:05.478+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 642374041935721654, PartitionId 0, IndexDefnId 4286621850367430051. 2023-10-21T04:38:05.479+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.479+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.480+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex] 2023-10-21T04:38:05.480+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex 2023-10-21T04:38:05.480+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex] 2023-10-21T04:38:05.480+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex] 2023-10-21T04:38:05.480+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex 2023-10-21T04:38:05.480+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex stopped 2023-10-21T04:38:05.480+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.480+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.480+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.480+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.488+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex] 2023-10-21T04:38:05.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex closed 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Mainstore#642374041935721654:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex] 2023-10-21T04:38:05.489+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex 2023-10-21T04:38:05.489+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex] 2023-10-21T04:38:05.489+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex] 2023-10-21T04:38:05.489+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex 2023-10-21T04:38:05.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex stopped 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.489+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.490+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_18194302e3f10df59bd076e1c4142d0b KeyspaceIds [bucket_1]. Err genServer.closed 2023-10-21T04:38:05.490+05:30 [Error] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:38:05.490+05:30 [Info] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_1] Treating genServer.closed As Success 2023-10-21T04:38:05.490+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 55822 2023-10-21T04:38:05.490+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 16.102732ms 2023-10-21T04:38:05.490+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 22.598858ms 2023-10-21T04:38:05.500+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex] 2023-10-21T04:38:05.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex closed 2023-10-21T04:38:05.501+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': swap is disabled 2023-10-21T04:38:05.501+05:30 [Info] bucket_1/idx_secondary/Backstore#642374041935721654:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex ... 2023-10-21T04:38:05.501+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex ... 2023-10-21T04:38:05.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex ... 2023-10-21T04:38:05.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 55 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex ... 2023-10-21T04:38:05.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_642374041935721654_0.index destroyed 2023-10-21T04:38:05.515+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 642374041935721654, PartitionId 0, IndexDefnId 4286621850367430051. 2023-10-21T04:38:05.526+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 64.667825ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:05.565+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. err: operation_canceled 2023-10-21T04:38:05.566+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:38:05.566+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:38:05.566+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:38:05.567+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[18194302e3f10df59bd076e1c4142d0b 8806f5706747af4e635b8da5fcd8aeee] IsBalanced:false Messages:[]} 2023-10-21T04:38:05.574+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:38:05.575+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:38:05.626+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.626+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.643+05:30 [Info] switched currmeta from 76 -> 80 force true 2023-10-21T04:38:05.659+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.659+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.679+05:30 [Info] switched currmeta from 80 -> 80 force true 2023-10-21T04:38:05.681+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 3580008349810282285 &{0} 2023-10-21T04:38:05.681+05:30 [Info] Indexer::handleDropIndex - IndexInstId 3580008349810282285 2023-10-21T04:38:05.681+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [3580008349810282285] 2023-10-21T04:38:05.681+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [3580008349810282285] 2023-10-21T04:38:05.681+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843222668465492 bucket_1:1697843222622515965] 2023-10-21T04:38:05.681+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [3580008349810282285] 2023-10-21T04:38:05.681+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:38:05.681+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:38:05.681+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:38:05.681+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:38:05.681+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex 2023-10-21T04:38:05.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex stopped 2023-10-21T04:38:05.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.681+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.683+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.683+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.683+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.683+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:38:05.683+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:38:05.683+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:38:05.683+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:38:05.683+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex 2023-10-21T04:38:05.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex stopped 2023-10-21T04:38:05.684+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.684+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.684+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.684+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.684+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.684+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.684+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.684+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:38:05.684+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:38:05.684+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:38:05.684+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:38:05.684+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex 2023-10-21T04:38:05.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex stopped 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.685+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:38:05.685+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:38:05.685+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:38:05.685+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:38:05.685+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex 2023-10-21T04:38:05.685+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex stopped 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.685+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.685+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 3580008349810282285 Defn: DefnId: 16337157693634799600 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 2 ReplicaId: 0 PartitionContainer: &{map[1:{1 2 [:9123] [4718895791748289908 15531034489713578274]} 2:{2 2 [:9123] [4718895791748289908 15531034489713578274]} 3:{3 2 [:9123] [4718895791748289908 15531034489713578274]} 4:{4 2 [:9123] [4718895791748289908 15531034489713578274]} 5:{5 2 [:9123] [4718895791748289908 15531034489713578274]} 6:{6 2 [:9123] [4718895791748289908 15531034489713578274]} 7:{7 2 [:9123] [4718895791748289908 15531034489713578274]} 8:{8 2 [:9123] [4718895791748289908 15531034489713578274]}] 8 KEY 0} 2023-10-21T04:38:05.686+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 5.200615ms 2023-10-21T04:38:05.686+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 5.24403ms 2023-10-21T04:38:05.686+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.686+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.686+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.686+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:38:05.686+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:38:05.686+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:38:05.686+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:38:05.686+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex 2023-10-21T04:38:05.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex stopped 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.687+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:38:05.687+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:38:05.687+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:38:05.687+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:38:05.687+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex 2023-10-21T04:38:05.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex stopped 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.687+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.688+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 3580008349810282285 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex] 2023-10-21T04:38:05.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex closed 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:38:05.689+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:38:05.689+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:38:05.689+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:38:05.689+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex 2023-10-21T04:38:05.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex stopped 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.689+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.689+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.690+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:38:05.690+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:38:05.690+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:38:05.690+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:38:05.690+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex 2023-10-21T04:38:05.690+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex stopped 2023-10-21T04:38:05.690+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.690+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.690+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.690+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.696+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 3580008349810282285, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.696+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.696+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.696+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:38:05.697+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:38:05.697+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:38:05.697+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:38:05.697+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex 2023-10-21T04:38:05.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex stopped 2023-10-21T04:38:05.697+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.697+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.697+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.697+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.731+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex] 2023-10-21T04:38:05.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex closed 2023-10-21T04:38:05.731+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.731+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex ... 2023-10-21T04:38:05.732+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex ... 2023-10-21T04:38:05.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex ... 2023-10-21T04:38:05.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex ... 2023-10-21T04:38:05.771+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_6.index destroyed 2023-10-21T04:38:05.774+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 6, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.774+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 6 Destroy Done 2023-10-21T04:38:05.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex] 2023-10-21T04:38:05.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex closed 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:38:05.775+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:38:05.775+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:38:05.775+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:38:05.775+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex 2023-10-21T04:38:05.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex stopped 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.775+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.777+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex] 2023-10-21T04:38:05.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex closed 2023-10-21T04:38:05.778+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.778+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.779+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.779+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:38:05.779+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:38:05.779+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:38:05.779+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:38:05.779+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex 2023-10-21T04:38:05.780+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex stopped 2023-10-21T04:38:05.780+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.780+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.780+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.780+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex] 2023-10-21T04:38:05.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex closed 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:38:05.782+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:38:05.782+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:38:05.782+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:38:05.782+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex 2023-10-21T04:38:05.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex stopped 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.782+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.785+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex] 2023-10-21T04:38:05.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex closed 2023-10-21T04:38:05.785+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.785+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex ... 2023-10-21T04:38:05.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 50 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex ... 2023-10-21T04:38:05.785+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex] 2023-10-21T04:38:05.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex closed 2023-10-21T04:38:05.787+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.787+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.791+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.791+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.788+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.789+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.791+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:38:05.792+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex] 2023-10-21T04:38:05.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex closed 2023-10-21T04:38:05.792+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.792+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.792+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.792+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:38:05.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex ... 2023-10-21T04:38:05.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex ... 2023-10-21T04:38:05.792+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:38:05.793+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:38:05.793+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:38:05.793+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex 2023-10-21T04:38:05.794+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 119.897637ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:05.796+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:38:05.796+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:38:05.796+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:38:05.796+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex 2023-10-21T04:38:05.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex stopped 2023-10-21T04:38:05.796+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.796+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.796+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.796+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex stopped 2023-10-21T04:38:05.798+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.798+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.798+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.798+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.807+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex] 2023-10-21T04:38:05.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex closed 2023-10-21T04:38:05.813+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.814+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex ... 2023-10-21T04:38:05.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex ... 2023-10-21T04:38:05.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.847+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:05.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_7.index destroyed 2023-10-21T04:38:05.861+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 7, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.861+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 7 Destroy Done 2023-10-21T04:38:05.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex] 2023-10-21T04:38:05.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex closed 2023-10-21T04:38:05.861+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.861+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.861+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.861+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:38:05.861+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:38:05.861+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:38:05.861+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:38:05.862+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex 2023-10-21T04:38:05.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex stopped 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex] 2023-10-21T04:38:05.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex closed 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#3580008349810282285:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:38:05.862+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:38:05.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:38:05.862+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:38:05.862+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:38:05.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex ... 2023-10-21T04:38:05.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex ... 2023-10-21T04:38:05.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.904+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_1.index destroyed 2023-10-21T04:38:05.906+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 1, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.907+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 1 Destroy Done 2023-10-21T04:38:05.911+05:30 [Info] switched currmeta from 80 -> 83 force true 2023-10-21T04:38:05.911+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.914+05:30 [Info] bucket_1_1fc3689e709c62b9ee184f2ec13e6f43: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:38:05.944+05:30 [Info] controller 'bucket_1_1fc3689e709c62b9ee184f2ec13e6f43/bucket_1_1fc3689e709c62b9ee184f2ec13e6f43': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex 2023-10-21T04:38:05.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex stopped 2023-10-21T04:38:05.944+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:38:05.944+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:38:05.944+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:38:05.944+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:38:05.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex] 2023-10-21T04:38:05.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex closed 2023-10-21T04:38:05.944+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.944+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex ... 2023-10-21T04:38:05.944+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 53 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex ... 2023-10-21T04:38:05.953+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex ... 2023-10-21T04:38:05.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 54 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex ... 2023-10-21T04:38:05.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_2.index destroyed 2023-10-21T04:38:05.993+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 2, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.993+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 2 Destroy Done 2023-10-21T04:38:05.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex] 2023-10-21T04:38:05.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex closed 2023-10-21T04:38:05.993+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.993+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex ... 2023-10-21T04:38:05.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 51 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex ... 2023-10-21T04:38:05.994+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:05.994+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:05.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/docIndex sucessfully destroyed 2023-10-21T04:38:05.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex ... 2023-10-21T04:38:05.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 52 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex ... 2023-10-21T04:38:05.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:05.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_8.index destroyed 2023-10-21T04:38:05.998+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 8, IndexDefnId 16337157693634799600. 2023-10-21T04:38:05.998+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 8 Destroy Done 2023-10-21T04:38:05.998+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:05.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex] 2023-10-21T04:38:05.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex closed 2023-10-21T04:38:05.998+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:05.998+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:05.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex ... 2023-10-21T04:38:05.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex ... 2023-10-21T04:38:06.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:06.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/docIndex sucessfully destroyed 2023-10-21T04:38:06.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex ... 2023-10-21T04:38:06.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex ... 2023-10-21T04:38:06.002+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:06.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex] 2023-10-21T04:38:06.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex closed 2023-10-21T04:38:06.002+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:06.002+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:06.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex ... 2023-10-21T04:38:06.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 48 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex ... 2023-10-21T04:38:06.002+05:30 [Info] switched currmeta from 83 -> 84 force true 2023-10-21T04:38:06.004+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:38:06.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:06.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_4.index destroyed 2023-10-21T04:38:06.012+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:06.012+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:06.016+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:06.021+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 4, IndexDefnId 16337157693634799600. 2023-10-21T04:38:06.021+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 4 Destroy Done 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/docIndex sucessfully destroyed 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex ... 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 49 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex ... 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex] 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex closed 2023-10-21T04:38:06.029+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#3580008349810282285:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:38:06.029+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex ... 2023-10-21T04:38:06.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : removed plasmaId 46 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex ... 2023-10-21T04:38:06.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : metadata saved successfully 2023-10-21T04:38:06.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard15531034489713578274(15531034489713578274)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/docIndex sucessfully destroyed 2023-10-21T04:38:06.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:06.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_5.index destroyed 2023-10-21T04:38:06.033+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 5, IndexDefnId 16337157693634799600. 2023-10-21T04:38:06.033+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 5 Destroy Done 2023-10-21T04:38:06.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex ... 2023-10-21T04:38:06.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : removed plasmaId 47 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex ... 2023-10-21T04:38:06.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : metadata saved successfully 2023-10-21T04:38:06.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard4718895791748289908(4718895791748289908)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_3580008349810282285_3.index destroyed 2023-10-21T04:38:06.036+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 3580008349810282285, PartitionId 3, IndexDefnId 16337157693634799600. 2023-10-21T04:38:06.036+05:30 [Info] Indexer::cleanupIndexData IndexInst 3580008349810282285 Partition 3 Destroy Done 2023-10-21T04:38:06.216+05:30 WARN REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:360 [id 189] [bucket_1] unexpected error on bucket 127.0.0.1:9003 observe; will retry after 100ms: EOF 2023-10-21T04:38:06.233+05:30 [Warn] Client:runObserveStreamingEndpoint streaming endpoint for /pools/default/bs/bucket_1 returned err EOF 2023-10-21T04:38:06.233+05:30 [Warn] serviceChangeNotifier: Connection terminated for collection manifest notifier instance of http://%40index@127.0.0.1:9003-default, bucket: bucket_1, (EOF) 2023-10-21T04:38:06.233+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /poolsStreaming/default 2023-10-21T04:38:06.233+05:30 [Info] clusterInfoCacheLiteManager watchClusterChanges: restarting.. 2023-10-21T04:38:06.233+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /pools/default/bs/bucket_%2 2023-10-21T04:38:06.233+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /pools/default/nodeServicesStreaming 2023-10-21T04:38:06.233+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /poolsStreaming/default 2023-10-21T04:38:06.233+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /pools/default/bs/bucket_%2 2023-10-21T04:38:06.233+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /pools/default/nodeServicesStreaming 2023-10-21T04:38:06.307+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 218] bucketNames has been updated (was map[bucket_%2:b9538b29d7e025c819e734b646db039c bucket_1:1fc3689e709c62b9ee184f2ec13e6f43], now map[bucket_%2:b9538b29d7e025c819e734b646db039c]) 2023-10-21T04:38:06.307+05:30 INFO REGU.impl.(*meteringService).watchForBucketUpdates.func1() at metrics.go:129 [id 218] bucket bucket_1 has been dropped, will reset its metrics 2023-10-21T04:38:06.307+05:30 DEBU REGU.impl.dumpStats() at aggrecorder.go:364 [id 218] aggRecorder: [bucket_1] recorded 22312 aborted 0 refunded 0 2023-10-21T04:38:06.657+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9968469015920487, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:38:07.236+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:38:07.238+05:30 [Info] clusterInfoCacheLiteManager: watchClusterChanges sent notification on self restart 2023-10-21T04:38:07.238+05:30 [Info] handleBucketInfoChanges: stopped observing collection manifest for bucket bucket_1 2023-10-21T04:38:07.238+05:30 [Info] handleCollectionManifestChanges: stopped observing collection manifest for bucket bucket_1 2023-10-21T04:38:07.245+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:38:07.250+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:38:07.250+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:07.262+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:07.262+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:07.265+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:07.287+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:07.287+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:07.289+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:07.657+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9280322989290681, throttleDelayMs (new, old, change): (3, 5, -2) 2023-10-21T04:38:07.877+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:07.950+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:07.950+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:07.978+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:08.014+05:30 [Warn] Client:runObserveStreamingEndpoint streaming endpoint for /pools/default/bs/bucket_%2 returned err EOF 2023-10-21T04:38:08.014+05:30 [Warn] serviceChangeNotifier: Connection terminated for collection manifest notifier instance of http://%40index@127.0.0.1:9003-default, bucket: bucket_%2, (EOF) 2023-10-21T04:38:08.014+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /poolsStreaming/default 2023-10-21T04:38:08.014+05:30 [Info] clusterInfoCacheLiteManager watchClusterChanges: restarting.. 2023-10-21T04:38:08.014+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint for /pools/default/nodeServicesStreaming 2023-10-21T04:38:08.015+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /poolsStreaming/default 2023-10-21T04:38:08.015+05:30 [Info] Client:runObserveStreamingEndpoint closing streaming endpoint /pools/default/nodeServicesStreaming 2023-10-21T04:38:08.052+05:30 WARN REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:360 [id 188] [bucket_%2] unexpected error on bucket 127.0.0.1:9003 observe; will retry after 100ms: EOF 2023-10-21T04:38:08.097+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 218] bucketNames has been updated (was map[bucket_%2:b9538b29d7e025c819e734b646db039c], now map[]) 2023-10-21T04:38:08.097+05:30 INFO REGU.impl.(*meteringService).watchForBucketUpdates.func1() at metrics.go:129 [id 218] bucket bucket_%2 has been dropped, will reset its metrics 2023-10-21T04:38:08.097+05:30 DEBU REGU.impl.dumpStats() at aggrecorder.go:364 [id 218] aggRecorder: [bucket_%2] recorded 21635 aborted 0 refunded 0 2023-10-21T04:38:08.657+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9049110169745771, throttleDelayMs (new, old, change): (0, 3, -3) 2023-10-21T04:38:09.019+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:38:09.020+05:30 [Info] clusterInfoCacheLiteManager: watchClusterChanges sent notification on self restart 2023-10-21T04:38:09.020+05:30 [Info] handleBucketInfoChanges: stopped observing collection manifest for bucket bucket_%2 2023-10-21T04:38:09.020+05:30 [Info] handleCollectionManifestChanges: stopped observing collection manifest for bucket bucket_%2 2023-10-21T04:38:09.023+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:38:09.025+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:09.054+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:09.054+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:09.056+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:09.062+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:09.062+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:09.064+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:10.608+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:10.613+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:10.613+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:10.615+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:23.822+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:38:23.885+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:23.885+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:23.888+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:23.898+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:23.911+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 218] updated cluster rebalancing status to true 2023-10-21T04:38:23.911+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 218] updating rebalancing status to true 2023-10-21T04:38:23.917+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:23.917+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:23.920+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:24.004+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:38:24.031+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:24.031+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:38:24.048+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:38:24.083+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {937b7004839fdaed0d1098797d8cf481 [] topology-change-rebalance [] [{18194302e3f10df59bd076e1c4142d0b 7060000 } {8806f5706747af4e635b8da5fcd8aeee 7060000 }]} 2023-10-21T04:38:24.089+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.447µs 2023-10-21T04:38:24.089+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:38:24.089+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:38:24.089+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:38:24.090+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:38:24.091+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:38:24.091+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 1.932883ms 2023-10-21T04:38:24.091+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:38:24.091+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:38:24.091+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:true Messages:[]} 2023-10-21T04:38:24.091+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 5.285µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:24.091+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/937b7004839fdaed0d1098797d8cf481 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:937b7004839fdaed0d1098797d8cf481]}]} 2023-10-21T04:38:24.092+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:38:24.093+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {937b7004839fdaed0d1098797d8cf481 [] topology-change-rebalance [] [{18194302e3f10df59bd076e1c4142d0b 7060000 } {8806f5706747af4e635b8da5fcd8aeee 7060000 }]} 2023-10-21T04:38:24.095+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:38:24.095+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:38:24.111+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"18194302e3f10df59bd076e1c4142d0b","RebalId":"937b7004839fdaed0d1098797d8cf481","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:38:24.113+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{18194302e3f10df59bd076e1c4142d0b 937b7004839fdaed0d1098797d8cf481 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:38:24.115+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{18194302e3f10df59bd076e1c4142d0b 937b7004839fdaed0d1098797d8cf481 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:38:24.115+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:38:25.127+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:38:25.127+05:30 [Info] NewShardRebalancer nodeId 18194302e3f10df59bd076e1c4142d0b rebalToken &{18194302e3f10df59bd076e1c4142d0b 937b7004839fdaed0d1098797d8cf481 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:38:25.127+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:38:25.127+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.033452527s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:38:25.127+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{18194302e3f10df59bd076e1c4142d0b 937b7004839fdaed0d1098797d8cf481 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:38:25.127+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/937b7004839fdaed0d1098797d8cf481 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:937b7004839fdaed0d1098797d8cf481]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/937b7004839fdaed0d1098797d8cf481 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:937b7004839fdaed0d1098797d8cf481]}]} 2023-10-21T04:38:25.127+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:true Messages:[]} 2023-10-21T04:38:25.128+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:38:25.130+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:38:25.130+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/937b7004839fdaed0d1098797d8cf481 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:937b7004839fdaed0d1098797d8cf481]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/937b7004839fdaed0d1098797d8cf481 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:937b7004839fdaed0d1098797d8cf481]}]} 2023-10-21T04:38:25.136+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:38:25.136+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:true Messages:[]} 2023-10-21T04:38:25.137+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:38:25.138+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:38:25.152+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{18194302e3f10df59bd076e1c4142d0b 18194302e3f10df59bd076e1c4142d0b plasma map[] [{78 bucket_%2 _default c1 []} {17 bucket_%2 _system _query []} {107 bucket_1 _default c1 []} {37 bucket_1 _system _query []}] [] 0 0 0 false} {8806f5706747af4e635b8da5fcd8aeee 8806f5706747af4e635b8da5fcd8aeee plasma map[] [{78 bucket_%2 _default c1 []} {21 bucket_%2 _system _query []} {101 bucket_1 _default c1 []} {26 bucket_1 _system _query []}] [] 0 0 0 false}] map[]} 2023-10-21T04:38:25.199+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52286 2023-10-21T04:38:25.202+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:38:25.202+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:38:25.202+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52286 2023-10-21T04:38:25.223+05:30 [Info] Planner::processDeleteToken: processing 52 tokens for node 127.0.0.1:9003 2023-10-21T04:38:25.223+05:30 [Info] Planner::processDeleteToken: processing 52 tokens for node 127.0.0.1:9004 2023-10-21T04:38:25.223+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:52286 2023-10-21T04:38:25.227+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:38:25.227+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:38:25.233+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:38:25.233+05:30 [Info] Planner::moveTenantsFromDeletedNodes No non-empty deleted nodes found. 2023-10-21T04:38:25.233+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9003 SG Group 2 Memory 0 Units 0 2023-10-21T04:38:25.233+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9004 SG Group 1 Memory 0 Units 0 2023-10-21T04:38:25.233+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [] 2023-10-21T04:38:25.233+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:38:25.233+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:38:25.233+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:38:25.233+05:30 [Info] ---------------------------------------- 2023-10-21T04:38:25.233+05:30 [Info] **************************************** 2023-10-21T04:38:25.233+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:38:25.233+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:38:25.233+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:38:25.233+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:true Messages:[]} 2023-10-21T04:38:25.233+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/937b7004839fdaed0d1098797d8cf481 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:937b7004839fdaed0d1098797d8cf481]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/937b7004839fdaed0d1098797d8cf481 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:937b7004839fdaed0d1098797d8cf481]}]} 2023-10-21T04:38:25.234+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress done received. progress collector stopped 2023-10-21T04:38:25.233+05:30 [Info] ShardRebalancer::updateProgress Done Received 2023-10-21T04:38:25.234+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:38:25.234+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:38:25.235+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {18194302e3f10df59bd076e1c4142d0b 937b7004839fdaed0d1098797d8cf481 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:38:25.239+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:38:25.239+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:38:25.241+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:38:25.241+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:38:25.241+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 2.091µs 2023-10-21T04:38:25.241+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:38:25.241+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:38:25.241+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:38:25.243+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:38:25.243+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:38:25.243+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:38:25.244+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:38:25.244+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 28.961µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:38:25.245+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:38:25.245+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:38:25.245+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster true, forceUnbalanced false, err: 2023-10-21T04:38:25.245+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:38:25.246+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:true Messages:[]} 2023-10-21T04:38:25.246+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:38:25.255+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:38:25.255+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:38:25.259+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:38:25.259+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[]} 2023-10-21T04:38:25.260+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:38:25.260+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[18194302e3f10df59bd076e1c4142d0b] IsBalanced:true Messages:[]} 2023-10-21T04:38:25.262+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:38:25.262+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:38:25.286+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:38:25.286+05:30 [Info] MetadataProvider: Updating cluster version to 18446744073709551615 2023-10-21T04:38:25.286+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:38:25.286+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:38:25.367+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:38:25.388+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:25.388+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:38:25.388+05:30 [Info] UnwatchMetadata(): indexer 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:38:25.388+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:38:25.388+05:30 [Info] UnwatchMetadata(): finish for indexer 18194302e3f10df59bd076e1c4142d0b 2023-10-21T04:38:25.388+05:30 [Info] UnwatchMetadata(): indexer 8806f5706747af4e635b8da5fcd8aeee 2023-10-21T04:38:25.388+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:38:25.388+05:30 [Info] UnwatchMetadata(): finish for indexer 8806f5706747af4e635b8da5fcd8aeee 2023-10-21T04:38:25.389+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:39192->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:38:25.389+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:35568->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:38:25.389+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39192. Reason = EOF. Cleaning Pipe. 2023-10-21T04:38:25.389+05:30 [Info] messageListener.start(): message channel closed. Remove peer 7d:ea:7c:b6:49:52:3:ce as follower. 2023-10-21T04:38:25.393+05:30 [Info] switched currmeta from 84 -> 86 force true 2023-10-21T04:38:25.393+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:38:25.393+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] ... stopped 2023-10-21T04:38:25.393+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:38:25.393+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:50776 closed 2023-10-21T04:38:25.393+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] ... stopped 2023-10-21T04:38:25.402+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:25.406+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:25.406+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:38:25.412+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39064. Reason = EOF. Cleaning Pipe. 2023-10-21T04:38:25.412+05:30 [Info] messageListener.start(): message channel closed. Remove peer 1e:97:5:8a:37:50:4c:8 as follower. 2023-10-21T04:38:25.431+05:30 [Info] switched currmeta from 86 -> 86 force true 2023-10-21T04:38:25.442+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39480. Reason = EOF. Cleaning Pipe. 2023-10-21T04:38:25.442+05:30 [Info] messageListener.start(): message channel closed. Remove peer 28:17:e4:a8:a6:18:3a:74 as follower. 2023-10-21T04:38:25.444+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39478. Reason = EOF. Cleaning Pipe. 2023-10-21T04:38:25.444+05:30 [Info] messageListener.start(): message channel closed. Remove peer c6:5a:9d:0:dd:d4:1f:d4 as follower. 2023-10-21T04:38:25.444+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:39476. Reason = EOF. Cleaning Pipe. 2023-10-21T04:38:25.444+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2b:8c:e7:c7:10:2c:1b:19 as follower. 2023-10-21T04:38:25.447+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:51242 closed 2023-10-21T04:38:25.552+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:38:25.603+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 1 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:38:25.603+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:25.603+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:38:25.609+05:30 [Info] switched currmeta from 86 -> 86 force true 2023-10-21T04:38:25.617+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:25.654+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:25.654+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:38:25.658+05:30 [Info] switched currmeta from 86 -> 86 force true 2023-10-21T04:38:25.808+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:38:25.827+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:25.827+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:38:25.828+05:30 [Info] switched currmeta from 86 -> 86 force true 2023-10-21T04:38:25.830+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:38:25.844+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:38:25.844+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:38:25.849+05:30 [Info] switched currmeta from 86 -> 86 force true 2023-10-21T04:38:25.851+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 218] updated cluster rebalancing status to false 2023-10-21T04:40:30.751+05:30 [Info] Indexer started with command line: [/opt/build/install/bin/indexer -adminPort=9118 -scanPort=9119 -httpPort=9120 -streamInitPort=9121 -streamCatchupPort=9122 -streamMaintPort=9123 --httpsPort=19120 --certFile=/opt/build/ns_server/data/n_3/config/certs/chain.pem --keyFile=/opt/build/ns_server/data/n_3/config/certs/pkey.pem --caFile=/opt/build/ns_server/data/n_3/config/certs/ca.pem --clientCertFile=/opt/build/ns_server/data/n_3/config/certs/client_chain.pem --clientKeyFile=/opt/build/ns_server/data/n_3/config/certs/client_pkey.pem -ipv4=required -ipv6=optional -cluster=127.0.0.1:9003 -storageDir=/opt/build/ns_server/data/n_3/data/@2i -diagDir=/opt/build/ns_server/data/n_3/crash -logDir=/opt/build/ns_server/logs/n_3 -nodeUUID=4325b30e7f9620a998388078247335a8 -isEnterprise=true --deploymentModel=serverless] 2023-10-21T04:40:30.752+05:30 [Info] Indexer DeploymentModel is set to: serverless 2023-10-21T04:40:30.755+05:30 [Info] Setting ipv6=false 2023-10-21T04:40:30.839+05:30 [Info] Indexer::NewIndexer Status Warmup 2023-10-21T04:40:30.886+05:30 [Info] IndexerSettingsManager: Setting numSliceWriters to 4 cgroup.max_cpu_percent: 400 runtime.NumCPU: 400 2023-10-21T04:40:30.891+05:30 [Info] Setting buffer block size to 16384 bytes 2023-10-21T04:40:30.891+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:40:30.891+05:30 [Info] Setting log level to Info 2023-10-21T04:40:30.891+05:30 [Warn] SetCICLMgrTimeDiffToForceFetch: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:40:30.891+05:30 [Warn] SetCICLMgrSleepTimeOnNotifierRestart: Singleton Manager in ClusterInfoCacheLite is not set 2023-10-21T04:40:30.892+05:30 [Info] CpuThrottle::SetCpuTarget: New cpuTarget: 0.95 2023-10-21T04:40:30.892+05:30 [Info] audit::InitAuditService using address http://127.0.0.1:9003 2023-10-21T04:40:30.908+05:30 [Info] SettingsMgr::metaKVCallback Dropped request /indexing/settings/config {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100}. Any setting change will get applied once Indexer is ready. 2023/10/21 04:40:31 audit: created new audit service 2023-10-21T04:40:31.155+05:30 [Info] InitSystemEventLogger: Started SystemEventLogger 2023-10-21T04:40:31.155+05:30 [Info] Receive security change notification. code 7 2023-10-21T04:40:31.156+05:30 [Info] TLS config refreshed successfully 2023-10-21T04:40:31.157+05:30 [Info] Certificate refreshed successfully with certFile /opt/build/ns_server/data/n_3/config/certs/chain.pem, keyFile /opt/build/ns_server/data/n_3/config/certs/pkey.pem, caFile /opt/build/ns_server/data/n_3/config/certs/ca.pem 2023-10-21T04:40:31.157+05:30 [Info] Encryption config refresh successfully. Encryption enabled=false 2023-10-21T04:40:31.157+05:30 [Info] security context: encryptLocalHost true Local IP's: map[127.0.0.1:true 172.31.5.112:true ::1:true fe80::9d:f9ff:fed2:4d1c:true] 2023-10-21T04:40:31.226+05:30 [Info] security context initialized 2023-10-21T04:40:31.226+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9120:19120 9499:19499 9999:9999] 2023-10-21T04:40:31.227+05:30 [Info] Indexer::NewIndexer Build Mode Set Enterprise 2023-10-21T04:40:31.227+05:30 [Info] Indexer::Cluster Storage Mode Set plasma 2023-10-21T04:40:31.227+05:30 [Info] memcachedTimeout set to 120 2023-10-21T04:40:31.227+05:30 [Info] Indexer::NewIndexer done initializing from config 2023-10-21T04:40:31.239+05:30 [Info] SetMajorVersionCICL: Setting cluster version to 7 2023-10-21T04:40:31.240+05:30 [Info] newClusterInfoCacheLiteManager: started New clusterInfoCacheManager 2023-10-21T04:40:31.240+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for indexer 2023-10-21T04:40:31.244+05:30 [Warn] PollForDeletedBucketsV2: Falling back to pollForDeletedBuckets() as use_cinfo_lite is false or not present 2023-10-21T04:40:31.251+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:40:31.251+05:30 [Info] MutationMgr::checkPortAvailability for dataport servers ... 2023-10-21T04:40:31.252+05:30 [Info] serviceChangeNotifier: Creating new notifier instance for http://%40index@127.0.0.1:9003-default 2023-10-21T04:40:31.254+05:30 [Info] MutationMgr::checkPortAvailability all dataport server ports are available. 2023-10-21T04:40:31.257+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:40:31.259+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:40:31.270+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_%2 2023-10-21T04:40:31.270+05:30 [Info] serviceChangeNotifier: Starting to monitor the bucket streaming endpoint for bucket: bucket_1 2023-10-21T04:40:31.270+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:40:31.294+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:40:31.294+05:30 [Info] OperationsMonitor(kvSender) starting monitor ... 2023-10-21T04:40:31.294+05:30 [Info] [Queryport ":9119"] started ... 2023-10-21T04:40:31.299+05:30 [Info] Unable to fetch storage mode from metakv during bootrap. Use storage mode setting for bootstrap 2023-10-21T04:40:31.299+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:40:31.302+05:30 [Info] Indexer::NewIndexer bootstrap storage mode plasma 2023-10-21T04:40:31.302+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for IndexMgr 2023-10-21T04:40:31.302+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for RequestHandler 2023-10-21T04:40:31.349+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for LifecycleMgr 2023-10-21T04:40:31.352+05:30 [Info] Starting metadadta repo: quota 1048576 sleep duration 900 threshold 30 min file size 0 2023-10-21T04:40:31.354+05:30 [INFO][FDB] Forestdb blockcache size 1048576 initialized in 222 us 2023-10-21T04:40:31.355+05:30 [INFO][FDB] Forestdb opened database file /opt/build/ns_server/data/n_3/data/@2i/MetadataStore.0 2023-10-21T04:40:31.378+05:30 [Info] Repo.upgradeAndOpenDBFile(/opt/build/ns_server/data/n_3/data/@2i/MetadataStore): Opened with COMPACT_AUTO mode 2023-10-21T04:40:31.387+05:30 [Info] EmbeddedServer.runOnce() : Start Running Server 2023-10-21T04:40:31.395+05:30 [Info] indexer:: Staring http server : :9120 2023-10-21T04:40:31.395+05:30 [Info] HTTPS server created for :19120 2023-10-21T04:40:31.396+05:30 [Info] indexer:: SSL server started: :19120 2023-10-21T04:40:31.396+05:30 [Info] Indexer::indexer version 9 with priority 7.6.0 2023-10-21T04:40:31.396+05:30 [Info] ClustMgr:handleGetLocalValue Key IndexerId 2023-10-21T04:40:31.410+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerId Value 4325b30e7f9620a998388078247335a8 2023-10-21T04:40:31.417+05:30 [Info] Indexer Id 4325b30e7f9620a998388078247335a8 2023-10-21T04:40:31.417+05:30 INFO REGU.logging.SetCustomLogger() at logging.go:96 [id 1] configured custom logger to () 2023-10-21T04:40:31.418+05:30 INFO REGU.config.startMetakvConfigMonitor() at metakv.go:22 [id 1] starting metakv config monitor 2023-10-21T04:40:31.420+05:30 INFO REGU.utils.(*ClusterClient).initEventLogger() at eventlog.go:44 [id 1] initEventLogger: Started SystemEventLogger 2023-10-21T04:40:31.421+05:30 INFO REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:146 [id 1] configuring node capacity-based throttling for index 2023-10-21T04:40:31.423+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:748 [id 170] updating cluster rev epoch 0 -> 1 2023-10-21T04:40:31.423+05:30 INFO REGU.utils.(*server).watchNodeServices.func2() at nsserver.go:831 [id 170] starting new pool watcher 127.0.0.1:9003 2023-10-21T04:40:31.429+05:30 DEBU REGU.utils.(*ClusterClient).dispatchBucketNamesUpdate() at nsserver.go:862 [id 215] bucketNames has been updated (was map[], now map[bucket_%2:d5fd9a12d20c726b6a3f9002eb18d52b bucket_1:c25fee48adbe357b2d711be58a1deb67]) 2023-10-21T04:40:31.429+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 216] (re)starting bucket observer for bucket_%2 on 127.0.0.1:9003 2023-10-21T04:40:31.430+05:30 DEBU REGU.utils.(*ClusterClient).RunObserveBucketUntilCancelled() at nsserver.go:350 [id 217] (re)starting bucket observer for bucket_1 on 127.0.0.1:9003 2023-10-21T04:40:31.430+05:30 INFO REGU.throttle.NodeCapThrottleInit() at nodecap.go:90 [id 1] [index] node capacity-throttling initialized 2023-10-21T04:40:31.436+05:30 DEBU REGU.impl.(*regulatorImpl).ensureThrottling() at regulator_impl.go:172 [id 1] successfully updated throttle impls from [] to [[index]] 2023-10-21T04:40:31.436+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:108 [id 1] initialized index storage reporting service 2023-10-21T04:40:31.459+05:30 DEBU REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:323 [id 245] disregarding & deleting stale (age: 2m29.445910873s) storage report from node [index 18194302e3f10df59bd076e1c4142d0b] (delete err: ) 2023-10-21T04:40:31.463+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:40:31.463+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 266] updating rebalancing status to true 2023-10-21T04:40:31.468+05:30 INFO REGU.impl.NewStorageLimitingService() at storagelimiting.go:155 [id 1] initialized storage monitoring service 2023-10-21T04:40:31.468+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/metering_data: no such file or directory 2023-10-21T04:40:31.468+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceRunning 2023-10-21T04:40:31.468+05:30 [Info] ClustMgr:handleGetLocalValue Key RebalanceToken 2023-10-21T04:40:31.468+05:30 [Info] Indexer::recoverRebalanceState RebalanceRunning false RebalanceToken 2023-10-21T04:40:31.468+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseResumeRunning 2023-10-21T04:40:31.469+05:30 [Info] ClustMgr:handleGetLocalValuesWithPrefix Key PauseToken 2023-10-21T04:40:31.469+05:30 [Info] ClustMgr:handleGetGlobalTopology indexInstMap : 2023-10-21T04:40:31.469+05:30 [Info] MeteringThrottlingMgr:: started 2023-10-21T04:40:31.500+05:30 DEBU REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:323 [id 245] disregarding & deleting stale (age: 2m29.458681965s) storage report from node [index 8806f5706747af4e635b8da5fcd8aeee] (delete err: ) 2023-10-21T04:40:31.500+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 245] /regulator/report/s/index/18194302e3f10df59bd076e1c4142d0b has 0-length value 2023-10-21T04:40:31.500+05:30 WARN REGU.impl.(*storageLimitingImpl).processStorageReport() at storagelimiting.go:304 [id 245] /regulator/report/s/index/8806f5706747af4e635b8da5fcd8aeee has 0-length value 2023-10-21T04:40:31.529+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:40:31.529+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:40:31.531+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:40:31.609+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:40:31.609+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:40:31.626+05:30 [Info] Indexer::initFromPersistedState Recovered IndexInstMap , elapsed: 156.439178ms 2023-10-21T04:40:31.626+05:30 [Info] DDLServiceMgr: intialized. Local nodeUUID 4325b30e7f9620a998388078247335a8 2023-10-21T04:40:31.626+05:30 [Info] NewClusterInfoCacheLiteClient started new cicl client for schedIndexCreator 2023-10-21T04:40:31.626+05:30 [Info] schedIndexCreator: intialized. 2023-10-21T04:40:31.626+05:30 [Info] RebalanceServiceManager::NewRebalanceServiceManager false 2023-10-21T04:40:31.626+05:30 [Info] DDLServiceMgr::runTokenCleaner: Starting with period 5m0s 2023-10-21T04:40:31.626+05:30 [Info] RebalanceServiceManager::initService Init 2023-10-21T04:40:31.626+05:30 [Error] RebalanceServiceManager::getCachedIndexerNodeUUIDs: 127.0.0.1:9120/getCachedIndexerNodeUUIDs returned err: Unable to find given hostport in cbauth database: `127.0.0.1:9120' 2023-10-21T04:40:31.626+05:30 [Info] MasterServiceManager::registerWithServer: *indexer.MasterServiceManager implements service.AutofailoverManager; *indexer.MasterServiceManager implements service.Manager; *indexer.MasterServiceManager implements service.ServerlessManager 2023-10-21T04:40:31.627+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 2023-10-21T04:40:31.629+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8. Err projector.topicMissing 2023-10-21T04:40:31.629+05:30 [Error] KVSender::closeMutationStream MAINT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:40:31.629+05:30 [Info] KVSender::closeMutationStream MAINT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:40:31.629+05:30 [Info] KVSender::sendShutdownTopic Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 2023-10-21T04:40:31.631+05:30 [Error] KVSender::sendShutdownTopic Unexpected Error During Shutdown Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8. Err projector.topicMissing 2023-10-21T04:40:31.631+05:30 [Error] KVSender::closeMutationStream INIT_STREAM Error Received projector.topicMissing from 172.31.5.112:10000 2023-10-21T04:40:31.631+05:30 [Info] KVSender::closeMutationStream INIT_STREAM Treating projector.topicMissing As Success 2023-10-21T04:40:31.631+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:40:31.631+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:40:31.661+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:40:31.661+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 0] Tasks:[]} 2023-10-21T04:40:31.664+05:30 [Info] CpuThrottle::runThrottling: Starting. cpuTarget: 0.95, throttleDelayMs: 0 2023-10-21T04:40:31.676+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:40:31.676+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 0] Nodes:[4325b30e7f9620a998388078247335a8] IsBalanced:true Messages:[]} 2023-10-21T04:40:31.692+05:30 [Info] indexer.upgradeStorage: check index for storage upgrade. disable true overrride not_set 2023-10-21T04:40:31.692+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:40:31.702+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 0] 2023-10-21T04:40:31.718+05:30 [Info] Indexer::local storage mode plasma 2023-10-21T04:40:31.718+05:30 [Warn] Encountered error while reading persisted stats. Skipping read. Error: open /opt/build/ns_server/data/n_3/data/@2i/indexstats/stats: no such file or directory 2023-10-21T04:40:31.718+05:30 [Info] MeteringThrottlingMgr::handleIndexerReady Received notification after indexer is ready 2023-10-21T04:40:31.718+05:30 [Info] ClustMgr:handleIndexerReady mType: CLUST_MGR_INDEXER_READY indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:40:31.718+05:30 [Info] SettingsMgr::handleIndexerReady apply pending setting changes 2023-10-21T04:40:31.720+05:30 [Info] PauseServiceManager::recoverPauseResume: Done cleanup 2023-10-21T04:40:31.720+05:30 [Info] RebalanceServiceManager::listenMoveIndex &{4325b30e7f9620a998388078247335a8 7060000 } 2023-10-21T04:40:31.720+05:30 [Info] ClustMgr:handleSetLocalValue Key IndexerNodeUUID Value 4325b30e7f9620a998388078247335a8 2023-10-21T04:40:31.721+05:30 DEBU REGU.impl.init.0.func1() at aggrecorder.go:57 [id 15] will report aggregate recorder stats every 5m0s 2023-10-21T04:40:31.724+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 0] 2023-10-21T04:40:31.726+05:30 [Info] Indexer::NewIndexer Status Active 2023-10-21T04:40:31.727+05:30 [Info] 127.0.0.1:9003 starting RESTful services 2023-10-21T04:40:31.727+05:30 [Info] Indexer::monitorMemUsage started... 2023-10-21T04:40:31.728+05:30 [Info] progress stats collection done. 2023-10-21T04:40:31.728+05:30 [Info] send progress stats to clients 2023-10-21T04:40:31.728+05:30 [Info] memstats {"Alloc":0, "TotalAlloc":0, "Sys":0, "Lookups":0, "Mallocs":0,"Frees":0, "HeapAlloc":0, "HeapSys":0, "HeapIdle":0, "HeapInuse":0,"HeapReleased":0, "HeapObjects":0,"MSpanInuse":0, "MSpanSys": 0, "StackInuse": 0,"GCSys":0, "LastGC":0,"PauseTotalNs":0, "PauseNs":[], "NumGC":0} 2023-10-21T04:40:31.754+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config, value: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:40:31.756+05:30 [Info] MapSettings: config before mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:40:31.756+05:30 [Info] MapSettings: config after mapping: {"indexer.api.enableTestServer":true,"indexer.client_stats_refresh_interval":500,"indexer.plasma.backIndex.enableInMemoryCompression":true,"indexer.plasma.backIndex.enablePageBloomFilter":true,"indexer.plasma.mainIndex.enableInMemoryCompression":true,"indexer.plasma.shardCopy.dbg":true,"indexer.rebalance.serverless.perNodeTransferBatchSize":1,"indexer.rebalance.serverless.transferBatchSize":2,"indexer.settings.allow_large_keys":true,"indexer.settings.bufferPoolBlockSize":16384,"indexer.settings.build.batch_size":5,"indexer.settings.compaction.abort_exceed_interval":false,"indexer.settings.compaction.check_period":30,"indexer.settings.compaction.compaction_mode":"circular","indexer.settings.compaction.days_of_week":"Sunday,Monday,Tuesday,Wednesday,Thursday,Friday,Saturday","indexer.settings.compaction.interval":"00:00,00:00","indexer.settings.compaction.min_frag":30,"indexer.settings.compaction.min_size":524288000,"indexer.settings.compaction.plasma.manual":false,"indexer.settings.compaction.plasma.optional.decrement":5,"indexer.settings.compaction.plasma.optional.min_frag":20,"indexer.settings.compaction.plasma.optional.quota":25,"indexer.settings.corrupt_index_num_backups":1,"indexer.settings.cpuProfDir":"","indexer.settings.cpuProfile":false,"indexer.settings.eTagPeriod":240,"indexer.settings.enable_corrupt_index_backup":false,"indexer.settings.enable_page_bloom_filter":true,"indexer.settings.enable_shard_affinity":false,"indexer.settings.fast_flush_mode":true,"indexer.settings.gc_percent":100,"indexer.settings.inmemory_snapshot.fdb.interval":200,"indexer.settings.inmemory_snapshot.interval":200,"indexer.settings.inmemory_snapshot.moi.interval":10,"indexer.settings.largeSnapshotThreshold":200,"indexer.settings.log_level":"info","indexer.settings.maxVbQueueLength":0,"indexer.settings.max_array_seckey_size":10240,"indexer.settings.max_cpu_percent":0,"indexer.settings.max_seckey_size":4608,"indexer.settings.max_writer_lock_prob":20,"indexer.settings.memProfDir":"","indexer.settings.memProfile":false,"indexer.settings.memory_quota":1572864000,"indexer.settings.minVbQueueLength":30,"indexer.settings.moi.debug":false,"indexer.settings.moi.persistence_threads":2,"indexer.settings.moi.recovery.max_rollbacks":2,"indexer.settings.moi.recovery_threads":4,"indexer.settings.num_replica":1,"indexer.settings.percentage_memory_quota":0,"indexer.settings.persisted_snapshot.fdb.interval":5000,"indexer.settings.persisted_snapshot.interval":5000,"indexer.settings.persisted_snapshot.moi.interval":60000,"indexer.settings.persisted_snapshot_init_build.fdb.interval":5000,"indexer.settings.persisted_snapshot_init_build.interval":5000,"indexer.settings.persisted_snapshot_init_build.moi.interval":60000,"indexer.settings.plasma.recovery.max_rollbacks":2,"indexer.settings.rebalance.blob_storage_bucket":"/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests","indexer.settings.rebalance.blob_storage_prefix":"shard_rebalance_storage_dir","indexer.settings.rebalance.blob_storage_region":"","indexer.settings.rebalance.blob_storage_scheme":"","indexer.settings.rebalance.redistribute_indexes":false,"indexer.settings.recovery.max_rollbacks":2,"indexer.settings.scan_getseqnos_retries":30,"indexer.settings.scan_timeout":120000,"indexer.settings.send_buffer_size":1024,"indexer.settings.serverless.indexLimit":201,"indexer.settings.sliceBufSize":800,"indexer.settings.smallSnapshotThreshold":30,"indexer.settings.snapshotListeners":2,"indexer.settings.snapshotRequestWorkers":2,"indexer.settings.statsLogDumpInterval":60,"indexer.settings.storage_mode":"plasma","indexer.settings.storage_mode.disable_upgrade":true,"indexer.settings.thresholds.mem_high":70,"indexer.settings.thresholds.mem_low":50,"indexer.settings.thresholds.units_high":60,"indexer.settings.thresholds.units_low":40,"indexer.settings.units_quota":10000,"indexer.settings.wal_size":4096,"indexer.shardRebalance.execTestAction":true,"indexer.stats_cache_timeout":500,"projector.settings.log_level":"info","queryport.client.log_level":"warn","queryport.client.settings.backfillLimit":0,"queryport.client.settings.closeActiveConnections":true,"queryport.client.settings.minPoolSizeWM":1000,"queryport.client.settings.poolOverflow":30,"queryport.client.settings.poolSize":5000,"queryport.client.settings.relConnBatchSize":100} 2023-10-21T04:40:31.758+05:30 [Info] settingsManager::setGlobalSettings: Indexer # CPU cores: 4, memory quota: 1572864000 bytes (1536000.000 KB, 1500.000 MB, 1.465 GB) 2023-10-21T04:40:31.758+05:30 [Info] Setting log level to Info 2023-10-21T04:40:31.758+05:30 [Info] SettingsManager::applySettings Updating 'plasma.minNumShard' setting to: 2 2023-10-21T04:40:31.762+05:30 [Warn] Indexer::promoteStorageModeIfNecessary(): storage mode upgrade is disabled. 2023-10-21T04:40:31.774+05:30 [Info] Indexer::updateStorageMode Try setting storage mode to plasma 2023-10-21T04:40:31.774+05:30 [Info] Indexer: Setting GC percent to 100 2023-10-21T04:40:31.775+05:30 [Info] 127.0.0.1:9003 starting internal RESTful services 2023-10-21T04:40:31.787+05:30 [Info] settingsManager::applySettings: New settings received on path: /indexing/settings/config/features/ShardAffinity, value: 2023-10-21T04:40:31.787+05:30 [Warn] settingsManager::applySettings: Empty value found for /indexing/settings/config/features/ShardAffinity. Will skip processing... 2023-10-21T04:40:32.041+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:40:32.167+05:30 [Info] security context initialized 2023-10-21T04:40:32.167+05:30 [Info] security port mapping updated : map[10000:10000 11210:11207 12000:11998 8093:18093 9000:19000 9001:19001 9002:19002 9003:19003 9004:19004 9100:9100 9101:9101 9102:19102 9103:9103 9104:9104 9105:9105 9106:9106 9107:9107 9108:19108 9109:9109 9110:9110 9111:9111 9112:9112 9113:9113 9114:19114 9115:9115 9116:9116 9117:9117 9499:19499 9999:9999] 2023-10-21T04:40:32.175+05:30 [Info] Terminating WatchClusterVersionChanges: Cluster version is >= 9 2023-10-21T04:40:32.234+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 4 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:40:32.234+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:32.234+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112] 2023-10-21T04:40:32.234+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:40:32.279+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:40:32.330+05:30 [Info] internalVersionMonitor:monitor starting. Term versions (9:7.6.0) 2023-10-21T04:40:32.330+05:30 [Info] internalVersionMonitor:ticker starting ... 2023-10-21T04:40:32.334+05:30 [Info] internalVersionMonitor:notifier starting ... 2023-10-21T04:40:32.338+05:30 [Info] internalVersionMonitor:monitor terminate. Cluster version reached 9 2023-10-21T04:40:32.338+05:30 [Info] internalVersionMonitor:ticker stopping ... 2023-10-21T04:40:32.412+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {4ba7e1420e46bd0aae5b8713b46bb0d0 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:40:32.433+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:40:32.487+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:40:32.487+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:40:32.501+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 244] aggregate storage info: map[bucket_%2:map[kv:2300096 index:1094793] bucket_1:map[kv:2369728 index:1368214]] 2023-10-21T04:40:32.516+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:47950 2023-10-21T04:40:32.533+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:47964 2023-10-21T04:40:32.564+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:47968 2023-10-21T04:40:32.572+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:47970 2023-10-21T04:40:32.628+05:30 [Info] requestHandlerContext::getCachedIndexerNodeUUIDs: Returning 2 NodeUUIDs: [d7ff2bfd429b21d9c24d822bc13e3acc 9b27839d59b06374125d80a2b12a21b6] 2023-10-21T04:40:32.628+05:30 [Info] RebalanceServiceManager::getCachedIndexerNodeUUIDs: Returning 3 NodeUUIDs: [d7ff2bfd429b21d9c24d822bc13e3acc 9b27839d59b06374125d80a2b12a21b6 4325b30e7f9620a998388078247335a8] 2023-10-21T04:40:32.628+05:30 [Info] RebalanceServiceManager::updateNodeList: Initialized with 3 NodeUUIDs: [d7ff2bfd429b21d9c24d822bc13e3acc 9b27839d59b06374125d80a2b12a21b6 4325b30e7f9620a998388078247335a8] 2023-10-21T04:40:32.628+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 1] Nodes:[d7ff2bfd429b21d9c24d822bc13e3acc 9b27839d59b06374125d80a2b12a21b6 4325b30e7f9620a998388078247335a8] IsBalanced:true Messages:[]} 2023-10-21T04:40:32.628+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 0] call. taskList: &{Rev:[0 0 0 0 0 0 0 1] Tasks:[]} 2023-10-21T04:40:32.645+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 1] 2023-10-21T04:40:32.654+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:47986 2023-10-21T04:40:32.710+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 1] 2023-10-21T04:40:32.726+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:40:32.753+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 217.990039ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.759+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 24.832µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.759+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:40:32.764+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 244.221136ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.784+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 209.09628ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.785+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 13.307µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.785+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:40:32.785+05:30 [Info] initialized currmeta 18 force true 2023-10-21T04:40:32.790+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 10.289µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.792+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 215.797406ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.793+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 11.42µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.799+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:40:32.799+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Started active connections updates consumer ... 2023-10-21T04:40:32.799+05:30 [Info] [GsiScanClient:"127.0.0.1:9107"] started ... 2023-10-21T04:40:32.800+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] open new connection ... 2023-10-21T04:40:32.842+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 172.093864ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.843+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 12.869µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:32.843+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59596" doAuth() ... 2023-10-21T04:40:32.844+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59596" doReceive() ... 2023-10-21T04:40:32.851+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:40:32.851+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Started active connections updates consumer ... 2023-10-21T04:40:32.851+05:30 [Info] [GsiScanClient:"127.0.0.1:9113"] started ... 2023-10-21T04:40:32.852+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] open new connection ... 2023-10-21T04:40:32.853+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:48022 2023-10-21T04:40:32.858+05:30 [Info] CompactionManager: Refreshing settings 2023-10-21T04:40:32.858+05:30 [Info] StreamState::UpdateConfig 30 2023-10-21T04:40:32.858+05:30 [Info] Timekeeper::setMaxTsQueueLen 1000 2023-10-21T04:40:32.858+05:30 [Info] MutationMgr::MaxQueueMemoryQuota 15728640 2023-10-21T04:40:32.858+05:30 [Info] mutationMgr::setEnableAuth: enableAuth set to 1 2023-10-21T04:40:32.858+05:30 [Info] MutationManager::setMinVbQueueLength 30 2023-10-21T04:40:32.858+05:30 [Info] ClustMgr:handleConfigUpdate 2023-10-21T04:40:32.863+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 5.796µs 2023-10-21T04:40:32.863+05:30 [Info] ShardTransferManager::ConfigUpdate - Updating maxDiskBw to 209715200, prev value: 0 2023-10-21T04:40:32.863+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:40:32.863+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:40:32.863+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:40:32.872+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:40:32.873+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:40:32.873+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 9.858319ms 2023-10-21T04:40:32.873+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:40:32.893+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:40:32.894+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 2] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:40:32.895+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CONFIG_UPDATE elapsed 34.662229ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 2 error 2023-10-21T04:40:32.895+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 111.867µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 1 error 2023-10-21T04:40:32.893+05:30 [Info] Indexer::ReadMemstats Time Taken 19.314119ms 2023-10-21T04:40:32.896+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 1] call. taskList: &{Rev:[0 0 0 0 0 0 0 2] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/4ba7e1420e46bd0aae5b8713b46bb0d0 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:4ba7e1420e46bd0aae5b8713b46bb0d0]}]} 2023-10-21T04:40:32.896+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9316375249999768625:[2 3 4 5 6 7 8 1]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:40:32.896+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9316375249999768625:[2 3 4 5 6 7 8 1] 16954811547177290607:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:40:32.896+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9316375249999768625:[2 3 4 5 6 7 8 1] 16954811547177290607:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 {Index inst : [partitions]}: map[12606056968936758026:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:40:32.896+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12606056968936758026:[3 4 5 6 7 8 1 2] 16884277835173449069:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 {Index inst : [partitions]}: map[9316375249999768625:[2 3 4 5 6 7 8 1] 16954811547177290607:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:40:32.896+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9316375249999768625:[2 3 4 5 6 7 8 1] 16954811547177290607:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 {Index inst : [partitions]}: map[12606056968936758026:[3 4 5 6 7 8 1 2] 16884277835173449069:[0] 17465522551378543576:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:40:32.896+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12606056968936758026:[3 4 5 6 7 8 1 2] 16884277835173449069:[0] 17465522551378543576:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 {Index inst : [partitions]}: map[9316375249999768625:[2 3 4 5 6 7 8 1] 16954811547177290607:[0] 18137927630988528516:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:40:32.896+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:40:32.902+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 2] 2023-10-21T04:40:32.911+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 2] 2023-10-21T04:40:32.916+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59624" doAuth() ... 2023-10-21T04:40:32.919+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59624" doReceive() ... 2023-10-21T04:40:32.896+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59616" doAuth() ... 2023-10-21T04:40:32.926+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59616" doReceive() ... 2023-10-21T04:40:32.958+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59630" doAuth() ... 2023-10-21T04:40:32.958+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:32.958+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:40:32.958+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:40:32.959+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59630" doReceive() ... 2023-10-21T04:40:32.969+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59636" doAuth() ... 2023-10-21T04:40:32.969+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59636" doReceive() ... 2023-10-21T04:40:32.994+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:48062 2023-10-21T04:40:32.994+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:40:33.007+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 141.245539ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:33.016+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 13.303µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:33.030+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 32.869256ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:33.030+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:40:33.030+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:40:33.031+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CLIENT_STATS elapsed 10.447µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:33.048+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:40:33.074+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:40:33.077+05:30 [Info] switched currmeta from 18 -> 20 force true 2023-10-21T04:40:33.077+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15867764812233117191:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:40:33.077+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15867764812233117191:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 {Index inst : [partitions]}: map[4397940545762329850:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:40:33.077+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[1 2 3 4 5 6 7 8] 12527610516055683489:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 {Index inst : [partitions]}: map[15867764812233117191:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:40:33.077+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15867764812233117191:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 {Index inst : [partitions]}: map[4397940545762329850:[1 2 3 4 5 6 7 8] 10158339249566909515:[0] 12527610516055683489:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:40:33.078+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[544693394183450628:[0] 15867764812233117191:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 {Index inst : [partitions]}: map[4397940545762329850:[1 2 3 4 5 6 7 8] 10158339249566909515:[0] 12527610516055683489:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:40:33.078+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[544693394183450628:[0] 15867764812233117191:[0] 16759912838651900043:[4 5 6 7 8 1 2 3]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 {Index inst : [partitions]}: map[4397940545762329850:[1 2 3 4 5 6 7 8] 10158339249566909515:[0] 12527610516055683489:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:40:33.089+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:40:33.089+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Started active connections updates consumer ... 2023-10-21T04:40:33.089+05:30 [Info] [GsiScanClient:"127.0.0.1:9119"] started ... 2023-10-21T04:40:33.090+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59650" doAuth() ... 2023-10-21T04:40:33.090+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59650" doReceive() ... 2023-10-21T04:40:33.091+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] open new connection ... 2023-10-21T04:40:33.092+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59652" doAuth() ... 2023-10-21T04:40:33.093+05:30 [Info] [Queryport ":9119"] connection "127.0.0.1:59652" doReceive() ... 2023-10-21T04:40:33.106+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] started poolsize 5000 overflow 30 low WM 1000 relConn batch size 100 ... 2023-10-21T04:40:33.106+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Started active connections updates consumer ... 2023-10-21T04:40:33.106+05:30 [Info] [GsiScanClient:"127.0.0.1:9125"] started ... 2023-10-21T04:40:33.108+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] open new connection ... 2023-10-21T04:40:33.153+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {4ba7e1420e46bd0aae5b8713b46bb0d0 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:40:33.157+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:40:33.217+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"4ba7e1420e46bd0aae5b8713b46bb0d0","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:40:33.220+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{4325b30e7f9620a998388078247335a8 4ba7e1420e46bd0aae5b8713b46bb0d0 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:40:33.222+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 4ba7e1420e46bd0aae5b8713b46bb0d0 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:40:33.256+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:33.264+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:33.284+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:33.293+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:33.343+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:33.509+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:33.531+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:34.231+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:40:34.241+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:40:34.241+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:40:34.247+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:40:34.247+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{4325b30e7f9620a998388078247335a8 4ba7e1420e46bd0aae5b8713b46bb0d0 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:40:34.247+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:40:34.247+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.094658643s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:40:34.248+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{4325b30e7f9620a998388078247335a8 4ba7e1420e46bd0aae5b8713b46bb0d0 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:40:34.248+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 2] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/4ba7e1420e46bd0aae5b8713b46bb0d0 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:4ba7e1420e46bd0aae5b8713b46bb0d0]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/4ba7e1420e46bd0aae5b8713b46bb0d0 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:4ba7e1420e46bd0aae5b8713b46bb0d0]}]} 2023-10-21T04:40:34.248+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:40:34.248+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:40:34.252+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:40:34.252+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/4ba7e1420e46bd0aae5b8713b46bb0d0 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:4ba7e1420e46bd0aae5b8713b46bb0d0]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/4ba7e1420e46bd0aae5b8713b46bb0d0 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:4ba7e1420e46bd0aae5b8713b46bb0d0]}]} 2023-10-21T04:40:34.254+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:40:34.254+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:40:34.261+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 3] 2023-10-21T04:40:34.261+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 3] 2023-10-21T04:40:34.307+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{d7ff2bfd429b21d9c24d822bc13e3acc d7ff2bfd429b21d9c24d822bc13e3acc plasma map[] [{13 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_secondary 4987105685507282285 [{16884277835173449069 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 1 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{12606056968936758026 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{8303154871847919944 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{17465522551378543576 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501] []}] 1 0 0 1 false plasma 0}]}]} {13 bucket_1 _default c1 [{bucket_1 _default c1 idx_secondary 16706481854123898870 [{16954811547177290607 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 1 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned 7790129686881525591 [{9316375249999768625 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{3751526798537645569 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{18137927630988528516 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16706481854123898870 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4987105685507282285 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {9b27839d59b06374125d80a2b12a21b6 9b27839d59b06374125d80a2b12a21b6 plasma map[] [{13 bucket_1 _default c1 [{bucket_1 _default c1 idx_secondary 16706481854123898870 [{15867764812233117191 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 1 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned 7790129686881525591 [{16759912838651900043 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{4186965058349029465 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{544693394183450628 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215] []}] 1 0 0 0 false plasma 0}]}]} {13 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_secondary 4987105685507282285 [{12527610516055683489 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 1 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{4397940545762329850 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{12396348969790516872 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{10158339249566909515 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16706481854123898870 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4987105685507282285 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {4325b30e7f9620a998388078247335a8 4325b30e7f9620a998388078247335a8 plasma map[] [] [] 0 0 0 false} {ba6304988a6335d65281381d6cfc5e32 ba6304988a6335d65281381d6cfc5e32 plasma map[] [] [] 0 0 0 false}] map[]} 2023-10-21T04:40:34.408+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:40:34.415+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:40:34.415+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:40:34.415+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:40:34.415+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:40:34.417+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:40:34.455+05:30 [Info] Planner::processDeleteToken: processing 52 tokens for node 127.0.0.1:9002 2023-10-21T04:40:34.456+05:30 [Info] Planner::processDeleteToken: processing 52 tokens for node 127.0.0.1:9004 2023-10-21T04:40:34.456+05:30 [Info] Planner::processDeleteToken: processing 52 tokens for node 127.0.0.1:9001 2023-10-21T04:40:34.456+05:30 [Info] Planner::processDeleteToken: processing 52 tokens for node 127.0.0.1:9003 2023-10-21T04:40:34.458+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:40:34.463+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:40:34.463+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:40:34.463+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:40:34.463+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:40:34.507+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:40:34.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:40:34.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:40:34.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:40:34.508+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4987105685507282285:12527610516055683489:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 5277316367626246544:10158339249566909515:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 16706481854123898870:15867764812233117191:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 668551233566132751:544693394183450628:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.508+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16706481854123898870:16954811547177290607:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 668551233566132751:18137927630988528516:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.509+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Moving index 4987105685507282285:16884277835173449069:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Moving index 5277316367626246544:17465522551378543576:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:40:34.510+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 17730412 Units 0 2023-10-21T04:40:34.510+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17720848 Units 0 2023-10-21T04:40:34.510+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 288716 Units 0 Tenants 0 2023-10-21T04:40:34.510+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 278572 Units 0 Tenants 0 2023-10-21T04:40:34.510+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:40:34.510+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:40:34.510+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:40:34.510+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:40:34.510+05:30 [Info] ---------------------------------------- 2023-10-21T04:40:34.510+05:30 [Info] 2023-10-21T04:40:34.510+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:ba6304988a6335d65281381d6cfc5e32, useLiveData:false 2023-10-21T04:40:34.510+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:40:34.510+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:40:34.510+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.510+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.510+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.510+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.510+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.510+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.511+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.511+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.511+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.511+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:16706481854123898870, instId:16954811547177290607, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:157665930 (150.362M), mem:110000 (107.422K), overhead:157555930 (150.257M), min:157665930 (150.362M), data:110000 (107.422K) cpu:0.0711 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:18137927630988528516, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:157294806 (150.008M), mem:7278 (7.10742K), overhead:157287528 (150.001M), min:157294806 (150.008M), data:7278 (7.10742K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.512+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.512+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.512+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.512+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.513+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.513+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.513+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.513+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.513+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.513+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.513+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.513+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.513+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.513+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.513+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.513+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.513+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.513+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.513+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.513+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.513+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.513+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.513+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.513+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.513+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.516+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.516+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:4987105685507282285, instId:16884277835173449069, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:40:34.516+05:30 [Info] Index total memory:157401900 (150.11M), mem:100000 (97.6562K), overhead:157301900 (150.015M), min:157401900 (150.11M), data:100000 (97.6562K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.516+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.517+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.517+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:17465522551378543576, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501], shardIdsInDefn: map[] 2023-10-21T04:40:34.517+05:30 [Info] Index total memory:157287799 (150.001M), mem:1212 (1.18359K), overhead:157286587 (150M), min:157287799 (150.001M), data:1212 (1.18359K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.517+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.517+05:30 [Info] 2023-10-21T04:40:34.517+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:4325b30e7f9620a998388078247335a8, useLiveData:false 2023-10-21T04:40:34.517+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:40:34.517+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:40:34.517+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.517+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4987105685507282285, instId:12527610516055683489, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.517+05:30 [Info] Index total memory:157401900 (150.11M), mem:100000 (97.6562K), overhead:157301900 (150.015M), min:157401900 (150.11M), data:100000 (97.6562K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.517+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.517+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.517+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:10158339249566909515, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589], shardIdsInDefn: map[] 2023-10-21T04:40:34.517+05:30 [Info] Index total memory:157287799 (150.001M), mem:1212 (1.18359K), overhead:157286587 (150M), min:157287799 (150.001M), data:1212 (1.18359K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.517+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.517+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.517+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.517+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.517+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.517+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.517+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.517+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.518+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.518+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.519+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.519+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.519+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.519+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16706481854123898870, instId:15867764812233117191, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:157660830 (150.357M), mem:109000 (106.445K), overhead:157551830 (150.253M), min:157660830 (150.357M), data:109000 (106.445K) cpu:0.0943 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:544693394183450628, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:157294709 (150.008M), mem:7194 (7.02539K), overhead:157287515 (150.001M), min:157294709 (150.008M), data:7194 (7.02539K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.520+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.520+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.520+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.520+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.521+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.521+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.521+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.521+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.521+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.521+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.521+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.521+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.521+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.521+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.521+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.521+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.521+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.521+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.521+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.521+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.521+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.521+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.521+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.522+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.522+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.522+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.522+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.522+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.522+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.522+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.522+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.522+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.522+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.522+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.522+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.522+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.524+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.524+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.524+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.524+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.524+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.525+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.525+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.525+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.525+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.525+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.525+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.525+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.525+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.525+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.525+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.525+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.525+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:40:34.525+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:40:34.526+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:40:34.526+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:40:34.526+05:30 [Info] **************************************** 2023-10-21T04:40:34.526+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 288716 Units 0 Tenants 0 2023-10-21T04:40:34.526+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 278572 Units 0 Tenants 0 2023-10-21T04:40:34.527+05:30 [Info] Generating Shard Transfer Token (ShardToken95:67:90:88:a3:f5:42:a9) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken63:5e:6f:93:50:1c:47:1 Shards: [11114961608238406563 5748926106587204814] Destination: Region: InstId: 16954811547177290607 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 16954811547177290607 Defn: DefnId: 16706481854123898870 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 1438807282966389924 RealInstId: 9316375249999768625 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 9316375249999768625 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 117474327967883953 RealInstId: 3751526798537645569 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3751526798537645569 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 18137927630988528516 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 18137927630988528516 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:40:34.528+05:30 [Info] Generating Shard Transfer Token (ShardToken0:62:86:e2:55:57:3b:3) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:1a:bd:c3:3d:5d:7a:87 Shards: [18374611443167983589 6455624821495495854] Destination: Region: InstId: 12527610516055683489 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 12527610516055683489 Defn: DefnId: 4987105685507282285 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 18172419679785906247 RealInstId: 4397940545762329850 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 2935911152716188465 RealInstId: 12396348969790516872 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10158339249566909515 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:40:34.529+05:30 [Info] Generating Shard Transfer Token (ShardToken63:5e:6f:93:50:1c:47:1) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken95:67:90:88:a3:f5:42:a9 Shards: [17327901024118572215 5629943257627822009] Destination: Region: InstId: 15867764812233117191 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 15867764812233117191 Defn: DefnId: 16706481854123898870 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 5040311435261117886 RealInstId: 16759912838651900043 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 7774786113586735969 RealInstId: 4186965058349029465 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 544693394183450628 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:40:34.530+05:30 [Info] Generating Shard Transfer Token (ShardToken85:1a:bd:c3:3d:5d:7a:87) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken0:62:86:e2:55:57:3b:3 Shards: [10838601320854145501 11711860383628641160] Destination: Region: InstId: 17273479011857235062 RealInstId: 12606056968936758026 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 15536514517640462487 RealInstId: 8303154871847919944 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 16884277835173449069 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 16884277835173449069 Defn: DefnId: 4987105685507282285 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17465522551378543576 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:40:34.531+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:40:34.531+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:40:34.538+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken85:1a:bd:c3:3d:5d:7a:87 2023-10-21T04:40:34.574+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken95:67:90:88:a3:f5:42:a9 2023-10-21T04:40:34.581+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken0:62:86:e2:55:57:3b:3 2023-10-21T04:40:34.607+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken63:5e:6f:93:50:1c:47:1 2023-10-21T04:40:34.607+05:30 [Info] ShardRebalancer::observeRebalance &{4325b30e7f9620a998388078247335a8 4ba7e1420e46bd0aae5b8713b46bb0d0 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:40:34.705+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken63:5e:6f:93:50:1c:47:1 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken95:67:90:88:a3:f5:42:a9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15867764812233117191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 5040311435261117886 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7774786113586735969 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:34.706+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken0:62:86:e2:55:57:3b:3 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:1a:bd:c3:3d:5d:7a:87 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12527610516055683489 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18172419679785906247 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2935911152716188465 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:34.706+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken95:67:90:88:a3:f5:42:a9 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken63:5e:6f:93:50:1c:47:1 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16954811547177290607 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1438807282966389924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 117474327967883953 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:34.707+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:1a:bd:c3:3d:5d:7a:87 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken0:62:86:e2:55:57:3b:3 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17273479011857235062 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15536514517640462487 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16884277835173449069 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:34.707+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"4ba7e1420e46bd0aae5b8713b46bb0d0","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:40:34.707+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:40:34.723+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:1a:bd:c3:3d:5d:7a:87 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken0:62:86:e2:55:57:3b:3 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17273479011857235062 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15536514517640462487 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16884277835173449069 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:34.795+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken0:62:86:e2:55:57:3b:3 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:1a:bd:c3:3d:5d:7a:87 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12527610516055683489 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18172419679785906247 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2935911152716188465 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:34.795+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:40:34.795+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:40:34.795+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:40:34.796+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 62.027µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:34.800+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:1a:bd:c3:3d:5d:7a:87 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken0:62:86:e2:55:57:3b:3 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17273479011857235062 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15536514517640462487 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16884277835173449069 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:34.837+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 1, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:40:34.969+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:40:34.970+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 3] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/4ba7e1420e46bd0aae5b8713b46bb0d0 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:4ba7e1420e46bd0aae5b8713b46bb0d0]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/4ba7e1420e46bd0aae5b8713b46bb0d0 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:4ba7e1420e46bd0aae5b8713b46bb0d0]}]} 2023-10-21T04:40:34.976+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:40:34.977+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 3] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:40:34.990+05:30 [Info] GenericServiceManager::CancelTask: called with id: rebalance/4ba7e1420e46bd0aae5b8713b46bb0d0, rev: [] 2023-10-21T04:40:34.990+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: rebalance/4ba7e1420e46bd0aae5b8713b46bb0d0, rev: [] 2023-10-21T04:40:34.990+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:40:34.990+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:40:34.990+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress cancel received. progress collector stopped 2023-10-21T04:40:34.990+05:30 [Info] ShardRebalancer::updateProgress Cancel Received. progress reporter stapped 2023-10-21T04:40:35.120+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:40:35.357+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:35.357+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:40:35.374+05:30 [Info] switched currmeta from 20 -> 20 force true 2023-10-21T04:40:35.402+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:40:35.425+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:40:35.475+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:35.475+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:40:35.479+05:30 [Info] switched currmeta from 20 -> 20 force true 2023-10-21T04:40:37.897+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:40:37.898+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken95:67:90:88:a3:f5:42:a9 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken63:5e:6f:93:50:1c:47:1 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16954811547177290607 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1438807282966389924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 117474327967883953 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:37.898+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:40:37.900+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {4325b30e7f9620a998388078247335a8 4ba7e1420e46bd0aae5b8713b46bb0d0 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:40:37.928+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:40:37.928+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken0:62:86:e2:55:57:3b:3 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 4ba7e1420e46bd0aae5b8713b46bb0d0 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:1a:bd:c3:3d:5d:7a:87 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12527610516055683489 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18172419679785906247 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2935911152716188465 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:40:37.943+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:40:37.943+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:40:37.943+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.659µs 2023-10-21T04:40:37.943+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:40:37.943+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:40:37.943+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:40:37.947+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:40:37.947+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:40:37.947+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:40:37.948+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:40:37.948+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 34.396µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:37.951+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:40:37.951+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:40:37.951+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced false, isMaster true, forceUnbalanced true, err: 2023-10-21T04:40:37.951+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 2.960638141s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:40:37.951+05:30 [Info] GenericServiceManager::CancelTask: return from id: rebalance/4ba7e1420e46bd0aae5b8713b46bb0d0, rev: [] call. err: 2023-10-21T04:40:37.951+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:40:37.951+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:40:37.951+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 3] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[]} 2023-10-21T04:40:37.953+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/4ba7e1420e46bd0aae5b8713b46bb0d0, rev: [] 2023-10-21T04:40:37.953+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/4ba7e1420e46bd0aae5b8713b46bb0d0, rev: [] 2023-10-21T04:40:37.953+05:30 [Info] PauseServiceManager::endTask: called with err for taskId prepare/4ba7e1420e46bd0aae5b8713b46bb0d0 2023-10-21T04:40:37.953+05:30 [Info] PauseServiceManager::endTask task with ID prepare/4ba7e1420e46bd0aae5b8713b46bb0d0 already cleaned up 2023-10-21T04:40:37.953+05:30 [Info] PauseServiceManager::PauseResumeCancelTask: removed all tasks with ID prepare/4ba7e1420e46bd0aae5b8713b46bb0d0 2023-10-21T04:40:37.953+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/4ba7e1420e46bd0aae5b8713b46bb0d0, rev: [] call. err: 2023-10-21T04:40:37.955+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:40:37.955+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[]} 2023-10-21T04:40:37.962+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:40:37.962+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:40:37.964+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:40:37.964+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:40:38.025+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9106 2023-10-21T04:40:38.025+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:40:38.043+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:40:38.086+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:40:38.086+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9112 2023-10-21T04:40:38.086+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:40:38.094+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:40:38.120+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:40:38.121+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:40:38.121+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:40:38.127+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:48234 2023-10-21T04:40:38.128+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:40:38.141+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 10.577218ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:40:38.141+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:40:38.141+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:40:38.141+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:40:38.147+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:40:38.168+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:40:38.169+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:40:38.169+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:40:38.169+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:40:38.169+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:40:38.173+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:40:38.173+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:40:38.173+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:40:38.173+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:40:38.173+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:40:38.173+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:40:38.173+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:40:38.174+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:39478->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:40:38.174+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:39942->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:40:38.174+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:44610->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:40:38.174+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:48234->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:40:38.174+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:48234. Reason = EOF. Cleaning Pipe. 2023-10-21T04:40:38.174+05:30 [Info] messageListener.start(): message channel closed. Remove peer 23:86:b:a5:b8:3c:ef:d6 as follower. 2023-10-21T04:40:38.641+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:40:39.416+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:40:39.442+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:40:52.933+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:52.933+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:40:52.944+05:30 [Info] switched currmeta from 20 -> 24 force true 2023-10-21T04:40:53.088+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:53.088+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:40:53.101+05:30 [Info] switched currmeta from 24 -> 28 force true 2023-10-21T04:40:53.192+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:53.192+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:40:53.194+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:40:57.790+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:40:57.795+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:40:57.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:40:57.797+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:41:01.720+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:41:04.835+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:41:04.856+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:04.856+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:04.860+05:30 [Info] switched currmeta from 28 -> 28 force true 2023-10-21T04:41:07.964+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. err: operation_canceled 2023-10-21T04:41:07.964+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:41:07.964+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[]} 2023-10-21T04:41:07.965+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:41:07.965+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:41:07.966+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:41:07.967+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:41:13.456+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 1301336828089581719 2023-10-21T04:41:13.456+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 310.612µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:13.526+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33816 2023-10-21T04:41:13.530+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33816 2023-10-21T04:41:13.569+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33816 2023-10-21T04:41:13.630+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 1301336828089581719 2023-10-21T04:41:13.630+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 367.023µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:13.803+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:13.803+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:13.809+05:30 [Info] switched currmeta from 28 -> 37 force true 2023-10-21T04:41:13.926+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:13.926+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:13.929+05:30 [Info] switched currmeta from 37 -> 37 force true 2023-10-21T04:41:14.476+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49012 2023-10-21T04:41:14.495+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 13.494488ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:14.504+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 1301336828089581719. No update is performed. 2023-10-21T04:41:14.504+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 91.818µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:14.512+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49012. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:14.512+05:30 [Info] messageListener.start(): message channel closed. Remove peer 34:85:dc:ca:4f:1e:ee:88 as follower. 2023-10-21T04:41:14.669+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:41:14.669+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:41:14.685+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:41:14.729+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:41:14.729+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:41:14.729+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:41:14.755+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:41:14.791+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49048 2023-10-21T04:41:14.818+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.070562ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:14.820+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49048. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:14.820+05:30 [Info] messageListener.start(): message channel closed. Remove peer 71:15:6e:4e:ff:3f:53:fd as follower. 2023-10-21T04:41:14.824+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49050 2023-10-21T04:41:14.837+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:41:14.837+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:41:14.837+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:41:14.866+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 38.121215ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:14.866+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49052 2023-10-21T04:41:14.867+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:41:14.892+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.207703ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:14.892+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:41:14.892+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:41:14.892+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:41:14.915+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49050. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:14.915+05:30 [Info] messageListener.start(): message channel closed. Remove peer 1a:b9:8f:15:90:f2:1b:4e as follower. 2023-10-21T04:41:14.918+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:41:14.933+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:41:14.933+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:41:14.933+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:41:14.933+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:41:14.933+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:41:14.933+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:41:14.934+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:41:14.934+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:41:14.934+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:41:14.934+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:41:14.934+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:40278->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:14.935+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49052. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:14.935+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:49052->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:14.935+05:30 [Info] messageListener.start(): message channel closed. Remove peer 46:14:8c:8b:61:fe:b8:b3 as follower. 2023-10-21T04:41:14.935+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:45430->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:14.936+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:40752->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:14.996+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:15.086+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:15.086+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:15.089+05:30 [Info] switched currmeta from 37 -> 38 force true 2023-10-21T04:41:15.128+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:15.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:15.133+05:30 [Info] switched currmeta from 38 -> 39 force true 2023-10-21T04:41:15.319+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:15.367+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:15.392+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:15.426+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:15.426+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:15.430+05:30 [Info] switched currmeta from 39 -> 40 force true 2023-10-21T04:41:15.460+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:15.461+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:15.465+05:30 [Info] switched currmeta from 40 -> 41 force true 2023-10-21T04:41:15.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13468795042188125131:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:41:15.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13468795042188125131:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:41:15.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13468795042188125131:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:41:15.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13468795042188125131:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:41:16.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11516149125615508476:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:41:16.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[11516149125615508476:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:41:17.861+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:41:17.866+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:17.866+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:17.868+05:30 [Info] switched currmeta from 41 -> 41 force true 2023-10-21T04:41:19.778+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 10761885902580101111 2023-10-21T04:41:19.778+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 177.728µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:19.841+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33816 2023-10-21T04:41:19.857+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33816 2023-10-21T04:41:19.903+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33816 2023-10-21T04:41:19.966+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 10761885902580101111 2023-10-21T04:41:19.966+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 294.998µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:20.026+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:20.027+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:20.029+05:30 [Info] switched currmeta from 41 -> 44 force true 2023-10-21T04:41:20.036+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:20.036+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:20.040+05:30 [Info] switched currmeta from 44 -> 45 force true 2023-10-21T04:41:20.047+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:20.047+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:20.053+05:30 [Info] switched currmeta from 45 -> 45 force true 2023-10-21T04:41:20.075+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:20.075+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:20.151+05:30 [Info] switched currmeta from 45 -> 49 force true 2023-10-21T04:41:20.238+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:20.238+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:20.245+05:30 [Info] switched currmeta from 49 -> 50 force true 2023-10-21T04:41:20.255+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:20.255+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:20.260+05:30 [Info] switched currmeta from 50 -> 50 force true 2023-10-21T04:41:20.360+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49196 2023-10-21T04:41:20.383+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.120919ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:20.423+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 10761885902580101111. No update is performed. 2023-10-21T04:41:20.423+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 200.221µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:20.433+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49196. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:20.433+05:30 [Info] messageListener.start(): message channel closed. Remove peer 54:9b:c5:29:c4:ff:7a:11 as follower. 2023-10-21T04:41:20.486+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49210 2023-10-21T04:41:20.558+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 59.909965ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:20.653+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:41:20.653+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:41:20.663+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49226 2023-10-21T04:41:20.664+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:41:20.686+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49210. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:20.686+05:30 [Info] messageListener.start(): message channel closed. Remove peer 61:b4:9d:c6:79:ff:c:10 as follower. 2023-10-21T04:41:20.689+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 21.937495ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:20.689+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:41:20.689+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:41:20.689+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:41:20.710+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:41:20.734+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:41:20.734+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:41:20.734+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:41:20.748+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:41:20.789+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49240 2023-10-21T04:41:20.797+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:41:20.797+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:41:20.797+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:41:20.811+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 17.15146ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:20.818+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:41:20.856+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49240. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:20.856+05:30 [Info] messageListener.start(): message channel closed. Remove peer a8:72:21:56:f:2a:32:65 as follower. 2023-10-21T04:41:20.876+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:41:20.876+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:41:20.876+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:41:20.876+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:41:20.876+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:41:20.876+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:41:20.877+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:41:20.877+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:41:20.877+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:41:20.877+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:41:20.877+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:49226->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:20.877+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:45606->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:20.877+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:40486->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:20.878+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:40952->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:20.878+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49226. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:20.878+05:30 [Info] messageListener.start(): message channel closed. Remove peer 78:5:74:85:fe:32:25:5d as follower. 2023-10-21T04:41:20.884+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:21.059+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:21.189+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:21.311+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:21.492+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:21.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:21.495+05:30 [Info] switched currmeta from 50 -> 51 force true 2023-10-21T04:41:21.619+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:21.619+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:21.627+05:30 [Info] switched currmeta from 51 -> 52 force true 2023-10-21T04:41:21.969+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:21.969+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:21.973+05:30 [Info] switched currmeta from 52 -> 53 force true 2023-10-21T04:41:22.133+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:22.133+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:22.136+05:30 [Info] switched currmeta from 53 -> 54 force true 2023-10-21T04:41:22.874+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:41:22.880+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:22.880+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:22.882+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:41:22.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[13132309598932905010:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:41:23.359+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9908440273132400823:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:41:23.360+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9908440273132400823:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:41:30.711+05:30 [Info] fragAutoTuner: FragRatio at 100. MaxFragRatio 100, MaxBandwidth 0. BandwidthUsage 0. AvailDisk 0. TotalUsed 0. BandwidthRatio 1. UsedSpaceRatio 1. CleanerBandwidth 0. Duration 0. 2023-10-21T04:41:31.720+05:30 [Info] janitor: running cleanup. 2023-10-21T04:41:31.720+05:30 [Info] builder: recovering scheduled index 2023-10-21T04:41:31.720+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[17797267771846799851 2048006213723315482 7392796230534034390 11619165464560280276 16337157693634799600 18026660730857541767 14741949230702299196 12582561312368048736 15431181238515168303 11276338709617092948 1252359852432998131 12833670934079048728 17656144409930763141 573521267164877540 10606939961365971246 8669649359784409226 8857269605176042330 7147407900589480727 16706481854123898870 4987105685507282285 12599498416578472779 12195715103909237684 16162012140118277065 4286621850367430051 4532813151040661705 5465835793930010953 12824340686309573428 13268712595133993155 4535197604682045916 4860549750398470299 3381524789655566900 8378417662949994532 8504686614261377811 14889401051926000176 15945117906390072636 15968368593314043432 7525085113233084253 14292583894719865678 12338036531800130039 12027054384338714804 16961575949607441659 4138047918192025493 13856704970496601958 18046371353728341033 4790214984019693894 6985049432225870291 18323460706027642575 249887442666643345 12140458060295645238 3674078063353481907 8910459001587546151 9665410041638025825 11365633301282701856 14549775068266281786]} 2023-10-21T04:41:31.722+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:41:31.728+05:30 [Info] memstats {"Alloc":93283752, "TotalAlloc":96911440, "Sys":112935192, "Lookups":0, "Mallocs":138917,"Frees":39595, "HeapAlloc":93283752, "HeapSys":98664448, "HeapIdle":2875392, "HeapInuse":95789056,"HeapReleased":2777088, "HeapObjects":99322,"MSpanInuse":280640, "MSpanSys": 293760, "StackInuse": 1998848,"GCSys":9866992, "LastGC":1697843431376743160,"PauseTotalNs":1531822, "PauseNs":[279858, 359068, 66765, 530410, 295721], "NumGC":5} 2023-10-21T04:41:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:41:32.787+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:41:32.836+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:41:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:41:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:41:32.891+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:41:33.132+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:41:33.150+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:41:33.389+05:30 [Info] Indexer::ReadMemstats Time Taken 386.466µs 2023-10-21T04:41:33.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9794995822644119, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:41:34.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.999493472281893, throttleDelayMs (new, old, change): (8, 3, 5) 2023-10-21T04:41:35.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9406684554158842, throttleDelayMs (new, old, change): (7, 8, -1) 2023-10-21T04:41:36.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6847489098740938, throttleDelayMs (new, old, change): (2, 7, -5) 2023-10-21T04:41:37.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.5618496991342987, throttleDelayMs (new, old, change): (0, 2, -2) 2023-10-21T04:41:37.716+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:41:37.870+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:37.870+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:37.883+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:41:37.961+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:41:37.968+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. err: operation_canceled 2023-10-21T04:41:37.968+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:41:37.968+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 4] Tasks:[]} 2023-10-21T04:41:37.968+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:41:37.968+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 4] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:41:37.981+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 4] 2023-10-21T04:41:37.983+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 4] 2023-10-21T04:41:37.985+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:37.985+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:37.991+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:41:38.019+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:41:38.020+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:41:38.135+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:41:38.310+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:38.310+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:38.314+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:41:39.649+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {d0d7b49425bd5957cb45e587657b5aa8 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:41:39.651+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 8.106µs 2023-10-21T04:41:39.651+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:41:39.651+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:41:39.651+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:41:39.655+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:41:39.655+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:41:39.655+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 4.063637ms 2023-10-21T04:41:39.655+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:41:39.655+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 9.063µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:39.655+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:41:39.655+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 4] call. taskList: &{Rev:[0 0 0 0 0 0 0 5] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/d0d7b49425bd5957cb45e587657b5aa8 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d0d7b49425bd5957cb45e587657b5aa8]}]} 2023-10-21T04:41:39.655+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 5] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:41:39.659+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 5] 2023-10-21T04:41:39.659+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 5] 2023-10-21T04:41:39.669+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {d0d7b49425bd5957cb45e587657b5aa8 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:41:39.672+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:41:39.702+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"d0d7b49425bd5957cb45e587657b5aa8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:41:39.704+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{4325b30e7f9620a998388078247335a8 d0d7b49425bd5957cb45e587657b5aa8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:41:39.706+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 d0d7b49425bd5957cb45e587657b5aa8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:41:40.715+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:41:40.721+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:41:40.721+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:41:40.727+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:41:40.727+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{4325b30e7f9620a998388078247335a8 d0d7b49425bd5957cb45e587657b5aa8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:41:40.727+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:41:40.728+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.058818506s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:41:40.727+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 5] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/d0d7b49425bd5957cb45e587657b5aa8 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d0d7b49425bd5957cb45e587657b5aa8]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/d0d7b49425bd5957cb45e587657b5aa8 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d0d7b49425bd5957cb45e587657b5aa8]}]} 2023-10-21T04:41:40.728+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{4325b30e7f9620a998388078247335a8 d0d7b49425bd5957cb45e587657b5aa8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:41:40.728+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:41:40.728+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:41:40.730+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:41:40.731+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:41:40.731+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:41:40.731+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/d0d7b49425bd5957cb45e587657b5aa8 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d0d7b49425bd5957cb45e587657b5aa8]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/d0d7b49425bd5957cb45e587657b5aa8 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d0d7b49425bd5957cb45e587657b5aa8]}]} 2023-10-21T04:41:40.734+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 6] 2023-10-21T04:41:40.734+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 6] 2023-10-21T04:41:40.778+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{d7ff2bfd429b21d9c24d822bc13e3acc d7ff2bfd429b21d9c24d822bc13e3acc plasma map[] [{21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{12606056968936758026 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{8303154871847919944 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 10761885902580101111 [{9908440273132400823 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{17465522551378543576 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501] []}] 1 0 0 1 false plasma 0}]}]} {21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{9316375249999768625 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{3751526798537645569 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 1301336828089581719 [{11516149125615508476 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{18137927630988528516 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 1301336828089581719 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10761885902580101111 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {9b27839d59b06374125d80a2b12a21b6 9b27839d59b06374125d80a2b12a21b6 plasma map[] [{21 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{16759912838651900043 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{4186965058349029465 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 1301336828089581719 [{13468795042188125131 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{544693394183450628 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215] []}] 1 0 0 0 false plasma 0}]}]} {21 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{4397940545762329850 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{12396348969790516872 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 10761885902580101111 [{13132309598932905010 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{10158339249566909515 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 1301336828089581719 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 10761885902580101111 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {4325b30e7f9620a998388078247335a8 4325b30e7f9620a998388078247335a8 plasma map[] [] [] 0 0 0 false} {ba6304988a6335d65281381d6cfc5e32 ba6304988a6335d65281381d6cfc5e32 plasma map[] [] [] 0 0 0 false}] map[]} 2023-10-21T04:41:40.828+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:41:40.835+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:41:40.835+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:41:40.835+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:41:40.835+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:41:40.846+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:41:40.887+05:30 [Info] Planner::processDeleteToken: processing 54 tokens for node 127.0.0.1:9001 2023-10-21T04:41:40.887+05:30 [Info] Planner::processDeleteToken: processing 54 tokens for node 127.0.0.1:9003 2023-10-21T04:41:40.888+05:30 [Info] Planner::processDeleteToken: processing 54 tokens for node 127.0.0.1:9004 2023-10-21T04:41:40.888+05:30 [Info] Planner::processDeleteToken: processing 54 tokens for node 127.0.0.1:9002 2023-10-21T04:41:40.889+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:41:40.898+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:41:40.898+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:41:40.898+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:41:40.898+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:41:40.940+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:41:40.940+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:41:40.940+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:41:40.940+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:41:40.940+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:41:40.940+05:30 [Info] Moving index 668551233566132751:544693394183450628:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 10761885902580101111:13132309598932905010:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 5277316367626246544:10158339249566909515:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.940+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 1301336828089581719:13468795042188125131:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:41:40.941+05:30 [Info] Moving index 5277316367626246544:17465522551378543576:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 1301336828089581719:11516149125615508476:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 668551233566132751:18137927630988528516:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.941+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.942+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.942+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.942+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.942+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.942+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.942+05:30 [Info] Moving index 10761885902580101111:9908440273132400823:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:41:40.942+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 17725104 Units 0 2023-10-21T04:41:40.942+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17720378 Units 0 2023-10-21T04:41:40.942+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 280570 Units 6 Tenants 0 2023-10-21T04:41:40.942+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 275572 Units 11 Tenants 0 2023-10-21T04:41:40.942+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:41:40.942+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:41:40.942+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:41:40.942+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:41:40.942+05:30 [Info] ---------------------------------------- 2023-10-21T04:41:40.942+05:30 [Info] 2023-10-21T04:41:40.942+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:ba6304988a6335d65281381d6cfc5e32, useLiveData:false 2023-10-21T04:41:40.942+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:41:40.942+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:41:40.942+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.942+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:17465522551378543576, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501], shardIdsInDefn: map[] 2023-10-21T04:41:40.942+05:30 [Info] Index total memory:157293731 (150.007M), mem:6348 (6.19922K), overhead:157287383 (150.001M), min:157293731 (150.007M), data:6348 (6.19922K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.942+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.942+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.942+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.942+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.942+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.942+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.942+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.942+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.942+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.942+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.942+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.942+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.942+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.942+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.942+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.942+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.942+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.942+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.942+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.942+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.942+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.942+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.943+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.943+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.943+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.943+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:1301336828089581719, instId:11516149125615508476, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:18137927630988528516, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157293787 (150.007M), mem:6396 (6.24609K), overhead:157287391 (150.001M), min:157293787 (150.007M), data:6396 (6.24609K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.944+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.944+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.944+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.944+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.945+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:10761885902580101111, instId:9908440273132400823, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:41:40.945+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.945+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.945+05:30 [Info] 2023-10-21T04:41:40.945+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:4325b30e7f9620a998388078247335a8, useLiveData:false 2023-10-21T04:41:40.945+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:41:40.946+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:544693394183450628, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157293731 (150.007M), mem:6348 (6.19922K), overhead:157287383 (150.001M), min:157293731 (150.007M), data:6348 (6.19922K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.946+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.946+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.946+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.946+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:10761885902580101111, instId:13132309598932905010, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:10158339249566909515, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:157293759 (150.007M), mem:6372 (6.22266K), overhead:157287387 (150.001M), min:157293759 (150.007M), data:6372 (6.22266K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.947+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.947+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.947+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.947+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.948+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.948+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.948+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.948+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.949+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.949+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.949+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.949+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.949+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.949+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.949+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.949+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.949+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.949+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.949+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:41:40.949+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.949+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.949+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:41:40.949+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:1301336828089581719, instId:13468795042188125131, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:41:40.949+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:41:40.949+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:41:40.949+05:30 [Info] **************************************** 2023-10-21T04:41:40.949+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 280570 Units 6 Tenants 0 2023-10-21T04:41:40.949+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 275572 Units 11 Tenants 0 2023-10-21T04:41:40.949+05:30 [Info] Generating Shard Transfer Token (ShardToken69:5:45:65:73:79:5:f3) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf0:a4:fe:7d:6d:5e:a:ed Shards: [17327901024118572215 5629943257627822009] Destination: Region: InstId: 9373487212359538092 RealInstId: 16759912838651900043 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12082738937420039247 RealInstId: 4186965058349029465 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 13468795042188125131 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 13468795042188125131 Defn: DefnId: 1301336828089581719 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 544693394183450628 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:41:40.950+05:30 [Info] Generating Shard Transfer Token (ShardTokenca:a:b0:9e:a5:72:f3:65) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Shards: [18374611443167983589 6455624821495495854] Destination: Region: InstId: 13132309598932905010 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 13132309598932905010 Defn: DefnId: 10761885902580101111 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12503118843503725381 RealInstId: 4397940545762329850 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 7034278872286690461 RealInstId: 12396348969790516872 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10158339249566909515 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:41:40.950+05:30 [Info] Generating Shard Transfer Token (ShardToken56:e8:7e:55:3a:22:3c:a2) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenca:a:b0:9e:a5:72:f3:65 Shards: [10838601320854145501 11711860383628641160] Destination: Region: InstId: 80078467528743387 RealInstId: 12606056968936758026 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 5518688293309066227 RealInstId: 8303154871847919944 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 9908440273132400823 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 9908440273132400823 Defn: DefnId: 10761885902580101111 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17465522551378543576 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:41:40.951+05:30 [Info] Generating Shard Transfer Token (ShardTokenf0:a4:fe:7d:6d:5e:a:ed) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken69:5:45:65:73:79:5:f3 Shards: [11114961608238406563 5748926106587204814] Destination: Region: InstId: 11516149125615508476 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 11516149125615508476 Defn: DefnId: 1301336828089581719 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10507745352028749098 RealInstId: 9316375249999768625 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 9316375249999768625 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 3700866087774394376 RealInstId: 3751526798537645569 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3751526798537645569 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 18137927630988528516 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 18137927630988528516 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:41:40.951+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:41:40.951+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:41:40.955+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken56:e8:7e:55:3a:22:3c:a2 2023-10-21T04:41:40.990+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenf0:a4:fe:7d:6d:5e:a:ed 2023-10-21T04:41:41.008+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken69:5:45:65:73:79:5:f3 2023-10-21T04:41:41.019+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenca:a:b0:9e:a5:72:f3:65 2023-10-21T04:41:41.019+05:30 [Info] ShardRebalancer::observeRebalance &{4325b30e7f9620a998388078247335a8 d0d7b49425bd5957cb45e587657b5aa8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:41:41.055+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenca:a:b0:9e:a5:72:f3:65 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 13132309598932905010 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12503118843503725381 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7034278872286690461 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.068+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"d0d7b49425bd5957cb45e587657b5aa8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:41:41.068+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:41:41.070+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken56:e8:7e:55:3a:22:3c:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenca:a:b0:9e:a5:72:f3:65 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 80078467528743387 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5518688293309066227 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9908440273132400823 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.071+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken69:5:45:65:73:79:5:f3 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf0:a4:fe:7d:6d:5e:a:ed Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9373487212359538092 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12082738937420039247 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13468795042188125131 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.075+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf0:a4:fe:7d:6d:5e:a:ed MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken69:5:45:65:73:79:5:f3 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11516149125615508476 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10507745352028749098 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3700866087774394376 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.112+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf0:a4:fe:7d:6d:5e:a:ed MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken69:5:45:65:73:79:5:f3 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11516149125615508476 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10507745352028749098 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3700866087774394376 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.146+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken56:e8:7e:55:3a:22:3c:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenca:a:b0:9e:a5:72:f3:65 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 80078467528743387 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5518688293309066227 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9908440273132400823 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.203+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken69:5:45:65:73:79:5:f3 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf0:a4:fe:7d:6d:5e:a:ed Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9373487212359538092 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12082738937420039247 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13468795042188125131 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.203+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:41:41.203+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:41:41.203+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:41:41.204+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.06µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:41.225+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenca:a:b0:9e:a5:72:f3:65 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 13132309598932905010 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12503118843503725381 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7034278872286690461 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.226+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:41:41.226+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:41:41.226+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:41:41.226+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.027µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:41.269+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken69:5:45:65:73:79:5:f3 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf0:a4:fe:7d:6d:5e:a:ed Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9373487212359538092 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12082738937420039247 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13468795042188125131 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.282+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf0:a4:fe:7d:6d:5e:a:ed MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken69:5:45:65:73:79:5:f3 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11516149125615508476 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10507745352028749098 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3700866087774394376 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.318+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenca:a:b0:9e:a5:72:f3:65 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 13132309598932905010 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12503118843503725381 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7034278872286690461 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.326+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:41:41.348+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 d0d7b49425bd5957cb45e587657b5aa8 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:41:41.348+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 1 2023-10-21T04:41:41.365+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[9b27839d59b06374125d80a2b12a21b6:ShardTokenca:a:b0:9e:a5:72:f3:65 d7ff2bfd429b21d9c24d822bc13e3acc:ShardToken56:e8:7e:55:3a:22:3c:a2], currActiveTransfers: map[9b27839d59b06374125d80a2b12a21b6:1 d7ff2bfd429b21d9c24d822bc13e3acc:1] 2023-10-21T04:41:41.366+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"d0d7b49425bd5957cb45e587657b5aa8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:41:41.366+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:41:41.366+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:41:41.366+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:41:41.366+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:41:41.366+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:41:41.366+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 17.328µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:41.367+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken56:e8:7e:55:3a:22:3c:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenca:a:b0:9e:a5:72:f3:65 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 80078467528743387 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5518688293309066227 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9908440273132400823 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.367+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenca:a:b0:9e:a5:72:f3:65 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 13132309598932905010 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12503118843503725381 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7034278872286690461 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.545+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:41:41.545+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 6] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/d0d7b49425bd5957cb45e587657b5aa8 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d0d7b49425bd5957cb45e587657b5aa8]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/d0d7b49425bd5957cb45e587657b5aa8 Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:d0d7b49425bd5957cb45e587657b5aa8]}]} 2023-10-21T04:41:41.555+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:41:41.555+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 6] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:41:41.582+05:30 [Info] GenericServiceManager::CancelTask: called with id: rebalance/d0d7b49425bd5957cb45e587657b5aa8, rev: [] 2023-10-21T04:41:41.582+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: rebalance/d0d7b49425bd5957cb45e587657b5aa8, rev: [] 2023-10-21T04:41:41.582+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:41:41.582+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:41:41.582+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress cancel received. progress collector stopped 2023-10-21T04:41:41.583+05:30 [Info] ShardRebalancer::updateProgress Cancel Received. progress reporter stapped 2023-10-21T04:41:41.583+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:41:41.583+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:41:41.597+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {4325b30e7f9620a998388078247335a8 d0d7b49425bd5957cb45e587657b5aa8 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:41:41.844+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:41:41.862+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:41:41.862+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenf0:a4:fe:7d:6d:5e:a:ed MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken69:5:45:65:73:79:5:f3 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 11516149125615508476 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10507745352028749098 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3700866087774394376 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.862+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken56:e8:7e:55:3a:22:3c:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenca:a:b0:9e:a5:72:f3:65 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 80078467528743387 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5518688293309066227 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9908440273132400823 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.862+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken69:5:45:65:73:79:5:f3 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf0:a4:fe:7d:6d:5e:a:ed Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 9373487212359538092 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12082738937420039247 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13468795042188125131 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.923+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenca:a:b0:9e:a5:72:f3:65 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 13132309598932905010 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12503118843503725381 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7034278872286690461 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:41:41.981+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokenca:a:b0:9e:a5:72:f3:65 2023-10-21T04:41:41.981+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: d0d7b49425bd5957cb45e587657b5aa8 TransferTokenId: ShardTokenca:a:b0:9e:a5:72:f3:65 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:41:41.981+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000121233 2023-10-21T04:41:41.981+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardToken56:e8:7e:55:3a:22:3c:a2 2023-10-21T04:41:41.981+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d0d7b49425bd5957cb45e587657b5aa8_ShardTokenca_a_b0_9e_a5_72_f3_65, shardId :0 2023-10-21T04:41:41.981+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d0d7b49425bd5957cb45e587657b5aa8_ShardTokenca_a_b0_9e_a5_72_f3_65, shardId :0 2023-10-21T04:41:41.981+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: d0d7b49425bd5957cb45e587657b5aa8 TransferTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:41:41.982+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 5.9467e-05 2023-10-21T04:41:41.982+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:41:41.982+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:41:41.982+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.463µs 2023-10-21T04:41:41.982+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d0d7b49425bd5957cb45e587657b5aa8_ShardToken56_e8_7e_55_3a_22_3c_a2, shardId :0 2023-10-21T04:41:41.982+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d0d7b49425bd5957cb45e587657b5aa8_ShardToken56_e8_7e_55_3a_22_3c_a2, shardId :0 2023-10-21T04:41:41.982+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:41:41.982+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:41:41.982+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:41:41.984+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:41:41.984+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:41:41.984+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:41:41.985+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:41:41.985+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 32.319µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:41.988+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:41:41.988+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:41:41.988+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced false, isMaster true, forceUnbalanced true, err: 2023-10-21T04:41:41.988+05:30 [Info] GenericServiceManager::CancelTask: return from id: rebalance/d0d7b49425bd5957cb45e587657b5aa8, rev: [] call. err: 2023-10-21T04:41:41.988+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 6] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[]} 2023-10-21T04:41:41.988+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:41:41.988+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:41:41.993+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/d0d7b49425bd5957cb45e587657b5aa8, rev: [] 2023-10-21T04:41:41.993+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/d0d7b49425bd5957cb45e587657b5aa8, rev: [] 2023-10-21T04:41:41.993+05:30 [Info] PauseServiceManager::endTask: called with err for taskId prepare/d0d7b49425bd5957cb45e587657b5aa8 2023-10-21T04:41:41.993+05:30 [Info] PauseServiceManager::endTask task with ID prepare/d0d7b49425bd5957cb45e587657b5aa8 already cleaned up 2023-10-21T04:41:41.993+05:30 [Info] PauseServiceManager::PauseResumeCancelTask: removed all tasks with ID prepare/d0d7b49425bd5957cb45e587657b5aa8 2023-10-21T04:41:41.993+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/d0d7b49425bd5957cb45e587657b5aa8, rev: [] call. err: 2023-10-21T04:41:42.006+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:41:42.006+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[]} 2023-10-21T04:41:42.009+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:41:42.009+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:41:42.023+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 7] 2023-10-21T04:41:42.023+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 7] 2023-10-21T04:41:42.159+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:42.160+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:42.168+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:41:42.227+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:41:42.253+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:41:42.253+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:41:42.259+05:30 [Info] switched currmeta from 54 -> 54 force true 2023-10-21T04:41:42.262+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:41:42.273+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9112 2023-10-21T04:41:42.274+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:41:42.319+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:41:42.483+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:41:42.483+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:41:42.483+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:41:42.505+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:49778 2023-10-21T04:41:42.507+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:41:42.527+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.24542ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:41:42.527+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:41:42.527+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:41:42.527+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:41:42.536+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:41:42.568+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:41:42.568+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9106 2023-10-21T04:41:42.568+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:41:42.578+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:41:42.608+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:41:42.608+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:41:42.608+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:41:42.608+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:41:42.608+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:41:42.611+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:41:42.611+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:41:42.611+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:41:42.611+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:41:42.611+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:41:42.611+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:41:42.611+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:41:42.611+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:41482->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:42.611+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:49778->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:42.611+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:46154->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:42.612+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:41036->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:41:42.612+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:49778. Reason = EOF. Cleaning Pipe. 2023-10-21T04:41:42.612+05:30 [Info] messageListener.start(): message channel closed. Remove peer 38:56:d6:42:f0:32:bd:11 as follower. 2023-10-21T04:41:42.920+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:42.955+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:43.027+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:41:43.960+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:44.965+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:45.970+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:46.975+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:47.980+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:48.985+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:49.990+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:50.995+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:52.000+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:53.005+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:54.010+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:55.015+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:56.021+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:57.026+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:58.031+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:41:59.037+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:42:00.042+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:42:01.048+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:42:01.729+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:42:01.732+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Found 1 tokens. Cleaning up. 2023-10-21T04:42:01.732+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:42:01.732+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenca:a:b0:9e:a5:72:f3:65 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: d0d7b49425bd5957cb45e587657b5aa8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken56:e8:7e:55:3a:22:3c:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 13132309598932905010 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 12503118843503725381 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7034278872286690461 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:42:01.732+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: d0d7b49425bd5957cb45e587657b5aa8 TransferTokenId: ShardTokenca:a:b0:9e:a5:72:f3:65 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:42:01.733+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000118206 2023-10-21T04:42:01.733+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenca:a:b0:9e:a5:72:f3:65, dropIndexes: false 2023-10-21T04:42:01.733+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d0d7b49425bd5957cb45e587657b5aa8_ShardTokenca_a_b0_9e_a5_72_f3_65, shardId :0 2023-10-21T04:42:01.733+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/d0d7b49425bd5957cb45e587657b5aa8_ShardTokenca_a_b0_9e_a5_72_f3_65, shardId :0 2023-10-21T04:42:01.733+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [18374611443167983589 6455624821495495854] 2023-10-21T04:42:02.053+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:42:02.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 18374611443167983589 2023-10-21T04:42:02.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 6455624821495495854 2023-10-21T04:42:02.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 6455624821495495854 2023-10-21T04:42:02.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 18374611443167983589 2023-10-21T04:42:02.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [18374611443167983589 6455624821495495854], elapsed: 960.404353ms 2023-10-21T04:42:02.693+05:30 [Info] Start shard recovery from /opt/build/ns_server/data/n_3/data/@2i/shards 2023-10-21T04:42:02.693+05:30 [Info] Directory /opt/build/ns_server/data/n_3/data/@2i/shards does not exist. Skip shard recovery. 2023-10-21T04:42:02.693+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 18374611443167983589 from local file system, err: error destroy shard, shardId 18374611443167983589 not found 2023-10-21T04:42:02.693+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 6455624821495495854 from local file system, err: error destroy shard, shardId 6455624821495495854 not found 2023-10-21T04:42:02.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [18374611443167983589 6455624821495495854], elapsed(sec): 0.960569791 2023-10-21T04:42:02.703+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 974.736276ms in RebalanceServiceManager::rebalanceJanitor 2023-10-21T04:42:03.034+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 244] aggregate storage info: map[bucket_%2:map[kv:2324672 index:3146059] bucket_1:map[kv:2394304 index:3113114]] 2023-10-21T04:42:03.081+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:42:11.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9965617593553989, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:42:12.024+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 7] call. err: operation_canceled 2023-10-21T04:42:12.024+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:42:12.024+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:42:12.024+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[]} 2023-10-21T04:42:12.024+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:42:12.026+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 7] 2023-10-21T04:42:12.026+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 7] 2023-10-21T04:42:12.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6379264352344456, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:42:16.514+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:16.514+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:16.524+05:30 [Info] switched currmeta from 54 -> 58 force true 2023-10-21T04:42:16.538+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:16.538+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:16.542+05:30 [Info] switched currmeta from 58 -> 58 force true 2023-10-21T04:42:16.792+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:16.792+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:16.799+05:30 [Info] switched currmeta from 58 -> 62 force true 2023-10-21T04:42:16.827+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:16.827+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:16.836+05:30 [Info] switched currmeta from 62 -> 62 force true 2023-10-21T04:42:18.202+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:42:18.208+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:18.208+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:18.213+05:30 [Info] switched currmeta from 62 -> 62 force true 2023-10-21T04:42:30.711+05:30 [Info] fragAutoTuner: FragRatio at 100. MaxFragRatio 100, MaxBandwidth 0. BandwidthUsage 0. AvailDisk 0. TotalUsed 0. BandwidthRatio 1. UsedSpaceRatio 1. CleanerBandwidth 9223372036854775807. Duration 0. 2023-10-21T04:42:31.728+05:30 [Info] memstats {"Alloc":103225384, "TotalAlloc":162654960, "Sys":165907752, "Lookups":0, "Mallocs":701680,"Frees":544426, "HeapAlloc":103225384, "HeapSys":148701184, "HeapIdle":39321600, "HeapInuse":109379584,"HeapReleased":2064384, "HeapObjects":157254,"MSpanInuse":430400, "MSpanSys": 848640, "StackInuse": 2293760,"GCSys":11655712, "LastGC":1697843477790042875,"PauseTotalNs":1840710, "PauseNs":[308888], "NumGC":6} 2023-10-21T04:42:32.704+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:42:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:42:32.787+05:30 [Info] client stats current counts: current: 12, not current: 0 2023-10-21T04:42:32.863+05:30 [Info] num concurrent scans {0} 2023-10-21T04:42:32.863+05:30 [Info] average scan response {0 ms} 2023-10-21T04:42:32.902+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:42:32.932+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:42:33.171+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:42:33.194+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:42:33.390+05:30 [Info] Indexer::ReadMemstats Time Taken 430.677µs 2023-10-21T04:42:34.310+05:30 [Info] janitor: running cleanup. 2023-10-21T04:42:34.310+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[4286621850367430051 15945117906390072636 7392796230534034390 13268712595133993155 14549775068266281786 10606939961365971246 18026660730857541767 11619165464560280276 16961575949607441659 3674078063353481907 16162012140118277065 4860549750398470299 12027054384338714804 4138047918192025493 12195715103909237684 11276338709617092948 15968368593314043432 14292583894719865678 573521267164877540 7525085113233084253 18046371353728341033 9665410041638025825 16337157693634799600 13856704970496601958 12824340686309573428 4535197604682045916 8504686614261377811 249887442666643345 4532813151040661705 1252359852432998131 11365633301282701856 12833670934079048728 1301336828089581719 8378417662949994532 8669649359784409226 4790214984019693894 8910459001587546151 8857269605176042330 17656144409930763141 15431181238515168303 16706481854123898870 10761885902580101111 7147407900589480727 3381524789655566900 5465835793930010953 18323460706027642575 12338036531800130039 12582561312368048736 12140458060295645238 2048006213723315482 6985049432225870291 14889401051926000176 14741949230702299196 4987105685507282285 17797267771846799851 12599498416578472779]} 2023-10-21T04:42:36.667+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9957730726705247, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:42:37.577+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 17917116660857103172 2023-10-21T04:42:37.577+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 121.278µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:37.649+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35744 2023-10-21T04:42:37.658+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35744 2023-10-21T04:42:37.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9889065917494722, throttleDelayMs (new, old, change): (9, 5, 4) 2023-10-21T04:42:37.729+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35744 2023-10-21T04:42:37.803+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 17917116660857103172 2023-10-21T04:42:37.803+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 326.803µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:37.878+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:37.879+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:37.885+05:30 [Info] switched currmeta from 62 -> 65 force true 2023-10-21T04:42:37.901+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:37.901+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:37.905+05:30 [Info] switched currmeta from 65 -> 67 force true 2023-10-21T04:42:37.914+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:37.914+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:37.919+05:30 [Info] switched currmeta from 67 -> 67 force true 2023-10-21T04:42:38.023+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:38.023+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:38.027+05:30 [Info] switched currmeta from 67 -> 69 force true 2023-10-21T04:42:38.036+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:38.036+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:38.043+05:30 [Info] switched currmeta from 69 -> 69 force true 2023-10-21T04:42:38.095+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:38.095+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:38.108+05:30 [Info] switched currmeta from 69 -> 71 force true 2023-10-21T04:42:38.117+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:38.117+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:38.120+05:30 [Info] switched currmeta from 71 -> 71 force true 2023-10-21T04:42:38.344+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:50928 2023-10-21T04:42:38.383+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:42:38.409+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:42:38.426+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:38.426+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:38.428+05:30 [Info] switched currmeta from 71 -> 71 force true 2023-10-21T04:42:38.438+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:38.438+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:38.443+05:30 [Info] switched currmeta from 71 -> 71 force true 2023-10-21T04:42:38.491+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 143.098957ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:38.537+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 17917116660857103172. No update is performed. 2023-10-21T04:42:38.537+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 116.067µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:38.573+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:50970 2023-10-21T04:42:38.580+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:50928. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:38.580+05:30 [Info] messageListener.start(): message channel closed. Remove peer ef:76:b5:0:49:e0:63:f1 as follower. 2023-10-21T04:42:38.643+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:42:38.643+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:42:38.649+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 58.348276ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:38.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9702931971380173, throttleDelayMs (new, old, change): (10, 9, 1) 2023-10-21T04:42:38.668+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:42:38.697+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:42:38.697+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:42:38.697+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:42:38.703+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:42:38.740+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:50970. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:38.740+05:30 [Info] messageListener.start(): message channel closed. Remove peer a8:68:8:9f:29:b5:a8:93 as follower. 2023-10-21T04:42:38.757+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:42:38.758+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:42:38.758+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:42:38.764+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:50994 2023-10-21T04:42:38.764+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:42:38.779+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 13.209302ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:38.779+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:42:38.779+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:42:38.779+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:42:38.801+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:42:38.822+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:42:38.822+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:42:38.822+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:42:38.822+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:42:38.822+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:42:38.822+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:42:38.822+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:42:38.822+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:42:38.822+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:42:38.823+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:42230->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:38.823+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:47370->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:38.823+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:42696->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:38.828+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:42:38.829+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:50994. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:38.830+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6c:28:e2:b2:8e:1e:bf:f4 as follower. 2023-10-21T04:42:38.831+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:50994->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:38.836+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51000 2023-10-21T04:42:38.865+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 21.610908ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:38.871+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51000. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:38.873+05:30 [Info] messageListener.start(): message channel closed. Remove peer 61:ce:7c:92:b3:25:b1:cb as follower. 2023-10-21T04:42:38.992+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:39.150+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:39.279+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:39.366+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:39.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9599201613173383 2023-10-21T04:42:39.886+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:39.886+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:39.891+05:30 [Info] switched currmeta from 71 -> 72 force true 2023-10-21T04:42:39.991+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:39.991+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:39.998+05:30 [Info] switched currmeta from 72 -> 73 force true 2023-10-21T04:42:40.121+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:40.121+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:40.126+05:30 [Info] switched currmeta from 73 -> 74 force true 2023-10-21T04:42:40.179+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:40.180+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:40.186+05:30 [Info] switched currmeta from 74 -> 75 force true 2023-10-21T04:42:40.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7484981719049391, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:42:40.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6609266364331808928:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:42:40.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6609266364331808928:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:42:40.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6609266364331808928:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:42:40.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6609266364331808928:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:42:40.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[6609266364331808928:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:42:40.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9408787052754122519:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:42:40.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9408787052754122519:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:42:41.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.5242471055206764, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:42:42.026+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:42:42.027+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 7] call. err: operation_canceled 2023-10-21T04:42:42.027+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 7] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:42:42.027+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:42:42.027+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 7] Tasks:[]} 2023-10-21T04:42:42.028+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 7] 2023-10-21T04:42:42.028+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 7] 2023-10-21T04:42:43.285+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:42:43.301+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:43.301+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:43.303+05:30 [Info] switched currmeta from 75 -> 75 force true 2023-10-21T04:42:49.605+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 15973521466438086342 2023-10-21T04:42:49.606+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 164.019µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:49.661+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35744 2023-10-21T04:42:49.668+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35744 2023-10-21T04:42:49.698+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:35744 2023-10-21T04:42:49.762+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 15973521466438086342 2023-10-21T04:42:49.762+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 263.042µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:49.830+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:49.830+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:49.834+05:30 [Info] switched currmeta from 75 -> 78 force true 2023-10-21T04:42:49.844+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:49.845+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:49.854+05:30 [Info] switched currmeta from 78 -> 78 force true 2023-10-21T04:42:49.932+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:49.933+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:49.973+05:30 [Info] switched currmeta from 78 -> 82 force true 2023-10-21T04:42:50.001+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:50.002+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:50.025+05:30 [Info] switched currmeta from 82 -> 83 force true 2023-10-21T04:42:50.127+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:50.127+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:50.138+05:30 [Info] switched currmeta from 83 -> 84 force true 2023-10-21T04:42:50.154+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:50.154+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:50.176+05:30 [Info] switched currmeta from 84 -> 84 force true 2023-10-21T04:42:50.351+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51264 2023-10-21T04:42:50.376+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 22.773601ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:50.409+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 15973521466438086342. No update is performed. 2023-10-21T04:42:50.409+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 199.687µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:50.435+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51264. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:50.435+05:30 [Info] messageListener.start(): message channel closed. Remove peer b1:50:28:59:c3:80:e6:fe as follower. 2023-10-21T04:42:50.678+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:42:50.678+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:42:50.685+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51290 2023-10-21T04:42:50.695+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51294 2023-10-21T04:42:50.695+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:42:50.718+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51296 2023-10-21T04:42:50.727+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 39.577866ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:50.735+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 37.819079ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:50.735+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:42:50.735+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:42:50.735+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:42:50.758+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:42:50.783+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:42:50.783+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:42:50.783+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:42:50.794+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:42:50.806+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 85.543921ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:42:50.818+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:42:50.818+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:42:50.818+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:42:50.827+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:42:50.842+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51290. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:50.842+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2:d4:cb:52:cc:f7:1e:50 as follower. 2023-10-21T04:42:50.861+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51296. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:50.861+05:30 [Info] messageListener.start(): message channel closed. Remove peer bf:61:d8:43:d6:4b:b4:20 as follower. 2023-10-21T04:42:50.878+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:50.893+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:42:50.893+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:42:50.893+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:42:50.893+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:42:50.893+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:42:50.893+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:42:50.893+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:42:50.893+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:42:50.893+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:42:50.893+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:42:50.893+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:47676->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:50.894+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:42558->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:50.894+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:43020->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:50.894+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:51294->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:42:50.894+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51294. Reason = EOF. Cleaning Pipe. 2023-10-21T04:42:50.894+05:30 [Info] messageListener.start(): message channel closed. Remove peer d1:33:d9:85:8b:cd:85:9e as follower. 2023-10-21T04:42:51.228+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:51.236+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:51.301+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:51.301+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:51.307+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:42:51.315+05:30 [Info] switched currmeta from 84 -> 85 force true 2023-10-21T04:42:51.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:51.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:51.415+05:30 [Info] switched currmeta from 85 -> 86 force true 2023-10-21T04:42:51.924+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:51.924+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:51.928+05:30 [Info] switched currmeta from 86 -> 87 force true 2023-10-21T04:42:52.139+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:52.140+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:52.142+05:30 [Info] switched currmeta from 87 -> 88 force true 2023-10-21T04:42:52.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[190111758852025426:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:42:52.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[190111758852025426:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:42:52.859+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[190111758852025426:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:42:52.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[190111758852025426:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:42:52.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[190111758852025426:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:42:52.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[190111758852025426:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:42:52.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[3285983737505597326:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:42:53.328+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:42:53.338+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:42:53.338+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:42:53.340+05:30 [Info] switched currmeta from 88 -> 88 force true 2023-10-21T04:43:02.708+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:43:04.351+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:43:04.436+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:04.436+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:04.449+05:30 [Info] switched currmeta from 88 -> 88 force true 2023-10-21T04:43:04.591+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:43:04.591+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:43:04.702+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:04.702+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:04.715+05:30 [Info] switched currmeta from 88 -> 88 force true 2023-10-21T04:43:04.773+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:43:04.773+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:43:04.867+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:04.867+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:04.873+05:30 [Info] switched currmeta from 88 -> 88 force true 2023-10-21T04:43:04.881+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:43:04.899+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:04.899+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:04.913+05:30 [Info] switched currmeta from 88 -> 88 force true 2023-10-21T04:43:06.152+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {12808878c8875c24b4dda4f00a0dfee1 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:43:06.155+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 6.376µs 2023-10-21T04:43:06.156+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:43:06.156+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:43:06.156+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:43:06.164+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:43:06.164+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:43:06.164+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 8.563291ms 2023-10-21T04:43:06.164+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:43:06.164+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:43:06.164+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 9.881µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:06.164+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 8] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:43:06.164+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 7] call. taskList: &{Rev:[0 0 0 0 0 0 0 8] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/12808878c8875c24b4dda4f00a0dfee1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:12808878c8875c24b4dda4f00a0dfee1]}]} 2023-10-21T04:43:06.167+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 8] 2023-10-21T04:43:06.167+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 8] 2023-10-21T04:43:07.240+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {ba6304988a6335d65281381d6cfc5e32 12808878c8875c24b4dda4f00a0dfee1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:43:07.243+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {ba6304988a6335d65281381d6cfc5e32 12808878c8875c24b4dda4f00a0dfee1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:43:07.243+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"12808878c8875c24b4dda4f00a0dfee1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:43:07.245+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{ba6304988a6335d65281381d6cfc5e32 12808878c8875c24b4dda4f00a0dfee1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:43:07.245+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{ba6304988a6335d65281381d6cfc5e32 12808878c8875c24b4dda4f00a0dfee1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:43:07.246+05:30 [Info] ShardRebalancer::observeRebalance &{ba6304988a6335d65281381d6cfc5e32 12808878c8875c24b4dda4f00a0dfee1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:43:07.248+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"12808878c8875c24b4dda4f00a0dfee1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:43:07.248+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:43:07.436+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:43:07.443+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:43:07.488+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:43:07.610+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc7:2:b9:66:eb:19:6:5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1e:42:56:40:ff:fb:3c:bc Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18145658940511009543 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17789336037235700674 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 190111758852025426 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.711+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken23:e5:18:5c:c5:e7:19:62 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17495827337792382465 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9408787052754122519 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2853024886069516112 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.712+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:83:7c:94:6a:c6:e7:b5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken23:e5:18:5c:c5:e7:19:62 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6609266364331808928 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16122831054735124528 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15172753502274569092 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.717+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc7:2:b9:66:eb:19:6:5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1e:42:56:40:ff:fb:3c:bc Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18145658940511009543 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17789336037235700674 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 190111758852025426 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.815+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1e:42:56:40:ff:fb:3c:bc MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc7:2:b9:66:eb:19:6:5 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3285983737505597326 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3786026946915438592 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17840282284522026272 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.837+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:83:7c:94:6a:c6:e7:b5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken23:e5:18:5c:c5:e7:19:62 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6609266364331808928 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16122831054735124528 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15172753502274569092 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.923+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc7:2:b9:66:eb:19:6:5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1e:42:56:40:ff:fb:3c:bc Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18145658940511009543 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17789336037235700674 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 190111758852025426 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.982+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1e:42:56:40:ff:fb:3c:bc MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc7:2:b9:66:eb:19:6:5 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3285983737505597326 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3786026946915438592 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17840282284522026272 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.983+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:43:07.983+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:43:07.983+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:43:07.983+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.748µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:07.998+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken23:e5:18:5c:c5:e7:19:62 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17495827337792382465 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9408787052754122519 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2853024886069516112 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:07.999+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:43:07.999+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:43:07.999+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:43:07.999+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 23.721µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:08.036+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:83:7c:94:6a:c6:e7:b5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken23:e5:18:5c:c5:e7:19:62 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6609266364331808928 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16122831054735124528 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15172753502274569092 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:08.036+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken1e:42:56:40:ff:fb:3c:bc MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc7:2:b9:66:eb:19:6:5 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3285983737505597326 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3786026946915438592 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17840282284522026272 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:08.041+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken23:e5:18:5c:c5:e7:19:62 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17495827337792382465 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9408787052754122519 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2853024886069516112 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:08.153+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"12808878c8875c24b4dda4f00a0dfee1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:43:08.153+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:43:08.153+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:43:08.154+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:43:08.154+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:43:08.154+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:43:08.154+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 20.126µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:08.160+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken23:e5:18:5c:c5:e7:19:62 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17495827337792382465 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9408787052754122519 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2853024886069516112 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:08.166+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{ba6304988a6335d65281381d6cfc5e32 12808878c8875c24b4dda4f00a0dfee1 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:43:08.221+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:83:7c:94:6a:c6:e7:b5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken23:e5:18:5c:c5:e7:19:62 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6609266364331808928 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16122831054735124528 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15172753502274569092 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:08.350+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken23:e5:18:5c:c5:e7:19:62 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17495827337792382465 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9408787052754122519 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2853024886069516112 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:08.350+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:43:08.351+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:43:08.351+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:43:08.351+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 24.06µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:08.352+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[5629943257627822009:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/5629943257627822009/shards/shard5629943257627822009 17327901024118572215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/17327901024118572215/shards/shard17327901024118572215] 12808878c8875c24b4dda4f00a0dfee1 ShardToken23:e5:18:5c:c5:e7:19:62 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc006b22180 0xc006b221e0 map[] 0xc007147080 0xc004ceb9e0 false} 2023-10-21T04:43:08.352+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/5629943257627822009/shards/shard5629943257627822009 2023-10-21T04:43:08.358+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/17327901024118572215/shards/shard17327901024118572215 2023-10-21T04:43:08.367+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/5629943257627822009/shards/shard5629943257627822009/data: super block snapshot head offset :0 tail offset:528384 sbgen :97 2023-10-21T04:43:08.368+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/5629943257627822009/shards/shard5629943257627822009/data/recovery: super block snapshot head offset :0 tail offset:450560 sbgen :97 2023-10-21T04:43:08.371+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/5629943257627822009/shards/shard5629943257627822009: metadata saved successfully 2023-10-21T04:43:08.372+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) added 2023-10-21T04:43:08.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:43:08.372+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 49901400 totalBytes: 998028 numShards: 2 duration: 20.04707ms 2023-10-21T04:43:08.372+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/5629943257627822009/shards/shard5629943257627822009, newshardId 5629943257627822009, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, shardsize 998028, duration 20.042204ms 2023-10-21T04:43:08.372+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 5629943257627822009, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, err: , elapsed(sec): 0.020334262 2023-10-21T04:43:08.372+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 5629943257627822009 bytesWritten: 998028, totalBytes: 998028, transferRate: 5.2527789473684214e+07 2023-10-21T04:43:08.374+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/17327901024118572215/shards/shard17327901024118572215/data: super block snapshot head offset :0 tail offset:634880 sbgen :106 2023-10-21T04:43:08.375+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/17327901024118572215/shards/shard17327901024118572215/data/recovery: super block snapshot head offset :0 tail offset:573440 sbgen :106 2023-10-21T04:43:08.419+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/17327901024118572215/shards/shard17327901024118572215: metadata saved successfully 2023-10-21T04:43:08.425+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) added 2023-10-21T04:43:08.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:43:08.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62/17327901024118572215/shards/shard17327901024118572215, newshardId 17327901024118572215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, shardsize 1227561, duration 67.310565ms 2023-10-21T04:43:08.425+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17327901024118572215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, err: , elapsed(sec): 0.073585155 2023-10-21T04:43:08.425+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 12808878c8875c24b4dda4f00a0dfee1, transferId: ShardToken23:e5:18:5c:c5:e7:19:62 2023-10-21T04:43:08.426+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 12808878c8875c24b4dda4f00a0dfee1, transferId: ShardToken23:e5:18:5c:c5:e7:19:62, elapsed(sec): 0.000282722 2023-10-21T04:43:08.426+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[5629943257627822009: 17327901024118572215:], shardPaths: map[5629943257627822009:/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 17327901024118572215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.074227939 2023-10-21T04:43:08.425+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17327901024118572215 bytesWritten: 1227561, totalBytes: 1227561, transferRate: 1.8321805970149253e+07 2023-10-21T04:43:08.426+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17327901024118572215 5629943257627822009], ttid: ShardToken23:e5:18:5c:c5:e7:19:62, elapsed(sec): 0.075869427 2023-10-21T04:43:08.457+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 3, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:43:08.529+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:43:08.529+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 8] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/12808878c8875c24b4dda4f00a0dfee1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:12808878c8875c24b4dda4f00a0dfee1]}]} 2023-10-21T04:43:08.543+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:43:08.543+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 8] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:43:08.556+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/12808878c8875c24b4dda4f00a0dfee1, rev: [] 2023-10-21T04:43:08.556+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/12808878c8875c24b4dda4f00a0dfee1, rev: [] 2023-10-21T04:43:08.556+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:43:08.556+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:43:08.556+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:43:08.556+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:43:08.681+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:08.681+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:08.695+05:30 [Info] switched currmeta from 88 -> 90 force true 2023-10-21T04:43:09.015+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:43:09.103+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:09.103+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:09.137+05:30 [Info] switched currmeta from 90 -> 91 force true 2023-10-21T04:43:09.567+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:09.568+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:09.619+05:30 [Info] switched currmeta from 91 -> 91 force true 2023-10-21T04:43:09.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9998881427852971, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:43:09.889+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:43:10.082+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:10.125+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:51784 2023-10-21T04:43:10.343+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:43:10.443+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:10.457+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:10.457+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:10.503+05:30 [Info] switched currmeta from 91 -> 91 force true 2023-10-21T04:43:10.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 1, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:43:10.721+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 592.691129ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:10.769+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:10.769+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:10.790+05:30 [Info] switched currmeta from 91 -> 91 force true 2023-10-21T04:43:11.125+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:51784. Reason = EOF. Cleaning Pipe. 2023-10-21T04:43:11.125+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5:f7:b6:cc:74:bb:76:cc as follower. 2023-10-21T04:43:11.221+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:43:11.558+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:43:11.584+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:11.602+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:43:11.602+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken23:e5:18:5c:c5:e7:19:62 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17495827337792382465 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9408787052754122519 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2853024886069516112 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:11.602+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 12808878c8875c24b4dda4f00a0dfee1 TransferTokenId: ShardToken23:e5:18:5c:c5:e7:19:62 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:43:11.602+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000103275 2023-10-21T04:43:11.602+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken23:e5:18:5c:c5:e7:19:62, dropIndexes: true 2023-10-21T04:43:11.602+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62, shardId :0 2023-10-21T04:43:11.603+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken23_e5_18_5c_c5_e7_19_62, shardId :0 2023-10-21T04:43:11.619+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16823504650100619022 instance id 17495827337792382465 real instance id 4186965058349029465 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:43:11.619+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 228.072µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:11.619+05:30 [Info] LifecycleMgr.DeleteIndex() : index 17917116660857103172 does not exist. 2023-10-21T04:43:11.619+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 56.95µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:11.620+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 7790129686881525591 instance id 2853024886069516112 real instance id 16759912838651900043 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:43:11.620+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 152.839µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:11.628+05:30 [Info] LifecycleMgr.DeleteIndex() : index 668551233566132751 does not exist. 2023-10-21T04:43:11.628+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 67.324µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:11.628+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17327901024118572215 5629943257627822009] 2023-10-21T04:43:11.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9998795986321004 2023-10-21T04:43:11.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17327901024118572215 2023-10-21T04:43:11.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 5629943257627822009 2023-10-21T04:43:11.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17327901024118572215 2023-10-21T04:43:11.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 5629943257627822009 2023-10-21T04:43:11.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17327901024118572215 5629943257627822009], elapsed: 64.575736ms 2023-10-21T04:43:11.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:43:11.693+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:43:11.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:43:11.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shutdown completed 2023-10-21T04:43:11.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:43:11.693+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:43:11.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:43:11.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance destroyed 2023-10-21T04:43:11.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:43:11.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:43:11.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:43:11.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:43:11.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shutdown completed 2023-10-21T04:43:11.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:43:11.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:43:11.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:43:11.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance destroyed 2023-10-21T04:43:11.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:43:11.696+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17327901024118572215 5629943257627822009], elapsed(sec): 0.067292779 2023-10-21T04:43:11.749+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken1e:42:56:40:ff:fb:3c:bc MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc7:2:b9:66:eb:19:6:5 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3285983737505597326 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3786026946915438592 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17840282284522026272 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:11.749+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 12808878c8875c24b4dda4f00a0dfee1 TransferTokenId: ShardToken1e:42:56:40:ff:fb:3c:bc Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:43:11.749+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000102672 2023-10-21T04:43:11.749+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken1e:42:56:40:ff:fb:3c:bc, dropIndexes: false 2023-10-21T04:43:11.749+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken1e_42_56_40_ff_fb_3c_bc, shardId :0 2023-10-21T04:43:11.749+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken1e_42_56_40_ff_fb_3c_bc, shardId :0 2023-10-21T04:43:11.749+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [18374611443167983589 6455624821495495854] 2023-10-21T04:43:12.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9991102588801565 2023-10-21T04:43:12.670+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:12.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 18374611443167983589 2023-10-21T04:43:12.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 6455624821495495854 2023-10-21T04:43:12.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 18374611443167983589 2023-10-21T04:43:12.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [18374611443167983589 6455624821495495854], elapsed: 943.984573ms 2023-10-21T04:43:12.693+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 18374611443167983589 from local file system, err: error destroy shard, shardId 18374611443167983589 not found 2023-10-21T04:43:12.693+05:30 [Error] ShardTransferManager::processDestroyLocalShardMessage Error cleaning-up shardId: 6455624821495495854 from local file system, err: error destroy shard, shardId 6455624821495495854 not found 2023-10-21T04:43:12.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [18374611443167983589 6455624821495495854], elapsed(sec): 0.944073108 2023-10-21T04:43:12.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 6455624821495495854 2023-10-21T04:43:12.708+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken9e:83:7c:94:6a:c6:e7:b5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardToken23:e5:18:5c:c5:e7:19:62 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6609266364331808928 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16122831054735124528 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15172753502274569092 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:12.708+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenc7:2:b9:66:eb:19:6:5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1e:42:56:40:ff:fb:3c:bc Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18145658940511009543 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17789336037235700674 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 190111758852025426 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:12.779+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardToken9e:83:7c:94:6a:c6:e7:b5 2023-10-21T04:43:12.779+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 12808878c8875c24b4dda4f00a0dfee1 TransferTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:43:12.779+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000123129 2023-10-21T04:43:12.779+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokenc7:2:b9:66:eb:19:6:5 2023-10-21T04:43:12.779+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken9e_83_7c_94_6a_c6_e7_b5, shardId :0 2023-10-21T04:43:12.779+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken9e_83_7c_94_6a_c6_e7_b5, shardId :0 2023-10-21T04:43:12.779+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 12808878c8875c24b4dda4f00a0dfee1 TransferTokenId: ShardTokenc7:2:b9:66:eb:19:6:5 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:43:12.779+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 7.2779e-05 2023-10-21T04:43:12.779+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:43:12.779+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:43:12.779+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.538µs 2023-10-21T04:43:12.779+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardTokenc7_2_b9_66_eb_19_6_5, shardId :0 2023-10-21T04:43:12.779+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardTokenc7_2_b9_66_eb_19_6_5, shardId :0 2023-10-21T04:43:12.779+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:43:12.779+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:43:12.779+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:43:12.785+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:43:12.785+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:43:12.785+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:43:12.786+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:43:12.786+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 50.422µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:12.787+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:43:12.787+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:43:12.787+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:43:12.787+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:43:12.787+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:43:12.787+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:43:12.787+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:43:12.788+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 25.766µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:43:12.791+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:43:12.791+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:43:12.791+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 4.234862738s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:43:12.791+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/12808878c8875c24b4dda4f00a0dfee1, rev: [] call. err: 2023-10-21T04:43:12.791+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:43:12.791+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 8] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:43:12.798+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:43:12.798+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:43:12.803+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:43:12.803+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:43:12.805+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:43:12.807+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 9] 2023-10-21T04:43:13.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6988181525922097, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:43:13.685+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:14.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.301695109340278, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:43:14.691+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:15.697+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:16.702+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:17.708+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:18.714+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:19.720+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:20.725+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:21.733+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:22.741+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:23.747+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:24.752+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:25.761+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:26.769+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:27.774+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:28.780+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:29.786+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:30.710+05:30 [Info] fragAutoTuner: FragRatio at 100. MaxFragRatio 100, MaxBandwidth 0. BandwidthUsage 0. AvailDisk 0. TotalUsed 0. BandwidthRatio 1. UsedSpaceRatio 1. CleanerBandwidth 0. Duration 0. 2023-10-21T04:43:30.792+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:31.729+05:30 [Info] memstats {"Alloc":147520784, "TotalAlloc":206950360, "Sys":170429736, "Lookups":0, "Mallocs":1181167,"Frees":645062, "HeapAlloc":147520784, "HeapSys":152764416, "HeapIdle":2318336, "HeapInuse":150446080,"HeapReleased":2285568, "HeapObjects":536105,"MSpanInuse":932160, "MSpanSys": 946560, "StackInuse": 2424832,"GCSys":11869304, "LastGC":1697843477790042875,"PauseTotalNs":1840710, "PauseNs":[], "NumGC":6} 2023-10-21T04:43:31.798+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:32.714+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:43:32.719+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Found 2 tokens. Cleaning up. 2023-10-21T04:43:32.719+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:43:32.719+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken9e:83:7c:94:6a:c6:e7:b5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardToken23:e5:18:5c:c5:e7:19:62 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 6609266364331808928 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 16122831054735124528 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15172753502274569092 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:32.719+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenc7:2:b9:66:eb:19:6:5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 12808878c8875c24b4dda4f00a0dfee1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken1e:42:56:40:ff:fb:3c:bc Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 18145658940511009543 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17789336037235700674 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 190111758852025426 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:43:32.723+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardToken9e:83:7c:94:6a:c6:e7:b5 2023-10-21T04:43:32.723+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 12808878c8875c24b4dda4f00a0dfee1 TransferTokenId: ShardToken9e:83:7c:94:6a:c6:e7:b5 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:43:32.723+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000106293 2023-10-21T04:43:32.723+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokenc7:2:b9:66:eb:19:6:5 2023-10-21T04:43:32.723+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken9e_83_7c_94_6a_c6_e7_b5, shardId :0 2023-10-21T04:43:32.723+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardToken9e_83_7c_94_6a_c6_e7_b5, shardId :0 2023-10-21T04:43:32.723+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 12808878c8875c24b4dda4f00a0dfee1 TransferTokenId: ShardTokenc7:2:b9:66:eb:19:6:5 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:43:32.723+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 6.6739e-05 2023-10-21T04:43:32.723+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardTokenc7_2_b9_66_eb_19_6_5, shardId :0 2023-10-21T04:43:32.723+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/12808878c8875c24b4dda4f00a0dfee1_ShardTokenc7_2_b9_66_eb_19_6_5, shardId :0 2023-10-21T04:43:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:43:32.787+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:43:32.804+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:43:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:43:32.958+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:43:32.978+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:43:33.219+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:43:33.235+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:43:33.809+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:33.890+05:30 [Info] Indexer::ReadMemstats Time Taken 397.007µs 2023-10-21T04:43:34.310+05:30 [Info] janitor: running cleanup. 2023-10-21T04:43:34.310+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[14889401051926000176 7392796230534034390 4532813151040661705 12582561312368048736 12140458060295645238 8504686614261377811 12027054384338714804 14741949230702299196 11365633301282701856 17797267771846799851 16337157693634799600 10606939961365971246 13268712595133993155 12338036531800130039 11276338709617092948 12824340686309573428 12599498416578472779 4138047918192025493 15431181238515168303 9665410041638025825 18046371353728341033 14549775068266281786 4790214984019693894 3674078063353481907 12833670934079048728 8910459001587546151 6985049432225870291 11619165464560280276 573521267164877540 10761885902580101111 13856704970496601958 15945117906390072636 17656144409930763141 8378417662949994532 16706481854123898870 8857269605176042330 1252359852432998131 3381524789655566900 2048006213723315482 12195715103909237684 14292583894719865678 7525085113233084253 18026660730857541767 8669649359784409226 18323460706027642575 4286621850367430051 7147407900589480727 249887442666643345 15968368593314043432 1301336828089581719 16961575949607441659 4987105685507282285 5465835793930010953 4535197604682045916 16162012140118277065 4860549750398470299]} 2023-10-21T04:43:34.813+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:43:36.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9982830457110784, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:43:37.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9976208252003634, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:43:38.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7274132086896563, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:43:39.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.31311282630101583, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:43:42.806+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. err: operation_canceled 2023-10-21T04:43:42.806+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:43:42.806+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:43:42.807+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:43:42.807+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:43:42.808+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:43:42.810+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 9] 2023-10-21T04:43:49.845+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:49.845+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:49.882+05:30 [Info] switched currmeta from 91 -> 95 force true 2023-10-21T04:43:50.030+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:50.031+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:50.033+05:30 [Info] switched currmeta from 95 -> 99 force true 2023-10-21T04:43:50.091+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:50.091+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:50.116+05:30 [Info] switched currmeta from 99 -> 99 force true 2023-10-21T04:43:53.552+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:43:53.558+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:43:53.558+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:43:53.560+05:30 [Info] switched currmeta from 99 -> 99 force true 2023-10-21T04:44:02.724+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:44:03.094+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 244] aggregate storage info: map[bucket_%2:map[kv:2349248 index:4538265] bucket_1:map[kv:2418880 index:5000940]] 2023-10-21T04:44:07.581+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52956 2023-10-21T04:44:07.604+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 20.518141ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:08.043+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:52980 2023-10-21T04:44:08.106+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:08.270+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 217.91795ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:08.771+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:09.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9982023776964335, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:44:10.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9993846221525027, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:44:11.023+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 16483001637761719171 2023-10-21T04:44:11.023+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 136.054µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:11.448+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37936 2023-10-21T04:44:11.502+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37936 2023-10-21T04:44:11.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.999213913542743 2023-10-21T04:44:11.748+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37936 2023-10-21T04:44:12.184+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 16483001637761719171 2023-10-21T04:44:12.184+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 248.45µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:12.486+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:12.486+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:12.502+05:30 [Info] switched currmeta from 99 -> 108 force true 2023-10-21T04:44:12.598+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:12.598+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:12.622+05:30 [Info] switched currmeta from 108 -> 108 force true 2023-10-21T04:44:12.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9995347468227497 2023-10-21T04:44:12.811+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:44:12.811+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 9] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:44:12.812+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. err: operation_canceled 2023-10-21T04:44:12.812+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:44:12.812+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 9] Tasks:[]} 2023-10-21T04:44:12.913+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 9] 2023-10-21T04:44:12.913+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 9] 2023-10-21T04:44:12.914+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:44:12.914+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:44:13.038+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:44:13.089+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52956. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:13.089+05:30 [Info] messageListener.start(): message channel closed. Remove peer 55:ed:5c:93:7a:7f:11:c6 as follower. 2023-10-21T04:44:13.139+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53126 2023-10-21T04:44:13.236+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 93.776778ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:13.336+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:44:13.336+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:44:13.336+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:44:13.397+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53142 2023-10-21T04:44:13.410+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53144 2023-10-21T04:44:13.410+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:44:13.488+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 82.388365ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:13.544+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 128.791555ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:13.544+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:44:13.544+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:44:13.544+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:44:13.572+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53148 2023-10-21T04:44:13.624+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:44:13.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9997261445806624 2023-10-21T04:44:13.704+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 129.069333ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:13.736+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:13.825+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:52980. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:13.825+05:30 [Info] messageListener.start(): message channel closed. Remove peer d9:2c:af:e2:4:87:33:18 as follower. 2023-10-21T04:44:13.856+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:44:13.856+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:44:13.856+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:44:13.958+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 16483001637761719171. No update is performed. 2023-10-21T04:44:13.958+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 122.025µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:13.976+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53148. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:13.976+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5d:6d:4d:54:1a:e8:58:2e as follower. 2023-10-21T04:44:13.985+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:13.988+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:44:13.989+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:14.045+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:14.110+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:44:14.110+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:44:14.110+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:44:14.110+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:44:14.110+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:44:14.110+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:44:14.110+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:44:14.110+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:44:14.110+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:44:14.110+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:44:14.110+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:44420->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:14.111+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:53144->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:14.111+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:49524->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:14.112+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:44826->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:14.112+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53144. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:14.112+05:30 [Info] messageListener.start(): message channel closed. Remove peer 7b:1a:af:83:17:29:6f:0 as follower. 2023-10-21T04:44:14.123+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:14.123+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:14.125+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53126. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:14.125+05:30 [Info] messageListener.start(): message channel closed. Remove peer 18:d9:7b:6d:60:3a:64:b0 as follower. 2023-10-21T04:44:14.133+05:30 [Info] switched currmeta from 108 -> 108 force true 2023-10-21T04:44:14.181+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53142. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:14.181+05:30 [Info] messageListener.start(): message channel closed. Remove peer 86:d6:22:97:8b:f8:78:e1 as follower. 2023-10-21T04:44:14.196+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53184 2023-10-21T04:44:14.205+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:14.213+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 11.165453ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:14.244+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53184. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:14.244+05:30 [Info] messageListener.start(): message channel closed. Remove peer 31:a1:12:39:41:2a:64:49 as follower. 2023-10-21T04:44:14.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9668157098459479 2023-10-21T04:44:14.714+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:15.510+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:15.510+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:15.516+05:30 [Info] switched currmeta from 108 -> 109 force true 2023-10-21T04:44:15.569+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:15.569+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:15.573+05:30 [Info] switched currmeta from 109 -> 110 force true 2023-10-21T04:44:15.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.835079520765783, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:44:15.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:15.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:15.974+05:30 [Info] switched currmeta from 110 -> 111 force true 2023-10-21T04:44:16.015+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:16.015+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:16.019+05:30 [Info] switched currmeta from 111 -> 112 force true 2023-10-21T04:44:16.468+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14297924784489035508:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:44:16.469+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14297924784489035508:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:44:16.471+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14297924784489035508:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:44:16.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6389412396345524, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:44:17.360+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[897640012706409382:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:44:18.636+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:18.641+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:18.641+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:18.643+05:30 [Info] switched currmeta from 112 -> 112 force true 2023-10-21T04:44:19.376+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 7045704526128519604 2023-10-21T04:44:19.376+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 124.797µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:19.443+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37936 2023-10-21T04:44:19.457+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37936 2023-10-21T04:44:19.513+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:37936 2023-10-21T04:44:19.629+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 7045704526128519604 2023-10-21T04:44:19.629+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 234.678µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:19.701+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:19.704+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:19.713+05:30 [Info] switched currmeta from 112 -> 116 force true 2023-10-21T04:44:19.718+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:19.718+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:19.726+05:30 [Info] switched currmeta from 116 -> 117 force true 2023-10-21T04:44:19.752+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:19.752+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:19.768+05:30 [Info] switched currmeta from 117 -> 119 force true 2023-10-21T04:44:19.785+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:19.787+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:19.854+05:30 [Info] switched currmeta from 119 -> 121 force true 2023-10-21T04:44:19.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:19.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:19.988+05:30 [Info] switched currmeta from 121 -> 121 force true 2023-10-21T04:44:20.337+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53374 2023-10-21T04:44:20.365+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.919913ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:20.433+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 7045704526128519604. No update is performed. 2023-10-21T04:44:20.433+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 117.821µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:20.458+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53374. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:20.458+05:30 [Info] messageListener.start(): message channel closed. Remove peer 85:66:c1:2c:87:3b:f8:5e as follower. 2023-10-21T04:44:20.689+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:44:20.689+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:44:20.723+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:44:20.742+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53392 2023-10-21T04:44:20.755+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53394 2023-10-21T04:44:20.766+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:44:20.766+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:44:20.766+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:44:20.779+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:44:20.785+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 21.015936ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:20.786+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 28.731242ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:20.841+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53392. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:20.841+05:30 [Info] messageListener.start(): message channel closed. Remove peer ea:56:e5:4d:96:20:b2:1f as follower. 2023-10-21T04:44:20.843+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:44:20.843+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:44:20.843+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:44:20.853+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:53410 2023-10-21T04:44:20.854+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:44:20.866+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:20.891+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 25.66917ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:20.891+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:44:20.891+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:44:20.891+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:44:20.911+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:44:20.913+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53394. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:20.914+05:30 [Info] messageListener.start(): message channel closed. Remove peer e4:93:4f:72:b7:e1:63:2e as follower. 2023-10-21T04:44:20.929+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:44:20.929+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:44:20.929+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:44:20.929+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:44:20.929+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:44:20.929+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:44:20.929+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:44:20.929+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:44:20.929+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:44:20.929+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:44:20.930+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:44642->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:20.930+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:45108->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:20.930+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:53410->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:20.930+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:49788->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:20.931+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:53410. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:20.931+05:30 [Info] messageListener.start(): message channel closed. Remove peer ff:75:cb:b0:26:22:e5:2d as follower. 2023-10-21T04:44:21.173+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:21.173+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:21.177+05:30 [Info] switched currmeta from 121 -> 123 force true 2023-10-21T04:44:21.186+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:21.186+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:21.202+05:30 [Info] switched currmeta from 123 -> 123 force true 2023-10-21T04:44:21.286+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:21.287+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:21.392+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:21.616+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:21.616+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:21.621+05:30 [Info] switched currmeta from 123 -> 124 force true 2023-10-21T04:44:21.706+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:21.706+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:21.729+05:30 [Info] switched currmeta from 124 -> 125 force true 2023-10-21T04:44:22.861+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12134217367291736519:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:44:22.861+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12134217367291736519:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:44:22.861+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12134217367291736519:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:44:22.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10682617070590310048:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:44:22.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10682617070590310048:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:44:23.654+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:23.659+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:23.659+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:23.661+05:30 [Info] switched currmeta from 125 -> 125 force true 2023-10-21T04:44:30.711+05:30 [Info] fragAutoTuner: FragRatio at 100. MaxFragRatio 100, MaxBandwidth 0. BandwidthUsage 0. AvailDisk 0. TotalUsed 0. BandwidthRatio 1. UsedSpaceRatio 1. CleanerBandwidth 0. Duration 0. 2023-10-21T04:44:31.729+05:30 [Info] memstats {"Alloc":126786384, "TotalAlloc":264636208, "Sys":187665704, "Lookups":0, "Mallocs":1735246,"Frees":1378147, "HeapAlloc":126786384, "HeapSys":169508864, "HeapIdle":37781504, "HeapInuse":131727360,"HeapReleased":0, "HeapObjects":357099,"MSpanInuse":719360, "MSpanSys": 1191360, "StackInuse": 2457600,"GCSys":12094632, "LastGC":1697843569976064831,"PauseTotalNs":2140939, "PauseNs":[300229], "NumGC":7} 2023-10-21T04:44:32.726+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:44:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:44:32.787+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:44:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:44:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:44:33.007+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:44:33.026+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:44:33.279+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:44:33.294+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:44:34.310+05:30 [Info] janitor: running cleanup. 2023-10-21T04:44:34.310+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[18026660730857541767 14889401051926000176 11365633301282701856 3381524789655566900 16961575949607441659 573521267164877540 1252359852432998131 16162012140118277065 14292583894719865678 4532813151040661705 4790214984019693894 15945117906390072636 4138047918192025493 11619165464560280276 17797267771846799851 12833670934079048728 7392796230534034390 13856704970496601958 6985049432225870291 4987105685507282285 10761885902580101111 15973521466438086342 15431181238515168303 12140458060295645238 10606939961365971246 12582561312368048736 8378417662949994532 7147407900589480727 249887442666643345 12195715103909237684 13268712595133993155 18046371353728341033 9665410041638025825 8669649359784409226 16706481854123898870 15968368593314043432 12027054384338714804 8504686614261377811 18323460706027642575 8910459001587546151 5465835793930010953 2048006213723315482 8857269605176042330 4286621850367430051 12599498416578472779 3674078063353481907 17656144409930763141 1301336828089581719 4535197604682045916 16337157693634799600 14549775068266281786 12338036531800130039 4860549750398470299 14741949230702299196 11276338709617092948 7525085113233084253 12824340686309573428 17917116660857103172]} 2023-10-21T04:44:34.390+05:30 [Info] Indexer::ReadMemstats Time Taken 391.491µs 2023-10-21T04:44:36.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996072067927472, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:44:37.178+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:44:37.448+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:37.448+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:37.461+05:30 [Info] switched currmeta from 125 -> 125 force true 2023-10-21T04:44:37.667+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996161580834135, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:44:37.682+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:37.734+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:37.734+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:37.735+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:44:37.735+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:44:37.737+05:30 [Info] switched currmeta from 125 -> 125 force true 2023-10-21T04:44:38.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9792909726656389 2023-10-21T04:44:38.897+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:44:39.026+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:39.026+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:39.030+05:30 [Info] switched currmeta from 125 -> 125 force true 2023-10-21T04:44:39.671+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8664714029046264, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:44:40.314+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {04c71e696cccf09893ec2b2a16b010ff [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:44:40.320+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.82µs 2023-10-21T04:44:40.320+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:44:40.320+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:44:40.320+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:44:40.323+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:44:40.323+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:44:40.323+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.585123ms 2023-10-21T04:44:40.323+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:44:40.323+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:44:40.323+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 10] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:44:40.323+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.083µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:40.323+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 9] call. taskList: &{Rev:[0 0 0 0 0 0 0 10] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/04c71e696cccf09893ec2b2a16b010ff Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:04c71e696cccf09893ec2b2a16b010ff]}]} 2023-10-21T04:44:40.324+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 10] 2023-10-21T04:44:40.325+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 10] 2023-10-21T04:44:40.340+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {04c71e696cccf09893ec2b2a16b010ff [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:44:40.344+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:44:40.379+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"04c71e696cccf09893ec2b2a16b010ff","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:44:40.382+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{4325b30e7f9620a998388078247335a8 04c71e696cccf09893ec2b2a16b010ff Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:44:40.384+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 04c71e696cccf09893ec2b2a16b010ff Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:44:40.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7510466986312806, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:44:41.397+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:44:41.405+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:44:41.405+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:44:41.410+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:44:41.410+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{4325b30e7f9620a998388078247335a8 04c71e696cccf09893ec2b2a16b010ff Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:44:41.410+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:44:41.410+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.069647717s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:44:41.410+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{4325b30e7f9620a998388078247335a8 04c71e696cccf09893ec2b2a16b010ff Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:44:41.410+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 10] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/04c71e696cccf09893ec2b2a16b010ff Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:04c71e696cccf09893ec2b2a16b010ff]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/04c71e696cccf09893ec2b2a16b010ff Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:04c71e696cccf09893ec2b2a16b010ff]}]} 2023-10-21T04:44:41.410+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:44:41.411+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:44:41.412+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:44:41.412+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/04c71e696cccf09893ec2b2a16b010ff Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:04c71e696cccf09893ec2b2a16b010ff]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/04c71e696cccf09893ec2b2a16b010ff Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:04c71e696cccf09893ec2b2a16b010ff]}]} 2023-10-21T04:44:41.413+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:44:41.413+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:44:41.417+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 11] 2023-10-21T04:44:41.418+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 11] 2023-10-21T04:44:41.506+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{d7ff2bfd429b21d9c24d822bc13e3acc d7ff2bfd429b21d9c24d822bc13e3acc plasma map[] [{37 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{9316375249999768625 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{3751526798537645569 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 16483001637761719171 [{897640012706409382 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{18137927630988528516 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563] []}] 1 0 0 1 false plasma 0}]}]} {37 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{12606056968936758026 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{8303154871847919944 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 7045704526128519604 [{12134217367291736519 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{17465522551378543576 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 16483001637761719171 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {9b27839d59b06374125d80a2b12a21b6 9b27839d59b06374125d80a2b12a21b6 plasma map[] [{6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{10158339249566909515 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589] []}] 1 0 0 0 false plasma 0}]}]} {37 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{16759912838651900043 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{4186965058349029465 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 16483001637761719171 [{14297924784489035508 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{544693394183450628 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215] []}] 1 0 0 0 false plasma 0}]}]} {37 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{4397940545762329850 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{12396348969790516872 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 7045704526128519604 [{10682617070590310048 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16483001637761719171 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {4325b30e7f9620a998388078247335a8 4325b30e7f9620a998388078247335a8 plasma map[] [] [] 0 0 0 false} {ba6304988a6335d65281381d6cfc5e32 ba6304988a6335d65281381d6cfc5e32 plasma map[] [{3 bucket_1 _system _query []}] [] 0 0 0 false}] map[]} 2023-10-21T04:44:41.555+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:44:41.561+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:44:41.561+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:44:41.561+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:44:41.561+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:44:41.561+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:44:41.606+05:30 [Info] Planner::processDeleteToken: processing 58 tokens for node 127.0.0.1:9001 2023-10-21T04:44:41.606+05:30 [Info] Planner::processDeleteToken: processing 58 tokens for node 127.0.0.1:9004 2023-10-21T04:44:41.606+05:30 [Info] Planner::processDeleteToken: processing 58 tokens for node 127.0.0.1:9002 2023-10-21T04:44:41.606+05:30 [Info] Planner::processDeleteToken: processing 58 tokens for node 127.0.0.1:9003 2023-10-21T04:44:41.612+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:44:41.615+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:44:41.615+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:44:41.615+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:44:41.615+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:44:41.671+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:44:41.671+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:44:41.671+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:44:41.671+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 7045704526128519604:10682617070590310048:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 5277316367626246544:10158339249566909515:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.671+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 16483001637761719171:14297924784489035508:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 668551233566132751:544693394183450628:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:44:41.672+05:30 [Info] Moving index 5277316367626246544:17465522551378543576:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 16483001637761719171:897640012706409382:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 668551233566132751:18137927630988528516:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.672+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Moving index 7045704526128519604:12134217367291736519:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:44:41.673+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 17716118 Units 0 2023-10-21T04:44:41.673+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17714212 Units 0 2023-10-21T04:44:41.673+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 287490 Units 1 Tenants 0 2023-10-21T04:44:41.673+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 285176 Units 1 Tenants 0 2023-10-21T04:44:41.673+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:44:41.673+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:44:41.673+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:44:41.673+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:44:41.673+05:30 [Info] ---------------------------------------- 2023-10-21T04:44:41.673+05:30 [Info] 2023-10-21T04:44:41.673+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:ba6304988a6335d65281381d6cfc5e32, useLiveData:false 2023-10-21T04:44:41.673+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:44:41.673+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:44:41.673+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.673+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:17465522551378543576, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501], shardIdsInDefn: map[] 2023-10-21T04:44:41.673+05:30 [Info] Index total memory:157293156 (150.006M), mem:5850 (5.71289K), overhead:157287306 (150.001M), min:157293156 (150.006M), data:5850 (5.71289K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.673+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.673+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.673+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.673+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.673+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.673+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.673+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.673+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.673+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.673+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.673+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.673+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.673+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.673+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.673+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.673+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.673+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.674+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.674+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.674+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.674+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16483001637761719171, instId:897640012706409382, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:18137927630988528516, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157293177 (150.006M), mem:5868 (5.73047K), overhead:157287309 (150.001M), min:157293177 (150.006M), data:5868 (5.73047K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.675+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.675+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.675+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.675+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:7045704526128519604, instId:12134217367291736519, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:44:41.676+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.676+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.676+05:30 [Info] 2023-10-21T04:44:41.676+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:4325b30e7f9620a998388078247335a8, useLiveData:false 2023-10-21T04:44:41.676+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:44:41.676+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:44:41.676+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.676+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.677+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.677+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.677+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.677+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.677+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.677+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.677+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.677+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.677+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:7045704526128519604, instId:10682617070590310048, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:10158339249566909515, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157293336 (150.007M), mem:6006 (5.86523K), overhead:157287330 (150.001M), min:157293336 (150.007M), data:6006 (5.86523K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.678+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.678+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.678+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.678+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:16483001637761719171, instId:14297924784489035508, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.679+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.679+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.679+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:544693394183450628, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215], shardIdsInDefn: map[] 2023-10-21T04:44:41.679+05:30 [Info] Index total memory:157293371 (150.007M), mem:6036 (5.89453K), overhead:157287335 (150.001M), min:157293371 (150.007M), data:6036 (5.89453K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:44:41.680+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:44:41.680+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:44:41.680+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:44:41.680+05:30 [Info] **************************************** 2023-10-21T04:44:41.680+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 287490 Units 1 Tenants 0 2023-10-21T04:44:41.681+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 285176 Units 1 Tenants 0 2023-10-21T04:44:41.681+05:30 [Info] Generating Shard Transfer Token (ShardToken19:fe:77:b2:bc:19:17:a2) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:4b:4c:c2:98:2a:73:e7 Shards: [10838601320854145501 11711860383628641160] Destination: Region: InstId: 5484913305943270311 RealInstId: 12606056968936758026 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 321401340639350282 RealInstId: 8303154871847919944 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 12134217367291736519 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 12134217367291736519 Defn: DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17465522551378543576 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:44:41.682+05:30 [Info] Generating Shard Transfer Token (ShardTokenf5:e6:93:c0:4d:37:29:98) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Shards: [11114961608238406563 5748926106587204814] Destination: Region: InstId: 897640012706409382 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 897640012706409382 Defn: DefnId: 16483001637761719171 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 2830391075920898418 RealInstId: 9316375249999768625 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 9316375249999768625 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 17398831371326996368 RealInstId: 3751526798537645569 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3751526798537645569 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 18137927630988528516 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 18137927630988528516 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:44:41.682+05:30 [Info] Generating Shard Transfer Token (ShardToken11:3e:79:72:8e:79:e:f5) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: Region: InstId: 14637134422972922160 RealInstId: 4186965058349029465 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 14297924784489035508 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14297924784489035508 Defn: DefnId: 16483001637761719171 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10357771762526486340 RealInstId: 16759912838651900043 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 544693394183450628 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:44:41.683+05:30 [Info] Generating Shard Transfer Token (ShardTokene3:4b:4c:c2:98:2a:73:e7) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: Region: InstId: 10682617070590310048 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10682617070590310048 Defn: DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 6403098372720496675 RealInstId: 4397940545762329850 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 3335851955840345084 RealInstId: 12396348969790516872 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10158339249566909515 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:44:41.683+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:44:41.683+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:44:41.687+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken19:fe:77:b2:bc:19:17:a2 2023-10-21T04:44:41.689+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenf5:e6:93:c0:4d:37:29:98 2023-10-21T04:44:41.697+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken11:3e:79:72:8e:79:e:f5 2023-10-21T04:44:41.766+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokene3:4b:4c:c2:98:2a:73:e7 2023-10-21T04:44:41.766+05:30 [Info] ShardRebalancer::observeRebalance &{4325b30e7f9620a998388078247335a8 04c71e696cccf09893ec2b2a16b010ff Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:44:41.775+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:4b:4c:c2:98:2a:73:e7 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10682617070590310048 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6403098372720496675 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3335851955840345084 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.776+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:3e:79:72:8e:79:e:f5 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14637134422972922160 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14297924784489035508 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10357771762526486340 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.786+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:e6:93:c0:4d:37:29:98 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 897640012706409382 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2830391075920898418 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17398831371326996368 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.805+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken19:fe:77:b2:bc:19:17:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:4b:4c:c2:98:2a:73:e7 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5484913305943270311 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 321401340639350282 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12134217367291736519 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.817+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"04c71e696cccf09893ec2b2a16b010ff","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:44:41.817+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:44:41.859+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken19:fe:77:b2:bc:19:17:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:4b:4c:c2:98:2a:73:e7 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5484913305943270311 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 321401340639350282 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12134217367291736519 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.894+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:41.897+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:3e:79:72:8e:79:e:f5 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14637134422972922160 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14297924784489035508 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10357771762526486340 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.897+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:44:41.897+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:44:41.897+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:44:41.898+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 65.733µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:41.938+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:e6:93:c0:4d:37:29:98 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 897640012706409382 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2830391075920898418 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17398831371326996368 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.939+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:4b:4c:c2:98:2a:73:e7 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10682617070590310048 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6403098372720496675 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3335851955840345084 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.939+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:44:41.939+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:44:41.940+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:44:41.940+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.084µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:41.972+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:3e:79:72:8e:79:e:f5 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14637134422972922160 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14297924784489035508 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10357771762526486340 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.982+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:41.982+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:4b:4c:c2:98:2a:73:e7 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10682617070590310048 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6403098372720496675 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3335851955840345084 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:41.987+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:42.053+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:e6:93:c0:4d:37:29:98 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 897640012706409382 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2830391075920898418 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17398831371326996368 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.061+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:42.062+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:44:42.074+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 04c71e696cccf09893ec2b2a16b010ff Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:44:42.075+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 1 2023-10-21T04:44:42.144+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[9b27839d59b06374125d80a2b12a21b6:ShardTokene3:4b:4c:c2:98:2a:73:e7 d7ff2bfd429b21d9c24d822bc13e3acc:ShardToken19:fe:77:b2:bc:19:17:a2], currActiveTransfers: map[9b27839d59b06374125d80a2b12a21b6:1 d7ff2bfd429b21d9c24d822bc13e3acc:1] 2023-10-21T04:44:42.144+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"04c71e696cccf09893ec2b2a16b010ff","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:44:42.144+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:44:42.144+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:44:42.145+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:44:42.145+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:44:42.145+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:44:42.145+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken19:fe:77:b2:bc:19:17:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:4b:4c:c2:98:2a:73:e7 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5484913305943270311 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 321401340639350282 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12134217367291736519 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.146+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:4b:4c:c2:98:2a:73:e7 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10682617070590310048 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6403098372720496675 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3335851955840345084 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.146+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 17.588µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:42.253+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken19:fe:77:b2:bc:19:17:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:4b:4c:c2:98:2a:73:e7 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5484913305943270311 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 321401340639350282 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12134217367291736519 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.253+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:44:42.464+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:42.464+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:42.480+05:30 [Info] switched currmeta from 125 -> 126 force true 2023-10-21T04:44:42.493+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken19:fe:77:b2:bc:19:17:a2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene3:4b:4c:c2:98:2a:73:e7 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 5484913305943270311 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 321401340639350282 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12134217367291736519 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.493+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:44:42.496+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:4b:4c:c2:98:2a:73:e7 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10682617070590310048 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6403098372720496675 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3335851955840345084 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.496+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:44:42.535+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[9b27839d59b06374125d80a2b12a21b6:ShardToken11:3e:79:72:8e:79:e:f5 d7ff2bfd429b21d9c24d822bc13e3acc:ShardTokenf5:e6:93:c0:4d:37:29:98], currActiveTransfers: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:1] 2023-10-21T04:44:42.542+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:44:42.542+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:44:42.542+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:44:42.542+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 33.431µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:42.543+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[6455624821495495854:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/6455624821495495854/shards/shard6455624821495495854 18374611443167983589:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/18374611443167983589/shards/shard18374611443167983589] 04c71e696cccf09893ec2b2a16b010ff ShardTokene3:4b:4c:c2:98:2a:73:e7 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00712f020 0xc00712f080 map[] 0xc006a00e40 0xc0083aeae0 false} 2023-10-21T04:44:42.543+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/6455624821495495854/shards/shard6455624821495495854 2023-10-21T04:44:42.544+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/18374611443167983589/shards/shard18374611443167983589 2023-10-21T04:44:42.546+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/6455624821495495854/shards/shard6455624821495495854/data: super block snapshot head offset :0 tail offset:679936 sbgen :119 2023-10-21T04:44:42.546+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:e6:93:c0:4d:37:29:98 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 897640012706409382 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2830391075920898418 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17398831371326996368 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.547+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/6455624821495495854/shards/shard6455624821495495854/data/recovery: super block snapshot head offset :0 tail offset:573440 sbgen :119 2023-10-21T04:44:42.548+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/18374611443167983589/shards/shard18374611443167983589/data: super block snapshot head offset :0 tail offset:831488 sbgen :130 2023-10-21T04:44:42.549+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/18374611443167983589/shards/shard18374611443167983589/data/recovery: super block snapshot head offset :0 tail offset:745472 sbgen :130 2023-10-21T04:44:42.552+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/18374611443167983589/shards/shard18374611443167983589: metadata saved successfully 2023-10-21T04:44:42.553+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) added 2023-10-21T04:44:42.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:44:42.553+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 159624400 totalBytes: 1596244 numShards: 2 duration: 10.649773ms 2023-10-21T04:44:42.554+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/18374611443167983589/shards/shard18374611443167983589, newshardId 18374611443167983589, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, shardsize 1596244, duration 9.871065ms 2023-10-21T04:44:42.554+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 18374611443167983589, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, err: , elapsed(sec): 0.010869313 2023-10-21T04:44:42.554+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 18374611443167983589 bytesWritten: 1596244, totalBytes: 1596244, transferRate: 1.7736044444444445e+08 2023-10-21T04:44:42.554+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/6455624821495495854/shards/shard6455624821495495854: metadata saved successfully 2023-10-21T04:44:42.555+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) added 2023-10-21T04:44:42.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:44:42.555+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7/6455624821495495854/shards/shard6455624821495495854, newshardId 6455624821495495854, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, shardsize 1272497, duration 11.999399ms 2023-10-21T04:44:42.555+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 6455624821495495854, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, err: , elapsed(sec): 0.012430457 2023-10-21T04:44:42.556+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 04c71e696cccf09893ec2b2a16b010ff, transferId: ShardTokene3:4b:4c:c2:98:2a:73:e7 2023-10-21T04:44:42.556+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 04c71e696cccf09893ec2b2a16b010ff, transferId: ShardTokene3:4b:4c:c2:98:2a:73:e7, elapsed(sec): 0.000323247 2023-10-21T04:44:42.556+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[6455624821495495854: 18374611443167983589:], shardPaths: map[6455624821495495854:/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 18374611443167983589:/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.013459673 2023-10-21T04:44:42.556+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 6455624821495495854 bytesWritten: 1272497, totalBytes: 1272497, transferRate: 1.1568154545454545e+08 2023-10-21T04:44:42.556+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [18374611443167983589 6455624821495495854], ttid: ShardTokene3:4b:4c:c2:98:2a:73:e7, elapsed(sec): 0.013965959 2023-10-21T04:44:42.558+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:3e:79:72:8e:79:e:f5 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14637134422972922160 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14297924784489035508 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10357771762526486340 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.568+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:42.592+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene3:4b:4c:c2:98:2a:73:e7 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10682617070590310048 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6403098372720496675 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3335851955840345084 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.592+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:1], batchSize: 1 2023-10-21T04:44:42.592+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardTokene3:4b:4c:c2:98:2a:73:e7 2023-10-21T04:44:42.592+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[6455624821495495854:true 18374611443167983589:true] 2023-10-21T04:44:42.592+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.538µs 2023-10-21T04:44:42.592+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [18374611443167983589 6455624821495495854], isLockedForRecovery: true 2023-10-21T04:44:42.592+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [18374611443167983589 6455624821495495854], errMap: map[6455624821495495854: 18374611443167983589:], elapsed: 13.185µs 2023-10-21T04:44:42.748+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 7045704526128519604, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:44:42.748+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 7045704526128519604, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:44:42.749+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:42.749+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:42.779+05:30 [Info] switched currmeta from 126 -> 127 force true 2023-10-21T04:44:42.818+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10682617070590310048,realInstId: 0, indexDefn DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:44:42.819+05:30 [Info] Indexer::handleRecoverIndex InstId: 10682617070590310048 Defn: DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:44:42.819+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 339.9µs 2023-10-21T04:44:42.819+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 384.192µs 2023-10-21T04:44:42.820+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10682617070590310048 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:42.821+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10682617070590310048 PartitionId: 0 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:42.821+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': added 2023-10-21T04:44:42.822+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10682617070590310048, indexDefn DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:44:42.822+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 74.171183ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:42.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCleaner initialized 2023-10-21T04:44:42.826+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:44:42.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:42.826+05:30 [Info] FBMemTuner: Started AutoTuner LSSFlushBuffer 2023-10-21T04:44:42.827+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCleaner initialized for recovery 2023-10-21T04:44:42.827+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:44:42.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery 2023-10-21T04:44:42.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:42.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true] 2023-10-21T04:44:42.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[573440] Data log Head [0] Tail [679936] 2023-10-21T04:44:42.827+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [573440] 2023-10-21T04:44:42.829+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCleaner initialized 2023-10-21T04:44:42.829+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:44:42.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:42.830+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.831+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:44:42.831+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:44:42.830+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCleaner initialized for recovery 2023-10-21T04:44:42.833+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:44:42.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery 2023-10-21T04:44:42.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:42.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true] 2023-10-21T04:44:42.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[745472] Data log Head [0] Tail [831488] 2023-10-21T04:44:42.834+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [745472] 2023-10-21T04:44:42.836+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.836+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:44:42.836+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.836+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:44:42.836+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.837+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:44:42.837+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.837+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:44:42.837+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.837+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:44:42.838+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.838+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:44:42.838+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.838+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:44:42.839+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.839+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:42.839+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.839+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:42.840+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.840+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:42.840+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.840+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:42.840+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.840+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:42.841+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.841+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:42.841+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.841+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:42.841+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.842+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:42.843+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.843+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex 2023-10-21T04:44:42.844+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : replayOffset: 668588 (RP:668588) (CP:655360) 2023-10-21T04:44:42.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [668588] took [16.856703ms] 2023-10-21T04:44:42.845+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.845+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812420 controller mem 2202 2023-10-21T04:44:42.845+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.845+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812420 controller mem 2202 2023-10-21T04:44:42.845+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1056088, resident 1, memSz 68350, memSzIndex 753, numRecords 1000 2023-10-21T04:44:42.846+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812420 controller mem 2202 2023-10-21T04:44:42.846+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.847+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812376 controller mem 2202 2023-10-21T04:44:42.847+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.847+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812376 controller mem 2202 2023-10-21T04:44:42.847+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.847+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812376 controller mem 2202 2023-10-21T04:44:42.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812376 controller mem 2202 2023-10-21T04:44:42.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812376 controller mem 2202 2023-10-21T04:44:42.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812332 controller mem 2202 2023-10-21T04:44:42.849+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.849+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812332 controller mem 2202 2023-10-21T04:44:42.853+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.854+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.854+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812288 controller mem 2202 2023-10-21T04:44:42.854+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.854+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812244 controller mem 2202 2023-10-21T04:44:42.855+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.855+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812200 controller mem 2202 2023-10-21T04:44:42.855+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.855+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812156 controller mem 2202 2023-10-21T04:44:42.855+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.856+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812156 controller mem 2202 2023-10-21T04:44:42.856+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.856+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 26812112 controller mem 2202 2023-10-21T04:44:42.859+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27860644 controller mem 2202 2023-10-21T04:44:42.861+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [679936] replayOffset [668588] 2023-10-21T04:44:42.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [33.826437ms] 2023-10-21T04:44:42.861+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.861+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.861+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.861+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.861+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:44:42.862+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.862+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:44:42.862+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.862+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:44:42.862+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.862+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:44:42.862+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.862+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:44:42.862+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.863+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:44:42.863+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.863+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:44:42.863+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.863+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:42.863+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.863+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:42.863+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.863+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:42.864+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.864+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:42.865+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.865+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:42.865+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.865+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:42.865+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.866+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:42.866+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.866+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:42.868+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:42.868+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex 2023-10-21T04:44:42.869+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : replayOffset: 823296 (RP:815104) (CP:823296) 2023-10-21T04:44:42.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [823296] took [34.876616ms] 2023-10-21T04:44:42.869+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.869+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699822 controller mem 3644 2023-10-21T04:44:42.869+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.869+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699822 controller mem 3644 2023-10-21T04:44:42.869+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.870+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699822 controller mem 3644 2023-10-21T04:44:42.870+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.870+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699778 controller mem 3644 2023-10-21T04:44:42.870+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.870+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699734 controller mem 3644 2023-10-21T04:44:42.870+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.871+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699734 controller mem 3644 2023-10-21T04:44:42.871+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1055476, resident 1, memSz 1191, memSzIndex 0, numRecords 10 2023-10-21T04:44:42.871+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699734 controller mem 3644 2023-10-21T04:44:42.871+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.871+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699690 controller mem 3644 2023-10-21T04:44:42.871+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.871+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699646 controller mem 3644 2023-10-21T04:44:42.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699602 controller mem 3644 2023-10-21T04:44:42.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699602 controller mem 3644 2023-10-21T04:44:42.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699602 controller mem 3644 2023-10-21T04:44:42.872+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1055964, resident 1, memSz 70566, memSzIndex 369, numRecords 1000 2023-10-21T04:44:42.873+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699602 controller mem 3644 2023-10-21T04:44:42.873+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.873+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699558 controller mem 3644 2023-10-21T04:44:42.873+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.873+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699514 controller mem 3644 2023-10-21T04:44:42.874+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.874+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699470 controller mem 3644 2023-10-21T04:44:42.874+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.874+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699470 controller mem 3644 2023-10-21T04:44:42.874+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:44:42.874+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699470 controller mem 3644 2023-10-21T04:44:42.874+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [831488] replayOffset [823296] 2023-10-21T04:44:42.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [41.040813ms] 2023-10-21T04:44:42.875+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.875+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.875+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.876+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.876+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.876+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.877+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.877+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true]. Built [17] plasmas, took [34.564349ms] 2023-10-21T04:44:42.877+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.877+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.881+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.882+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.882+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.882+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.882+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.882+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.883+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.883+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:44:42.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true]. Built [18] plasmas, took [49.549432ms] 2023-10-21T04:44:42.883+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1055964 memSz 866 2023-10-21T04:44:42.883+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 15 count: 1000 metaSz: 2505 rp [1] sn: 16 count: 1000 metaSz: 2513 2023-10-21T04:44:42.883+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 105256 recovery UsedSpace 22524 2023-10-21T04:44:42.877+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1056088 memSz 1690 2023-10-21T04:44:42.888+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 15 count: 1000 metaSz: 2505 rp [1] sn: 16 count: 1000 metaSz: 2513 2023-10-21T04:44:42.889+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 94064 recovery UsedSpace 24194 2023-10-21T04:44:42.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex 2023-10-21T04:44:42.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:42.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:42.910+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper started 2023-10-21T04:44:42.923+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex 2023-10-21T04:44:42.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 20 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex 2023-10-21T04:44:42.923+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:42.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:42.924+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex 2023-10-21T04:44:42.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex started 2023-10-21T04:44:42.925+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:42.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex started 2023-10-21T04:44:42.925+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10682617070590310048 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:44:42.926+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:42.926+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:42.926+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:42.939+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:42.939+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:42.944+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:42.954+05:30 [Info] switched currmeta from 127 -> 128 force true 2023-10-21T04:44:42.956+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10682617070590310048 PartitionId 0 Warmup took 135.110631ms 2023-10-21T04:44:42.957+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10682617070590310048 partitionId 0 WriterThreads 4 2023-10-21T04:44:42.957+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10682617070590310048 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index Index: 10682617070590310048 Partition: 0 2023-10-21T04:44:42.957+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 10682617070590310048, shardIdMap: map[0:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:44:42.957+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 10682617070590310048, partnShardIdMap: map[0:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:44:42.957+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10682617070590310048 Defn: DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:44:42.973+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10682617070590310048 2023-10-21T04:44:42.974+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10682617070590310048 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:44:42.974+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:44:42.974+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:44:42.974+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10682617070590310048 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:44:42.975+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 10682617070590310048 Defn: DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:44:42.975+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10682617070590310048 Partitions [0] 2023-10-21T04:44:42.977+05:30 [Info] StorageMgr::openSnapshot IndexInst:10682617070590310048 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:44:42.992+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 18.523365ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:42.994+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 94.56µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:42.997+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:e6:93:c0:4d:37:29:98 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 897640012706409382 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2830391075920898418 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17398831371326996368 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:42.997+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:44:43.035+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:3e:79:72:8e:79:e:f5 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14637134422972922160 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14297924784489035508 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10357771762526486340 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:43.035+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:44:43.035+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:44:43.035+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:44:43.035+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:44:43.036+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.172µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:43.036+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[5629943257627822009:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/5629943257627822009/shards/shard5629943257627822009 17327901024118572215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/17327901024118572215/shards/shard17327901024118572215] 04c71e696cccf09893ec2b2a16b010ff ShardToken11:3e:79:72:8e:79:e:f5 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00712f020 0xc00712f080 map[] 0xc00807d8c0 0xc0006d3680 false} 2023-10-21T04:44:43.036+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/5629943257627822009/shards/shard5629943257627822009 2023-10-21T04:44:43.036+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/17327901024118572215/shards/shard17327901024118572215 2023-10-21T04:44:43.040+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/17327901024118572215/shards/shard17327901024118572215/data: super block snapshot head offset :0 tail offset:831488 sbgen :130 2023-10-21T04:44:43.040+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/17327901024118572215/shards/shard17327901024118572215/data/recovery: super block snapshot head offset :0 tail offset:745472 sbgen :130 2023-10-21T04:44:43.044+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/17327901024118572215/shards/shard17327901024118572215: metadata saved successfully 2023-10-21T04:44:43.045+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/5629943257627822009/shards/shard5629943257627822009/data: super block snapshot head offset :0 tail offset:688128 sbgen :119 2023-10-21T04:44:43.045+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/5629943257627822009/shards/shard5629943257627822009/data/recovery: super block snapshot head offset :0 tail offset:585728 sbgen :119 2023-10-21T04:44:43.045+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) added 2023-10-21T04:44:43.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:44:43.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/17327901024118572215/shards/shard17327901024118572215, newshardId 17327901024118572215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, shardsize 1596203, duration 9.911592ms 2023-10-21T04:44:43.046+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17327901024118572215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, err: , elapsed(sec): 0.010420822 2023-10-21T04:44:43.046+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17327901024118572215 bytesWritten: 1596203, totalBytes: 1596203, transferRate: 1.773558888888889e+08 2023-10-21T04:44:43.054+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/5629943257627822009/shards/shard5629943257627822009: metadata saved successfully 2023-10-21T04:44:43.055+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) added 2023-10-21T04:44:43.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:44:43.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5/5629943257627822009/shards/shard5629943257627822009, newshardId 5629943257627822009, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, shardsize 1292942, duration 25.901043ms 2023-10-21T04:44:43.062+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 5629943257627822009, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, err: , elapsed(sec): 0.02651304 2023-10-21T04:44:43.062+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 5629943257627822009 bytesWritten: 1292942, totalBytes: 1292942, transferRate: 5.171768e+07 2023-10-21T04:44:43.063+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 04c71e696cccf09893ec2b2a16b010ff, transferId: ShardToken11:3e:79:72:8e:79:e:f5 2023-10-21T04:44:43.063+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 04c71e696cccf09893ec2b2a16b010ff, transferId: ShardToken11:3e:79:72:8e:79:e:f5, elapsed(sec): 0.000608927 2023-10-21T04:44:43.063+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[5629943257627822009: 17327901024118572215:], shardPaths: map[5629943257627822009:/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 17327901024118572215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.027596353 2023-10-21T04:44:43.064+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17327901024118572215 5629943257627822009], ttid: ShardToken11:3e:79:72:8e:79:e:f5, elapsed(sec): 0.028152825 2023-10-21T04:44:43.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:43.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:43.131+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:43.155+05:30 [Info] switched currmeta from 128 -> 129 force true 2023-10-21T04:44:43.162+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:3e:79:72:8e:79:e:f5 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14637134422972922160 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14297924784489035508 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10357771762526486340 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:43.167+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:44:43.233+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:e6:93:c0:4d:37:29:98 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 897640012706409382 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2830391075920898418 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17398831371326996368 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:43.233+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:44:43.487+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:43.487+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:43.492+05:30 [Info] switched currmeta from 129 -> 130 force true 2023-10-21T04:44:43.527+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:43.527+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:43.533+05:30 [Info] switched currmeta from 130 -> 131 force true 2023-10-21T04:44:43.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9853676253046795, throttleDelayMs (new, old, change): (4, 0, 4) 2023-10-21T04:44:43.679+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno cache..., established connections: map[172.31.5.112:43430:172.31.5.112:12000] 2023-10-21T04:44:43.829+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:43.838+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:43.838+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:43.841+05:30 [Info] switched currmeta from 131 -> 131 force true 2023-10-21T04:44:43.947+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[10682617070590310048:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:44:43.948+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:44:43.948+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:44:43.971+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4397940545762329850,realInstId: 0, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:43.975+05:30 [Info] Indexer::handleRecoverIndex InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:44:43.977+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 2.775476ms 2023-10-21T04:44:43.981+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 6.941482ms 2023-10-21T04:44:43.982+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4397940545762329850, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:44:43.982+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 33.951504ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:43.980+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:43.988+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 4 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:43.992+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:43.993+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:43.994+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 19 count: 0 metaSz: 2476 rp [1] sn: 20 count: 0 metaSz: 2477 rp [2] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:43.996+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 43950 recovery UsedSpace 62406 2023-10-21T04:44:43.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:43.998+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:43.998+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 20 count: 0 metaSz: 2477 rp [1] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:43.998+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 51275 recovery UsedSpace 70620 2023-10-21T04:44:44.003+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:44.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:44:44.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.004+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:44:44.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex started 2023-10-21T04:44:44.004+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.004+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:44:44.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.013+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:44:44.013+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex started 2023-10-21T04:44:44.013+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.013+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.013+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:44:44.016+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43492:172.31.5.112:12000] 2023-10-21T04:44:44.018+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:44.019+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:44.023+05:30 [Info] switched currmeta from 131 -> 132 force true 2023-10-21T04:44:44.028+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Warmup took 39.527145ms 2023-10-21T04:44:44.028+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 4 WriterThreads 1 2023-10-21T04:44:44.028+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Stopped Billing WUs 2023-10-21T04:44:44.028+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index Index: 4397940545762329850 Partition: 4 2023-10-21T04:44:44.028+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:44.028+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 5 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:44.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:44.029+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.029+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 19 count: 0 metaSz: 2476 rp [1] sn: 20 count: 0 metaSz: 2477 rp [2] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:44.029+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 45577 recovery UsedSpace 62406 2023-10-21T04:44:44.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:44.030+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.030+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 20 count: 0 metaSz: 2477 rp [1] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:44.030+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 51275 recovery UsedSpace 70620 2023-10-21T04:44:44.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:44:44.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.036+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:44:44.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex started 2023-10-21T04:44:44.036+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.036+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:44:44.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.037+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:44:44.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex started 2023-10-21T04:44:44.037+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:44:44.038+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.039+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.051+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43494:172.31.5.112:12000] 2023-10-21T04:44:44.054+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43496:172.31.5.112:12000] 2023-10-21T04:44:44.071+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Warmup took 42.724217ms 2023-10-21T04:44:44.072+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 5 WriterThreads 1 2023-10-21T04:44:44.074+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Stopped Billing WUs 2023-10-21T04:44:44.075+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index Index: 4397940545762329850 Partition: 5 2023-10-21T04:44:44.075+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:44.075+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 6 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:44.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:44.076+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.076+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 19 count: 0 metaSz: 2477 rp [1] sn: 20 count: 0 metaSz: 2477 2023-10-21T04:44:44.076+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 53504 recovery UsedSpace 70620 2023-10-21T04:44:44.077+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:44.077+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.077+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 18 count: 0 metaSz: 2476 rp [1] sn: 19 count: 0 metaSz: 2477 rp [2] sn: 20 count: 0 metaSz: 2477 2023-10-21T04:44:44.078+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 47330 recovery UsedSpace 62406 2023-10-21T04:44:44.087+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43498:172.31.5.112:12000] 2023-10-21T04:44:44.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:44:44.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.088+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:44:44.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex started 2023-10-21T04:44:44.088+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.088+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:44:44.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.095+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:44:44.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex started 2023-10-21T04:44:44.095+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:44:44.096+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.097+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.099+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43500:172.31.5.112:12000] 2023-10-21T04:44:44.117+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Warmup took 40.978174ms 2023-10-21T04:44:44.117+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 6 WriterThreads 1 2023-10-21T04:44:44.117+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Stopped Billing WUs 2023-10-21T04:44:44.117+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index Index: 4397940545762329850 Partition: 6 2023-10-21T04:44:44.117+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:44.118+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 7 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:44.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:44.118+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:44.119+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.119+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 19 count: 0 metaSz: 2477 rp [1] sn: 20 count: 0 metaSz: 2477 2023-10-21T04:44:44.119+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 53504 recovery UsedSpace 70620 2023-10-21T04:44:44.119+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 18 count: 0 metaSz: 2476 rp [1] sn: 19 count: 0 metaSz: 2477 rp [2] sn: 20 count: 0 metaSz: 2477 2023-10-21T04:44:44.119+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 47330 recovery UsedSpace 62406 2023-10-21T04:44:44.122+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43502:172.31.5.112:12000] 2023-10-21T04:44:44.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:44:44.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.126+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:44:44.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex started 2023-10-21T04:44:44.126+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.126+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:44:44.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.127+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:44:44.127+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex started 2023-10-21T04:44:44.127+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:44:44.127+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.127+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.134+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43504:172.31.5.112:12000] 2023-10-21T04:44:44.141+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Warmup took 22.561368ms 2023-10-21T04:44:44.141+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 7 WriterThreads 1 2023-10-21T04:44:44.141+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Stopped Billing WUs 2023-10-21T04:44:44.141+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index Index: 4397940545762329850 Partition: 7 2023-10-21T04:44:44.141+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:44.141+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 8 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:44.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:44.142+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.142+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 19 count: 0 metaSz: 2477 rp [1] sn: 20 count: 0 metaSz: 2477 2023-10-21T04:44:44.142+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 55936 recovery UsedSpace 70620 2023-10-21T04:44:44.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:44.142+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.142+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 18 count: 0 metaSz: 2476 rp [1] sn: 19 count: 0 metaSz: 2477 rp [2] sn: 20 count: 0 metaSz: 2477 2023-10-21T04:44:44.142+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 49224 recovery UsedSpace 62406 2023-10-21T04:44:44.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:44:44.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.149+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:44:44.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex started 2023-10-21T04:44:44.149+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:44:44.149+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.149+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:44:44.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex started 2023-10-21T04:44:44.149+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:44:44.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.161+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43506:172.31.5.112:12000] 2023-10-21T04:44:44.168+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Warmup took 26.554358ms 2023-10-21T04:44:44.168+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 8 WriterThreads 1 2023-10-21T04:44:44.168+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Stopped Billing WUs 2023-10-21T04:44:44.168+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index Index: 4397940545762329850 Partition: 8 2023-10-21T04:44:44.168+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:44.169+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 1 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:44.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:44.169+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.169+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 19 count: 0 metaSz: 2476 rp [1] sn: 20 count: 0 metaSz: 2477 rp [2] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:44.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:44.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 20 count: 0 metaSz: 2477 rp [1] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:44.169+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 51275 recovery UsedSpace 62406 2023-10-21T04:44:44.169+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 55936 recovery UsedSpace 70620 2023-10-21T04:44:44.172+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43508:172.31.5.112:12000] 2023-10-21T04:44:44.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:44:44.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:44:44.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.174+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:44:44.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex started 2023-10-21T04:44:44.174+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.174+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.174+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:44:44.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex started 2023-10-21T04:44:44.174+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:44:44.174+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.174+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.186+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Warmup took 17.168899ms 2023-10-21T04:44:44.186+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 1 WriterThreads 1 2023-10-21T04:44:44.186+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Stopped Billing WUs 2023-10-21T04:44:44.187+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index Index: 4397940545762329850 Partition: 1 2023-10-21T04:44:44.187+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:44.187+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 2 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:44.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:44.187+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.187+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 19 count: 0 metaSz: 2476 rp [1] sn: 20 count: 0 metaSz: 2477 rp [2] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:44.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:44.187+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 51275 recovery UsedSpace 62406 2023-10-21T04:44:44.187+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.187+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 8 numRPs: 3 rp [0] sn: 19 count: 0 metaSz: 2476 rp [1] sn: 20 count: 0 metaSz: 2477 rp [2] sn: 21 count: 0 metaSz: 2477 2023-10-21T04:44:44.188+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 58600 recovery UsedSpace 62406 2023-10-21T04:44:44.196+05:30 [Info] {bucket,feeds} "bucket_%2" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:43510:172.31.5.112:12000] 2023-10-21T04:44:44.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:44:44.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.211+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:44:44.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex started 2023-10-21T04:44:44.213+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.213+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:44:44.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.215+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:44:44.215+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex started 2023-10-21T04:44:44.215+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:44:44.216+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.216+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.248+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Warmup took 61.044151ms 2023-10-21T04:44:44.249+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 2 WriterThreads 1 2023-10-21T04:44:44.249+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Stopped Billing WUs 2023-10-21T04:44:44.250+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index Index: 4397940545762329850 Partition: 2 2023-10-21T04:44:44.250+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:44.250+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 3 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:44.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:44.251+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.251+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 18 count: 0 metaSz: 2477 rp [1] sn: 19 count: 0 metaSz: 2477 2023-10-21T04:44:44.251+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 58600 recovery UsedSpace 70620 2023-10-21T04:44:44.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:44.252+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:44:44.252+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: RP version: 9 numRPs: 2 rp [0] sn: 18 count: 0 metaSz: 2477 rp [1] sn: 19 count: 0 metaSz: 2477 2023-10-21T04:44:44.253+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 53504 recovery UsedSpace 70620 2023-10-21T04:44:44.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:44:44.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:44.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:44.266+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:44:44.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex started 2023-10-21T04:44:44.266+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.266+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:44:44.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:44.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:44.270+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:44:44.270+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex started 2023-10-21T04:44:44.270+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:44.270+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:44:44.270+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:44.298+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Warmup took 47.577448ms 2023-10-21T04:44:44.299+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 3 WriterThreads 1 2023-10-21T04:44:44.299+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Stopped Billing WUs 2023-10-21T04:44:44.299+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index Index: 4397940545762329850 Partition: 3 2023-10-21T04:44:44.300+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4397940545762329850, shardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:44:44.300+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4397940545762329850, partnShardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:44:44.300+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:44:44.301+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4397940545762329850 2023-10-21T04:44:44.301+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:44:44.301+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:44:44.301+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:44:44.301+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:44:44.302+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:44:44.302+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4397940545762329850 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:44:44.303+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:7 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.305+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:8 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.307+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.78657ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:44.307+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 228.668µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:44:44.316+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:44.316+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:44.320+05:30 [Info] switched currmeta from 132 -> 133 force true 2023-10-21T04:44:44.307+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:1 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.339+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:2 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.340+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:3 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.341+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:4 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.342+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:5 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.343+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:6 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:44:44.603+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:44.603+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:44.605+05:30 [Info] switched currmeta from 133 -> 134 force true 2023-10-21T04:44:44.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9139063900541057, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:44:45.006+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4397940545762329850:INDEX_STATE_RECOVERED 10682617070590310048:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:44:45.007+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:44:45.007+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:44:45.030+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12396348969790516872,realInstId: 0, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:45.030+05:30 [Info] Indexer::handleRecoverIndex InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:44:45.030+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 425.587µs 2023-10-21T04:44:45.030+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 477.279µs 2023-10-21T04:44:45.031+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.031+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 8 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.031+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.031+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 35617 recovery UsedSpace 8236 2023-10-21T04:44:45.032+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12396348969790516872, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:44:45.032+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 25.431433ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:45.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.033+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.033+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 39009 recovery UsedSpace 8236 2023-10-21T04:44:45.042+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 4, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:44:45.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:45.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.048+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:45.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex started 2023-10-21T04:44:45.048+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.048+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:45.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.051+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:45.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex started 2023-10-21T04:44:45.051+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:44:45.051+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.051+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.051+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.051+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.051+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:45.051+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:45.051+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:45.051+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:45.051+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:45.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex stopped 2023-10-21T04:44:45.052+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.052+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.052+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.052+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.052+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:45.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex closed 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:45.064+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:45.064+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:45.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex stopped 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.064+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.069+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:44:45.069+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 11] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/04c71e696cccf09893ec2b2a16b010ff Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:04c71e696cccf09893ec2b2a16b010ff]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/04c71e696cccf09893ec2b2a16b010ff Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:04c71e696cccf09893ec2b2a16b010ff]}]} 2023-10-21T04:44:45.072+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:44:45.072+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 11] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:44:45.074+05:30 [Info] GenericServiceManager::CancelTask: called with id: rebalance/04c71e696cccf09893ec2b2a16b010ff, rev: [] 2023-10-21T04:44:45.074+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: rebalance/04c71e696cccf09893ec2b2a16b010ff, rev: [] 2023-10-21T04:44:45.074+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:44:45.074+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:44:45.074+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress cancel received. progress collector stopped 2023-10-21T04:44:45.074+05:30 [Info] ShardRebalancer::updateProgress Cancel Received. progress reporter stapped 2023-10-21T04:44:45.074+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:44:45.088+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:45.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex closed 2023-10-21T04:44:45.089+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:44:45.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:44:45.095+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:44:45.096+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:44:45.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.111+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index destroyed 2023-10-21T04:44:45.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.127+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.134+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:44:45.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.136+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:45.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.141+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:45.141+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:45.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex started 2023-10-21T04:44:45.141+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.141+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:45.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.145+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:45.146+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:45.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex started 2023-10-21T04:44:45.146+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Warmup took 115.243655ms 2023-10-21T04:44:45.146+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.146+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.149+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:45.149+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:45.152+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 8 WriterThreads 1 2023-10-21T04:44:45.152+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Stopped Billing WUs 2023-10-21T04:44:45.152+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index Index: 12396348969790516872 Partition: 8 2023-10-21T04:44:45.152+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.153+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 1 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.153+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.153+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 37236 recovery UsedSpace 8236 2023-10-21T04:44:45.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.155+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.155+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 8236 2023-10-21T04:44:45.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:45.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.178+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:45.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex started 2023-10-21T04:44:45.178+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.178+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:45.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.179+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:45.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex started 2023-10-21T04:44:45.179+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:44:45.179+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:45.179+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:45.179+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:45.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex stopped 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.179+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.180+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.180+05:30 [Info] switched currmeta from 134 -> 135 force true 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:45.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex closed 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:45.186+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:45.186+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:45.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex stopped 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.186+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.196+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:45.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex closed 2023-10-21T04:44:45.196+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:44:45.196+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:44:45.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:44:45.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:44:45.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index destroyed 2023-10-21T04:44:45.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.214+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.215+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.225+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.226+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:45.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.229+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.229+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:45.230+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:45.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex started 2023-10-21T04:44:45.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:45.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.247+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:45.297+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:45.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex started 2023-10-21T04:44:45.305+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.305+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.308+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Warmup took 155.464903ms 2023-10-21T04:44:45.315+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 1 WriterThreads 1 2023-10-21T04:44:45.316+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Stopped Billing WUs 2023-10-21T04:44:45.316+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index Index: 12396348969790516872 Partition: 1 2023-10-21T04:44:45.317+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.317+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 2 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.318+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 8236 2023-10-21T04:44:45.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:45.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.326+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:45.326+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex started 2023-10-21T04:44:45.326+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.326+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.331+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.333+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 37236 recovery UsedSpace 8236 2023-10-21T04:44:45.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:45.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.348+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:45.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex started 2023-10-21T04:44:45.348+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:44:45.348+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:45.349+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:45.349+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:45.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex stopped 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.349+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.350+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.350+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.350+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.355+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.355+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:45.355+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex closed 2023-10-21T04:44:45.356+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.356+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.356+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.356+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:45.357+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:45.357+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:45.357+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:45.358+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:45.358+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex stopped 2023-10-21T04:44:45.358+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.358+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.358+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.358+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.373+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:45.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex closed 2023-10-21T04:44:45.374+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:44:45.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:44:45.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:44:45.380+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:44:45.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.385+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index destroyed 2023-10-21T04:44:45.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.390+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.390+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.395+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.395+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.395+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:45.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.405+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:45.405+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:45.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex started 2023-10-21T04:44:45.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:45.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.412+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:45.412+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:45.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex started 2023-10-21T04:44:45.413+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Warmup took 95.042688ms 2023-10-21T04:44:45.413+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 2 WriterThreads 1 2023-10-21T04:44:45.413+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Stopped Billing WUs 2023-10-21T04:44:45.414+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index Index: 12396348969790516872 Partition: 2 2023-10-21T04:44:45.414+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.411+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.412+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.414+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.414+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.415+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 3 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.416+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.416+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 37236 recovery UsedSpace 8236 2023-10-21T04:44:45.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.417+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.417+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 8236 2023-10-21T04:44:45.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:45.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.424+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:45.424+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex started 2023-10-21T04:44:45.424+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.424+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:45.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.444+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:45.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex started 2023-10-21T04:44:45.445+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:44:45.445+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:45.445+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:45.445+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:45.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex stopped 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.445+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.446+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.446+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.446+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.449+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:45.449+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:45.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex closed 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:45.450+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:45.450+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:45.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex stopped 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.450+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.458+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:45.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex closed 2023-10-21T04:44:45.459+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:44:45.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:44:45.460+05:30 [Info] switched currmeta from 135 -> 135 force true 2023-10-21T04:44:45.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:44:45.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:44:45.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.465+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index destroyed 2023-10-21T04:44:45.468+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.477+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.477+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.477+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:45.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.481+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:45.481+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:45.481+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex started 2023-10-21T04:44:45.481+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.481+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.489+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.489+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:45.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.495+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:45.495+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:45.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex started 2023-10-21T04:44:45.495+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Warmup took 80.083625ms 2023-10-21T04:44:45.496+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 3 WriterThreads 1 2023-10-21T04:44:45.496+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Stopped Billing WUs 2023-10-21T04:44:45.496+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index Index: 12396348969790516872 Partition: 3 2023-10-21T04:44:45.496+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.496+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.496+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 4 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.496+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.497+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.497+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 40960 recovery UsedSpace 8236 2023-10-21T04:44:45.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.498+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.498+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 39009 recovery UsedSpace 8236 2023-10-21T04:44:45.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:45.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.503+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:45.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex started 2023-10-21T04:44:45.503+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.503+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:45.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.504+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:45.504+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex started 2023-10-21T04:44:45.505+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:44:45.505+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.506+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.506+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.506+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.508+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.508+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:45.509+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:45.509+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:45.509+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:45.509+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:45.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex stopped 2023-10-21T04:44:45.510+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.510+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.510+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.510+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.516+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:45.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex closed 2023-10-21T04:44:45.517+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.517+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.517+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.518+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:45.518+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:45.518+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:45.519+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:45.519+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:45.519+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex stopped 2023-10-21T04:44:45.519+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.519+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.520+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.520+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.526+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:45.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex closed 2023-10-21T04:44:45.526+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:44:45.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:44:45.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:44:45.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:44:45.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index destroyed 2023-10-21T04:44:45.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.549+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.552+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.552+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.552+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:45.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.561+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:45.561+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:45.561+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex started 2023-10-21T04:44:45.561+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.561+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:45.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.564+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.565+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:45.565+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:45.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex started 2023-10-21T04:44:45.565+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Warmup took 67.818479ms 2023-10-21T04:44:45.565+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 4 WriterThreads 1 2023-10-21T04:44:45.565+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Stopped Billing WUs 2023-10-21T04:44:45.565+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index Index: 12396348969790516872 Partition: 4 2023-10-21T04:44:45.565+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.565+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 5 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.565+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.565+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.565+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.566+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.566+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 39009 recovery UsedSpace 8236 2023-10-21T04:44:45.569+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:44:45.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:45.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.578+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:45.578+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex started 2023-10-21T04:44:45.578+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.578+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:45.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.580+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:45.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex started 2023-10-21T04:44:45.580+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:44:45.580+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.580+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.580+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.580+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:45.580+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:45.580+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:45.580+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:45.580+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:45.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex stopped 2023-10-21T04:44:45.581+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.581+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.583+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.584+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.584+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.587+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.605+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:45.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex closed 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:45.606+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:45.606+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:45.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex stopped 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.606+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.622+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:45.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex closed 2023-10-21T04:44:45.622+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:44:45.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:44:45.630+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:45.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:44:45.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:44:45.638+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:45.638+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:45.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.640+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index destroyed 2023-10-21T04:44:45.642+05:30 [Info] switched currmeta from 135 -> 136 force true 2023-10-21T04:44:45.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.657+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.660+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.662+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:45.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.671+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:45.671+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:45.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex started 2023-10-21T04:44:45.671+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.671+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.676+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.676+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:45.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.689+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:45.689+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:45.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex started 2023-10-21T04:44:45.690+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Warmup took 124.508354ms 2023-10-21T04:44:45.690+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.690+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.690+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 5 WriterThreads 1 2023-10-21T04:44:45.690+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Stopped Billing WUs 2023-10-21T04:44:45.690+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index Index: 12396348969790516872 Partition: 5 2023-10-21T04:44:45.690+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.691+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 6 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.691+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.691+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 39009 recovery UsedSpace 8236 2023-10-21T04:44:45.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.692+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.692+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:44:45.697+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:44:45.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:45.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.701+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:45.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex started 2023-10-21T04:44:45.701+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.701+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:45.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.706+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:45.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex started 2023-10-21T04:44:45.706+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:44:45.706+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:45.706+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:45.706+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:45.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex stopped 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.706+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.716+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:45.716+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex closed 2023-10-21T04:44:45.716+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.716+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.716+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.716+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:44:45.717+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:45.717+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:44:45.717+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:44:45.717+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:45.717+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex stopped 2023-10-21T04:44:45.717+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.717+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.717+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.717+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.723+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:44:45.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex closed 2023-10-21T04:44:45.723+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:44:45.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:44:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:44:45.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:44:45.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index destroyed 2023-10-21T04:44:45.751+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:45.751+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:45.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.752+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.752+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.756+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.764+05:30 [Info] switched currmeta from 136 -> 137 force true 2023-10-21T04:44:45.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:45.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.765+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:45.765+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:45.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex started 2023-10-21T04:44:45.765+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.765+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:45.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.768+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:45.768+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:45.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex started 2023-10-21T04:44:45.768+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Warmup took 77.663613ms 2023-10-21T04:44:45.769+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 6 WriterThreads 1 2023-10-21T04:44:45.769+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Stopped Billing WUs 2023-10-21T04:44:45.769+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index Index: 12396348969790516872 Partition: 6 2023-10-21T04:44:45.769+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:44:45.769+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 7 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:44:45.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.769+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.769+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.769+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.769+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 39009 recovery UsedSpace 8236 2023-10-21T04:44:45.770+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.770+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:44:45.770+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:44:45.773+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:45.773+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:45.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex started 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex started 2023-10-21T04:44:45.775+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:44:45.775+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:45.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:45.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:45.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex stopped 2023-10-21T04:44:45.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.776+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.776+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.776+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.776+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.778+05:30 [Info] switched currmeta from 137 -> 137 force true 2023-10-21T04:44:45.782+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:45.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex closed 2023-10-21T04:44:45.782+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:45.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:45.783+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:45.783+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:45.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex stopped 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:45.783+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:45.787+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:45.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:45.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex closed 2023-10-21T04:44:45.787+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:45.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:44:45.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:44:45.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex sucessfully destroyed 2023-10-21T04:44:45.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:44:45.791+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:44:45.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index destroyed 2023-10-21T04:44:45.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:44:45.797+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.797+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:45.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:44:45.800+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:44:45.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:44:45.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:45.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:44:45.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:44:45.805+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:45.805+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:45.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex started 2023-10-21T04:44:45.805+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.805+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:45.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:44:45.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:44:45.807+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:45.807+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:45.807+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex started 2023-10-21T04:44:45.807+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Warmup took 37.809481ms 2023-10-21T04:44:45.807+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 7 WriterThreads 1 2023-10-21T04:44:45.807+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Stopped Billing WUs 2023-10-21T04:44:45.807+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index Index: 12396348969790516872 Partition: 7 2023-10-21T04:44:45.807+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12396348969790516872, shardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:44:45.807+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12396348969790516872, partnShardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:44:45.807+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:44:45.808+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12396348969790516872 2023-10-21T04:44:45.808+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12396348969790516872 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:44:45.808+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:44:45.808+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:44:45.808+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12396348969790516872 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:44:45.808+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:44:45.808+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12396348969790516872 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:44:45.808+05:30 [Info] StorageMgr::openSnapshot IndexInst:12396348969790516872 Partition:7 No Snapshot Found. 2023-10-21T04:44:45.808+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12396348969790516872 Adding Nil Snapshot. 2023-10-21T04:44:45.808+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:44:45.808+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:44:45.816+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.204259ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:45.816+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 106.491µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:44:45.837+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:45.837+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:45.854+05:30 [Info] switched currmeta from 137 -> 138 force true 2023-10-21T04:44:47.372+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:44:47.406+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:44:48.069+05:30 [Info] ShardRebalancer::waitForIndexState Cancel Received 2023-10-21T04:44:48.069+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardTokene3:4b:4c:c2:98:2a:73:e7 2023-10-21T04:44:48.069+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:44:48.071+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {4325b30e7f9620a998388078247335a8 04c71e696cccf09893ec2b2a16b010ff Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:44:48.092+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:44:48.093+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken11:3e:79:72:8e:79:e:f5 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:e6:93:c0:4d:37:29:98 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 14637134422972922160 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14297924784489035508 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10357771762526486340 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:48.099+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 04c71e696cccf09893ec2b2a16b010ff TransferTokenId: ShardToken11:3e:79:72:8e:79:e:f5 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:44:48.099+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.00014668 2023-10-21T04:44:48.100+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken11:3e:79:72:8e:79:e:f5, dropIndexes: true 2023-10-21T04:44:48.099+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5, shardId :0 2023-10-21T04:44:48.101+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardToken11_3e_79_72_8e_79_e_f5, shardId :0 2023-10-21T04:44:48.102+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16823504650100619022 instance id 14637134422972922160 real instance id 4186965058349029465 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:48.102+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 232.719µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:48.108+05:30 [Info] LifecycleMgr.DeleteIndex() : index 16483001637761719171 does not exist. 2023-10-21T04:44:48.108+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 70.374µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:48.114+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 7790129686881525591 instance id 10357771762526486340 real instance id 16759912838651900043 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:48.114+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 191.803µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:48.114+05:30 [Info] LifecycleMgr.DeleteIndex() : index 668551233566132751 does not exist. 2023-10-21T04:44:48.114+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 69.216µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:48.115+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17327901024118572215 5629943257627822009] 2023-10-21T04:44:48.412+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:44:48.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17327901024118572215 2023-10-21T04:44:48.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 5629943257627822009 2023-10-21T04:44:48.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 5629943257627822009 2023-10-21T04:44:48.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17327901024118572215 2023-10-21T04:44:48.694+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17327901024118572215 5629943257627822009], elapsed: 579.027107ms 2023-10-21T04:44:48.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:48.694+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:44:48.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:44:48.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shutdown completed 2023-10-21T04:44:48.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:48.694+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:44:48.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:44:48.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance destroyed 2023-10-21T04:44:48.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:44:48.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:48.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:44:48.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:44:48.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shutdown completed 2023-10-21T04:44:48.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:48.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:44:48.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:44:48.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance destroyed 2023-10-21T04:44:48.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:44:48.696+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17327901024118572215 5629943257627822009], elapsed(sec): 0.581564899 2023-10-21T04:44:48.699+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokene3:4b:4c:c2:98:2a:73:e7 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 04c71e696cccf09893ec2b2a16b010ff ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken19:fe:77:b2:bc:19:17:a2 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10682617070590310048 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6403098372720496675 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 3335851955840345084 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:44:48.699+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 04c71e696cccf09893ec2b2a16b010ff TransferTokenId: ShardTokene3:4b:4c:c2:98:2a:73:e7 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:44:48.699+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000102575 2023-10-21T04:44:48.699+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokene3:4b:4c:c2:98:2a:73:e7, dropIndexes: true 2023-10-21T04:44:48.700+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7, shardId :0 2023-10-21T04:44:48.700+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/04c71e696cccf09893ec2b2a16b010ff_ShardTokene3_4b_4c_c2_98_2a_73_e7, shardId :0 2023-10-21T04:44:48.736+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10682617070590310048 &{1} 2023-10-21T04:44:48.736+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10682617070590310048 2023-10-21T04:44:48.736+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 7045704526128519604, instId: 10682617070590310048 2023-10-21T04:44:48.737+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10682617070590310048] 2023-10-21T04:44:48.737+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10682617070590310048] 2023-10-21T04:44:48.737+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:44:48.737+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10682617070590310048] 2023-10-21T04:44:48.737+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 10682617070590310048 Defn: DefnId: 7045704526128519604 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:44:48.737+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 774.488µs 2023-10-21T04:44:48.737+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 819.01µs 2023-10-21T04:44:48.737+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10682617070590310048, PartitionId 0, IndexDefnId 7045704526128519604. 2023-10-21T04:44:48.737+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.737+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.737+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex] 2023-10-21T04:44:48.738+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex 2023-10-21T04:44:48.738+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex] 2023-10-21T04:44:48.738+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex] 2023-10-21T04:44:48.738+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex 2023-10-21T04:44:48.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex stopped 2023-10-21T04:44:48.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.738+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10682617070590310048 2023-10-21T04:44:48.754+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 43.927824ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:48.756+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex] 2023-10-21T04:44:48.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex closed 2023-10-21T04:44:48.757+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.757+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.757+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.757+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.757+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 4390029557575378783 instance id 6403098372720496675 real instance id 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:48.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex] 2023-10-21T04:44:48.758+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 4390029557575378783 instance 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:48.758+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 4390029557575378783 instance id 4397940545762329850 2023-10-21T04:44:48.758+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 4390029557575378783 2023-10-21T04:44:48.758+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex 2023-10-21T04:44:48.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex] 2023-10-21T04:44:48.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex] 2023-10-21T04:44:48.760+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex 2023-10-21T04:44:48.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex stopped 2023-10-21T04:44:48.761+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.761+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.761+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.761+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.766+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex] 2023-10-21T04:44:48.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex closed 2023-10-21T04:44:48.766+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.766+05:30 [Info] Indexer::cleanupIndexData IndexInst 10682617070590310048 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex ... 2023-10-21T04:44:48.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex ... 2023-10-21T04:44:48.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/mainIndex sucessfully destroyed 2023-10-21T04:44:48.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex ... 2023-10-21T04:44:48.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 20 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex ... 2023-10-21T04:44:48.771+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4397940545762329850 &{1} 2023-10-21T04:44:48.771+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4397940545762329850 2023-10-21T04:44:48.771+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 4390029557575378783, instId: 4397940545762329850 2023-10-21T04:44:48.771+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4397940545762329850] 2023-10-21T04:44:48.771+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4397940545762329850] 2023-10-21T04:44:48.771+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:44:48.771+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4397940545762329850] 2023-10-21T04:44:48.771+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.771+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.771+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.771+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:44:48.771+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:44:48.771+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:44:48.771+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:44:48.771+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:44:48.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex stopped 2023-10-21T04:44:48.772+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.772+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.772+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.772+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.772+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_10682617070590310048_0.index destroyed 2023-10-21T04:44:48.773+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10682617070590310048, PartitionId 0, IndexDefnId 7045704526128519604. 2023-10-21T04:44:48.773+05:30 [Info] Indexer::cleanupIndexData IndexInst 10682617070590310048 Partition 0 Destroy Done 2023-10-21T04:44:48.773+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:44:48.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex stopped 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.773+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.773+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.773+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:44:48.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex stopped 2023-10-21T04:44:48.773+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:44:48.773+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:44:48.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex stopped 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.773+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.774+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:44:48.774+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:44:48.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex stopped 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.774+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:44:48.774+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:44:48.774+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:44:48.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex stopped 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.774+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.774+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:44:48.774+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.861887ms 2023-10-21T04:44:48.774+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 3.898953ms 2023-10-21T04:44:48.774+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:44:48.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:44:48.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:44:48.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex stopped 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.775+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:44:48.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:44:48.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:44:48.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex stopped 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.775+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4397940545762329850 2023-10-21T04:44:48.781+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:44:48.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex closed 2023-10-21T04:44:48.781+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.781+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.781+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.781+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:44:48.782+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:44:48.782+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:44:48.782+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:44:48.782+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:44:48.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex stopped 2023-10-21T04:44:48.782+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.782+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.782+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.782+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.787+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:44:48.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex closed 2023-10-21T04:44:48.787+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.787+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex ... 2023-10-21T04:44:48.787+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex ... 2023-10-21T04:44:48.788+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.790+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex sucessfully destroyed 2023-10-21T04:44:48.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex ... 2023-10-21T04:44:48.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex ... 2023-10-21T04:44:48.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:44:48.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex closed 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:44:48.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:44:48.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex closed 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:44:48.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index destroyed 2023-10-21T04:44:48.801+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.801+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 2 Destroy Done 2023-10-21T04:44:48.801+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:44:48.801+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:44:48.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex stopped 2023-10-21T04:44:48.801+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.802+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.802+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.802+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.808+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:44:48.808+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:44:48.808+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:44:48.808+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:44:48.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex stopped 2023-10-21T04:44:48.808+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.808+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.808+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.808+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.814+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:44:48.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex closed 2023-10-21T04:44:48.818+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:44:48.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex closed 2023-10-21T04:44:48.820+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.821+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex ... 2023-10-21T04:44:48.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex ... 2023-10-21T04:44:48.821+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.821+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.822+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.825+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.826+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex sucessfully destroyed 2023-10-21T04:44:48.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex ... 2023-10-21T04:44:48.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex ... 2023-10-21T04:44:48.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex ... 2023-10-21T04:44:48.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex ... 2023-10-21T04:44:48.833+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.835+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 78.267291ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:48.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex sucessfully destroyed 2023-10-21T04:44:48.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:44:48.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex closed 2023-10-21T04:44:48.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:44:48.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex closed 2023-10-21T04:44:48.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:44:48.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex closed 2023-10-21T04:44:48.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:44:48.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex closed 2023-10-21T04:44:48.840+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.844+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.841+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.841+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.841+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.843+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.847+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.847+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.848+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex sucessfully destroyed 2023-10-21T04:44:48.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:44:48.849+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:44:48.850+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.850+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.851+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex ... 2023-10-21T04:44:48.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex closed 2023-10-21T04:44:48.853+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 17703248484023971553 instance id 3335851955840345084 real instance id 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:48.856+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index destroyed 2023-10-21T04:44:48.857+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:44:48.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex ... 2023-10-21T04:44:48.858+05:30 [Info] bucket_%2/idx_secondary/Mainstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.859+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 17703248484023971553 instance 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:44:48.862+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:44:48.865+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.867+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.868+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:44:48.865+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 17703248484023971553 instance id 12396348969790516872 2023-10-21T04:44:48.868+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 17703248484023971553 2023-10-21T04:44:48.867+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.869+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 5 Destroy Done 2023-10-21T04:44:48.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index destroyed 2023-10-21T04:44:48.873+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.873+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 3 Destroy Done 2023-10-21T04:44:48.873+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:44:48.873+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:44:48.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex stopped 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.873+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:44:48.873+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:44:48.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex stopped 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.873+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:44:48.873+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:44:48.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex stopped 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.873+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.874+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:44:48.874+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:44:48.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex stopped 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.874+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:44:48.874+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:44:48.874+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex stopped 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:48.874+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:48.924+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:44:48.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex closed 2023-10-21T04:44:48.925+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.925+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex ... 2023-10-21T04:44:48.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex ... 2023-10-21T04:44:48.926+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12396348969790516872 &{1} 2023-10-21T04:44:48.949+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12396348969790516872 2023-10-21T04:44:48.949+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17703248484023971553, instId: 12396348969790516872 2023-10-21T04:44:48.949+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12396348969790516872] 2023-10-21T04:44:48.949+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12396348969790516872] 2023-10-21T04:44:48.949+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:44:48.949+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12396348969790516872] 2023-10-21T04:44:48.949+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:44:48.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex closed 2023-10-21T04:44:48.950+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.950+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.950+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.950+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.952+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex sucessfully destroyed 2023-10-21T04:44:48.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex ... 2023-10-21T04:44:48.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex ... 2023-10-21T04:44:48.966+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.952+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:44:48.967+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 18.732864ms 2023-10-21T04:44:48.968+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 18.975276ms 2023-10-21T04:44:48.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex ... 2023-10-21T04:44:48.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex ... 2023-10-21T04:44:48.968+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12396348969790516872 2023-10-21T04:44:48.952+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:48.952+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:48.952+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:48.952+05:30 [Info] plasmaSlice::Close Soft Closing Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7, IndexDefnId 17703248484023971553 2023-10-21T04:44:48.969+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 7 Close Done 2023-10-21T04:44:48.969+05:30 [Info] plasmaSlice::Destroy Soft deleted Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7 IndexDefnId 17703248484023971553 RefCount 1 NumOpenSnapshots 0 2023-10-21T04:44:48.969+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 7 Destroy Done 2023-10-21T04:44:48.952+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:48.952+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:48.953+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:48.952+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:48.972+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:44:48.972+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:48.972+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:48.972+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:48.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:48.975+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.990+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:48.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex sucessfully destroyed 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index destroyed 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex closed 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex closed 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex closed 2023-10-21T04:44:48.994+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.994+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.994+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:44:48.994+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 8 Destroy Done 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex ... 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex ... 2023-10-21T04:44:48.996+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.996+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.996+05:30 [Info] bucket_%2/idx_secondary/Backstore#10682617070590310048:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:48.996+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:48.994+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex ... 2023-10-21T04:44:48.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex ... 2023-10-21T04:44:49.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index destroyed 2023-10-21T04:44:49.003+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:49.003+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:49.003+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:44:49.003+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 6 Destroy Done 2023-10-21T04:44:49.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex ... 2023-10-21T04:44:49.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex ... 2023-10-21T04:44:49.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex ... 2023-10-21T04:44:49.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex ... 2023-10-21T04:44:49.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index destroyed 2023-10-21T04:44:49.009+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:44:49.009+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 4 Destroy Done 2023-10-21T04:44:49.024+05:30 [Info] switched currmeta from 138 -> 143 force true 2023-10-21T04:44:49.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:44:49.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:49.027+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:49.027+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:49.027+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:44:49.027+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:49.027+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:49.027+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:49.027+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:44:49.027+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:49.027+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:49.028+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:49.028+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:44:49.028+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:49.028+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:49.028+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:49.028+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:44:49.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex stopped 2023-10-21T04:44:49.029+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.030+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex stopped 2023-10-21T04:44:49.030+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.030+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.030+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.030+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.030+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.030+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex ... 2023-10-21T04:44:49.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex ... 2023-10-21T04:44:49.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex stopped 2023-10-21T04:44:49.031+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.031+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.031+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex stopped 2023-10-21T04:44:49.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex stopped 2023-10-21T04:44:49.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex stopped 2023-10-21T04:44:49.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex stopped 2023-10-21T04:44:49.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex stopped 2023-10-21T04:44:49.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.036+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index destroyed 2023-10-21T04:44:49.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex ... 2023-10-21T04:44:49.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex ... 2023-10-21T04:44:49.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.040+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:44:49.040+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 1 Destroy Done 2023-10-21T04:44:49.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex ... 2023-10-21T04:44:49.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex ... 2023-10-21T04:44:49.052+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:44:49.052+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex closed 2023-10-21T04:44:49.052+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.052+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.053+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.053+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:49.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:49.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index destroyed 2023-10-21T04:44:49.054+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:44:49.054+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 7 Destroy Done 2023-10-21T04:44:49.054+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:49.054+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:49.054+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:49.054+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:44:49.054+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex stopped 2023-10-21T04:44:49.054+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.054+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.054+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.054+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.056+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:49.056+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:49.053+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 203.709949ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:49.075+05:30 [Info] switched currmeta from 143 -> 144 force true 2023-10-21T04:44:49.081+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:44:49.082+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex closed 2023-10-21T04:44:49.082+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.083+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:44:49.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:44:49.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:44:49.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:44:49.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index destroyed 2023-10-21T04:44:49.096+05:30 [Info] LifecycleMgr.DeleteIndex() : index 5277316367626246544 does not exist. 2023-10-21T04:44:49.098+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 2.834993ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:49.098+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.102+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 3 Destroy Done 2023-10-21T04:44:49.107+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:49.107+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [18374611443167983589 6455624821495495854] 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:44:49.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex closed 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:49.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:49.108+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:49.108+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:44:49.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex stopped 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.109+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:44:49.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex closed 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:49.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:49.120+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:49.120+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:44:49.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex stopped 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:44:49.121+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex closed 2023-10-21T04:44:49.121+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.122+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:44:49.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:44:49.124+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.125+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.126+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:49.126+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:49.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:44:49.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:44:49.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:44:49.133+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.136+05:30 [Info] switched currmeta from 144 -> 144 force true 2023-10-21T04:44:49.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex closed 2023-10-21T04:44:49.137+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.138+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.138+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.138+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:49.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:44:49.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex closed 2023-10-21T04:44:49.140+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.138+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:49.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index destroyed 2023-10-21T04:44:49.139+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:44:49.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex closed 2023-10-21T04:44:49.140+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:44:49.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex closed 2023-10-21T04:44:49.141+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.141+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:44:49.142+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.142+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.143+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 4 Close Done 2023-10-21T04:44:49.142+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:49.142+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 2 Destroy Done 2023-10-21T04:44:49.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex closed 2023-10-21T04:44:49.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:49.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:44:49.144+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:49.144+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:49.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:49.145+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:44:49.146+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:49.148+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.149+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.149+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Stopped monitor Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex closed 2023-10-21T04:44:49.149+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:44:49.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:44:49.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:44:49.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:49.149+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:44:49.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex stopped 2023-10-21T04:44:49.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.150+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.151+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.151+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:44:49.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:44:49.151+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:44:49.152+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:44:49.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex stopped 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.152+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:49.152+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:44:49.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex stopped 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.152+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:49.152+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:44:49.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex stopped 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:49.156+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:49.156+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:49.157+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:49.157+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:49.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.167+05:30 [Info] switched currmeta from 144 -> 144 force true 2023-10-21T04:44:49.189+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.195+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.196+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.224+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:49.224+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:44:49.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex stopped 2023-10-21T04:44:49.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:44:49.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:44:49.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:44:49.224+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:44:49.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:44:49.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex sucessfully destroyed 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex closed 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index destroyed 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex closed 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:44:49.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex closed 2023-10-21T04:44:49.230+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 1 Close Done 2023-10-21T04:44:49.230+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:44:49.230+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 4 Destroy Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex closed 2023-10-21T04:44:49.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:44:49.230+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:44:49.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.230+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.230+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.239+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:44:49.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:44:49.239+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex closed 2023-10-21T04:44:49.239+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:44:49.239+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:44:49.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:44:49.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:44:49.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:44:49.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:44:49.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:44:49.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:44:49.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index destroyed 2023-10-21T04:44:49.246+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.246+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 1 Destroy Done 2023-10-21T04:44:49.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:44:49.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:44:49.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:44:49.249+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:44:49.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index destroyed 2023-10-21T04:44:49.250+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.250+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 6 Destroy Done 2023-10-21T04:44:49.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:44:49.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:44:49.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:44:49.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:44:49.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index destroyed 2023-10-21T04:44:49.254+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.254+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 8 Destroy Done 2023-10-21T04:44:49.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:44:49.254+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:44:49.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex sucessfully destroyed 2023-10-21T04:44:49.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index destroyed 2023-10-21T04:44:49.258+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.258+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 5 Destroy Done 2023-10-21T04:44:49.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:44:49.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:44:49.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:44:49.261+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index destroyed 2023-10-21T04:44:49.261+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:44:49.429+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:44:49.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 18374611443167983589 2023-10-21T04:44:49.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 6455624821495495854 2023-10-21T04:44:49.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 18374611443167983589 2023-10-21T04:44:49.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 6455624821495495854 2023-10-21T04:44:49.694+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [18374611443167983589 6455624821495495854], elapsed: 586.689582ms 2023-10-21T04:44:49.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:49.694+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:44:49.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper stopped 2023-10-21T04:44:49.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': stopped 2023-10-21T04:44:49.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:44:49.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:44:49.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:44:49.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:44:49.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:44:49.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:44:49.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:44:49.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:44:49.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:44:49.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:44:49.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:44:49.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:44:49.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:44:49.697+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:44:49.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:44:49.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:44:49.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:44:49.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:44:49.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shutdown completed 2023-10-21T04:44:49.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:49.698+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:44:49.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:44:49.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:44:49.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance destroyed 2023-10-21T04:44:49.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:44:49.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:49.699+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:44:49.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:44:49.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:44:49.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:44:49.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:44:49.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:44:49.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:44:49.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:44:49.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:44:49.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:44:49.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:44:49.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:44:49.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:44:49.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:44:49.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:44:49.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:44:49.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:44:49.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:44:49.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:44:49.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shutdown completed 2023-10-21T04:44:49.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:44:49.702+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:44:49.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:44:49.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:44:49.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance destroyed 2023-10-21T04:44:49.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:44:49.704+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Cleaned up Tenant with name:bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:44:49.704+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [18374611443167983589 6455624821495495854], elapsed(sec): 0.596680353 2023-10-21T04:44:49.710+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:44:49.710+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:44:49.710+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.822µs 2023-10-21T04:44:49.710+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:44:49.710+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:44:49.710+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:44:49.715+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:44:49.715+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:44:49.715+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:44:49.722+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:44:49.722+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 49.479µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:49.725+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:44:49.725+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:44:49.725+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced false, isMaster true, forceUnbalanced true, err: 2023-10-21T04:44:49.725+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 4.651444505s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:44:49.725+05:30 [Info] GenericServiceManager::CancelTask: return from id: rebalance/04c71e696cccf09893ec2b2a16b010ff, rev: [] call. err: 2023-10-21T04:44:49.725+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:44:49.726+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 11] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[]} 2023-10-21T04:44:49.726+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:44:49.733+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/04c71e696cccf09893ec2b2a16b010ff, rev: [] 2023-10-21T04:44:49.733+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/04c71e696cccf09893ec2b2a16b010ff, rev: [] 2023-10-21T04:44:49.733+05:30 [Info] PauseServiceManager::endTask: called with err for taskId prepare/04c71e696cccf09893ec2b2a16b010ff 2023-10-21T04:44:49.733+05:30 [Info] PauseServiceManager::endTask task with ID prepare/04c71e696cccf09893ec2b2a16b010ff already cleaned up 2023-10-21T04:44:49.733+05:30 [Info] PauseServiceManager::PauseResumeCancelTask: removed all tasks with ID prepare/04c71e696cccf09893ec2b2a16b010ff 2023-10-21T04:44:49.733+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/04c71e696cccf09893ec2b2a16b010ff, rev: [] call. err: 2023-10-21T04:44:49.742+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:44:49.742+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[]} 2023-10-21T04:44:49.750+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:44:49.750+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:44:49.764+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 12] 2023-10-21T04:44:49.767+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 12] 2023-10-21T04:44:49.786+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9106 2023-10-21T04:44:49.786+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:44:49.801+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:44:49.827+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:44:49.827+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9112 2023-10-21T04:44:49.827+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:44:49.833+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:44:49.856+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:44:49.856+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:44:49.856+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:44:49.863+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:54290 2023-10-21T04:44:49.864+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:44:49.877+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 11.471942ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:44:49.877+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:44:49.877+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:44:49.877+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:44:49.882+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:44:49.903+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:44:49.904+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:44:49.904+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:44:49.904+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:44:49.904+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:44:49.906+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:44:49.906+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:44:49.906+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:44:49.906+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:44:49.906+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:44:49.906+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:44:49.906+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:44:49.906+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:45536->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:49.907+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:50666->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:49.908+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:54290->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:49.909+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:45996->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:44:49.914+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:54290. Reason = EOF. Cleaning Pipe. 2023-10-21T04:44:49.914+05:30 [Info] messageListener.start(): message channel closed. Remove peer 20:8e:e5:9:4b:8d:7f:cb as follower. 2023-10-21T04:44:50.378+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:44:50.437+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:44:53.801+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:44:53.806+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:44:53.807+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:44:53.808+05:30 [Info] switched currmeta from 144 -> 144 force true 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:01.413+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:45:02.730+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:45:04.019+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:04.019+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:04.025+05:30 [Info] switched currmeta from 144 -> 147 force true 2023-10-21T04:45:04.083+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:04.083+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:04.101+05:30 [Info] switched currmeta from 147 -> 148 force true 2023-10-21T04:45:04.190+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:04.190+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:04.204+05:30 [Info] switched currmeta from 148 -> 149 force true 2023-10-21T04:45:04.289+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:04.289+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:04.301+05:30 [Info] switched currmeta from 149 -> 152 force true 2023-10-21T04:45:04.337+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:04.337+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:04.343+05:30 [Info] switched currmeta from 152 -> 152 force true 2023-10-21T04:45:09.197+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:45:09.213+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:09.214+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:09.228+05:30 [Info] switched currmeta from 152 -> 152 force true 2023-10-21T04:45:10.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9669632708253425, throttleDelayMs (new, old, change): (2, 0, 2) 2023-10-21T04:45:11.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9994069595082701, throttleDelayMs (new, old, change): (7, 2, 5) 2023-10-21T04:45:12.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9171554633337515, throttleDelayMs (new, old, change): (4, 7, -3) 2023-10-21T04:45:13.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.507562400146015, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:45:19.765+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 12] call. err: operation_canceled 2023-10-21T04:45:19.765+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:45:19.765+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 12] Tasks:[]} 2023-10-21T04:45:19.766+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 12] 2023-10-21T04:45:19.767+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:45:19.767+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 12] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:45:19.769+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 12] 2023-10-21T04:45:22.760+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:45:22.786+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:22.786+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:22.789+05:30 [Info] switched currmeta from 152 -> 152 force true 2023-10-21T04:45:24.854+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 5932387718257253322 2023-10-21T04:45:24.854+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 197.665µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:24.933+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39912 2023-10-21T04:45:24.941+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39912 2023-10-21T04:45:24.982+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39912 2023-10-21T04:45:25.052+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 5932387718257253322 2023-10-21T04:45:25.052+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 343.37µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:25.138+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:25.138+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:25.140+05:30 [Info] switched currmeta from 152 -> 155 force true 2023-10-21T04:45:25.145+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:25.145+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:25.147+05:30 [Info] switched currmeta from 155 -> 155 force true 2023-10-21T04:45:25.166+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:25.167+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:25.170+05:30 [Info] switched currmeta from 155 -> 157 force true 2023-10-21T04:45:25.192+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:25.192+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:25.197+05:30 [Info] switched currmeta from 157 -> 159 force true 2023-10-21T04:45:25.204+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:25.204+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:25.217+05:30 [Info] switched currmeta from 159 -> 159 force true 2023-10-21T04:45:25.266+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:25.268+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:25.274+05:30 [Info] switched currmeta from 159 -> 161 force true 2023-10-21T04:45:25.422+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:25.422+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:25.435+05:30 [Info] switched currmeta from 161 -> 161 force true 2023-10-21T04:45:25.664+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:45:25.664+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:45:25.668+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55088 2023-10-21T04:45:25.668+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55090 2023-10-21T04:45:25.678+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:45:25.701+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 27.901176ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:25.716+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 42.537182ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:25.730+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:45:25.730+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:45:25.730+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:45:25.737+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55104 2023-10-21T04:45:25.737+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:45:25.757+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 17.769763ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:25.757+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:45:25.757+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:45:25.757+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:45:25.770+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:45:25.791+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 5932387718257253322. No update is performed. 2023-10-21T04:45:25.791+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 108.92µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:25.804+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:45:25.805+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:45:25.805+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:45:25.819+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55088. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:25.819+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6f:cc:ff:9f:83:b7:20:ff as follower. 2023-10-21T04:45:25.837+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:45:25.954+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:45:25.954+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:45:25.955+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:45:25.955+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:45:25.955+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:45:25.955+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:45:25.955+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:45:25.955+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:45:25.955+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:45:25.955+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:45:25.955+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:46804->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:25.955+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:55104->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:25.955+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:51480->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:25.955+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55104. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:25.956+05:30 [Info] messageListener.start(): message channel closed. Remove peer fc:10:bb:97:2c:6b:8c:9f as follower. 2023-10-21T04:45:25.957+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:46370->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:26.053+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55090. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:26.053+05:30 [Info] messageListener.start(): message channel closed. Remove peer aa:b9:65:59:39:2e:d7:d0 as follower. 2023-10-21T04:45:26.202+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:26.217+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:26.257+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:26.347+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55142 2023-10-21T04:45:26.380+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 27.627561ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:26.420+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55142. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:26.421+05:30 [Info] messageListener.start(): message channel closed. Remove peer 57:51:3c:2b:66:e6:22:2d as follower. 2023-10-21T04:45:26.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:26.629+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:26.643+05:30 [Info] switched currmeta from 161 -> 162 force true 2023-10-21T04:45:26.721+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:26.721+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:26.724+05:30 [Info] switched currmeta from 162 -> 163 force true 2023-10-21T04:45:26.881+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:27.269+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:27.269+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:27.272+05:30 [Info] switched currmeta from 163 -> 164 force true 2023-10-21T04:45:27.411+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:27.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:27.417+05:30 [Info] switched currmeta from 164 -> 165 force true 2023-10-21T04:45:28.365+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16448845122371869558:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:45:28.469+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2259746612664550306:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:45:28.916+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:45:28.928+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:28.928+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:28.930+05:30 [Info] switched currmeta from 165 -> 165 force true 2023-10-21T04:45:29.635+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 17476654447546601131 2023-10-21T04:45:29.635+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 136.896µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:29.695+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39912 2023-10-21T04:45:29.703+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39912 2023-10-21T04:45:29.745+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:39912 2023-10-21T04:45:29.800+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 17476654447546601131 2023-10-21T04:45:29.800+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 242.927µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:29.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:29.863+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:29.869+05:30 [Info] switched currmeta from 165 -> 168 force true 2023-10-21T04:45:29.875+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:29.875+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:29.877+05:30 [Info] switched currmeta from 168 -> 168 force true 2023-10-21T04:45:30.003+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:30.004+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:30.012+05:30 [Info] switched currmeta from 168 -> 173 force true 2023-10-21T04:45:30.083+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:30.083+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:30.107+05:30 [Info] switched currmeta from 173 -> 174 force true 2023-10-21T04:45:30.119+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:30.119+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:30.123+05:30 [Info] switched currmeta from 174 -> 174 force true 2023-10-21T04:45:30.380+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55280 2023-10-21T04:45:30.417+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 33.982247ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:30.481+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 17476654447546601131. No update is performed. 2023-10-21T04:45:30.482+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 169.065µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:30.500+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55280. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:30.500+05:30 [Info] messageListener.start(): message channel closed. Remove peer 60:df:54:20:5f:bb:34:ef as follower. 2023-10-21T04:45:30.654+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:45:30.654+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:45:30.670+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55306 2023-10-21T04:45:30.682+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:45:30.690+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55310 2023-10-21T04:45:30.698+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 20.935571ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:30.708+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:45:30.708+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:45:30.708+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:45:30.711+05:30 [Info] Periodic TenantStats: { "bucket_1_c25fee48adbe357b2d711be58a1deb67": { "num_shards": 0, "num_instances": 0, "num_controllers": 0, "quota": 0, "mandatory_quota": 0, "max_mandatory_quota": 0, "discretionary_quota": 0, "resident_quota": 0, "mutation_quota": 0, "max_mutation_quota": 0, "idle_quota": 0, "derived_quota": 0, "working_set": 0, "mem_in_use": 0, "mem_index": 0, "total_records": 0, "avg_key_size": 0, "avg_mem_per_key": 0, "resident_ratio": 0.00000, "idle": false, "total_ops": 0, "read_ops": 0, "new_ops": 0, "new_read_ops": 0, "compacts": 0, "compresses": 0, "cache_hits": 0, "cache_misses": 0, "cache_hit_ratio": 0.00000, "lookup_dur": "0s", "writer_dur": "0s", "writer_throttle_dur": "0s", "controller": { "build": [], "swap": [], "instances": [], "pages": [], "quota": [], "mem_in_use": [], "mem_used_global": [], "mandatory_quota": [], "discretionary_quota": [], "resident_quota": [], "derived_quota": [], "mutation_quota": [], "idle_quota": [], "new_evict_cycle": [], "new_evict_visit": [], "new_evicted": [], "swapper_instances": [], "sweep_duration": [], "swapper_pages": [], "swapper_cycles": [], "evict_visit": [], "evict_burst": [], "evict_periodic": [], "evictable": [], "evicted": [], "compressed": [], "resident": [] } } } 2023-10-21T04:45:30.722+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 29.431972ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:30.726+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:45:30.750+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:45:30.750+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:45:30.750+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:45:30.757+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55322 2023-10-21T04:45:30.758+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:45:30.763+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55310. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:30.763+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5e:40:65:1f:1f:c1:fd:eb as follower. 2023-10-21T04:45:30.784+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 18.461488ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:30.785+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:45:30.785+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:45:30.785+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:45:30.805+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:45:30.829+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:45:30.829+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:45:30.829+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:45:30.830+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:45:30.830+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:45:30.830+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:45:30.831+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:45:30.831+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:45:30.831+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:45:30.832+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:46560->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:30.831+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:51700->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:30.832+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:45:30.832+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:47026->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:30.833+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:55322->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:30.834+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55322. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:30.835+05:30 [Info] messageListener.start(): message channel closed. Remove peer 71:62:d5:d7:2f:77:f2:c1 as follower. 2023-10-21T04:45:30.858+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55306. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:30.858+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4f:69:2a:f1:42:ff:c1:ee as follower. 2023-10-21T04:45:30.917+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:31.200+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:31.223+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:31.285+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:31.510+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:31.510+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:31.514+05:30 [Info] switched currmeta from 174 -> 175 force true 2023-10-21T04:45:31.532+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:31.532+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:31.552+05:30 [Info] switched currmeta from 175 -> 176 force true 2023-10-21T04:45:31.627+05:30 [Info] DDLServiceMgr checking create token progress 2023-10-21T04:45:31.648+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9106 2023-10-21T04:45:31.648+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:45:31.655+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:45:31.677+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:45:31.677+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9112 2023-10-21T04:45:31.677+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:45:31.682+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:45:31.704+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:45:31.705+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9118 2023-10-21T04:45:31.705+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:45:31.713+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55362 2023-10-21T04:45:31.713+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:45:31.730+05:30 [Info] memstats {"Alloc":92142720, "TotalAlloc":311567400, "Sys":191925544, "Lookups":0, "Mallocs":2172160,"Frees":2078957, "HeapAlloc":92142720, "HeapSys":173932544, "HeapIdle":67928064, "HeapInuse":106004480,"HeapReleased":147456, "HeapObjects":93203,"MSpanInuse":502720, "MSpanSys": 1191360, "StackInuse": 2228224,"GCSys":12162216, "LastGC":1697843664929153893,"PauseTotalNs":3247834, "PauseNs":[1106895], "NumGC":8} 2023-10-21T04:45:31.730+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.891184ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:31.731+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:45:31.731+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9124 2023-10-21T04:45:31.731+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:45:31.736+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:45:31.741+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:45:31.763+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:45:31.763+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:45:31.763+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:45:31.763+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:45:31.763+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:45:31.788+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/10606939961365971246 2023-10-21T04:45:31.813+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/10761885902580101111 2023-10-21T04:45:31.848+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/11276338709617092948 2023-10-21T04:45:31.919+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/11365633301282701856 2023-10-21T04:45:31.962+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/11619165464560280276 2023-10-21T04:45:32.103+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12027054384338714804 2023-10-21T04:45:32.162+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12140458060295645238 2023-10-21T04:45:32.174+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:32.174+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:32.206+05:30 [Info] switched currmeta from 176 -> 177 force true 2023-10-21T04:45:32.227+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12195715103909237684 2023-10-21T04:45:32.232+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:32.348+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12338036531800130039 2023-10-21T04:45:32.395+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:32.395+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:32.439+05:30 [Info] switched currmeta from 177 -> 178 force true 2023-10-21T04:45:32.457+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/1252359852432998131 2023-10-21T04:45:32.727+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12582561312368048736 2023-10-21T04:45:32.733+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:45:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:45:32.787+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:45:32.813+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12599498416578472779 2023-10-21T04:45:32.864+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4904025283960472989:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:45:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:45:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:45:32.897+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12824340686309573428 2023-10-21T04:45:32.980+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12833670934079048728 2023-10-21T04:45:33.035+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/1301336828089581719 2023-10-21T04:45:33.058+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:45:33.075+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:45:33.101+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/13268712595133993155 2023-10-21T04:45:33.121+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12676698187551977291:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:45:33.121+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12676698187551977291:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:45:33.121+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12676698187551977291:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:45:33.121+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12676698187551977291:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:45:33.121+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12676698187551977291:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:45:33.121+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12676698187551977291:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:45:33.157+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/13856704970496601958 2023-10-21T04:45:33.216+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14292583894719865678 2023-10-21T04:45:33.271+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14549775068266281786 2023-10-21T04:45:33.326+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:45:33.332+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14741949230702299196 2023-10-21T04:45:33.333+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:45:33.347+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55438 2023-10-21T04:45:33.397+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14889401051926000176 2023-10-21T04:45:33.499+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15431181238515168303 2023-10-21T04:45:33.529+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 178.577619ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:33.605+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15945117906390072636 2023-10-21T04:45:33.643+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15968368593314043432 2023-10-21T04:45:33.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9823187000729126, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:45:33.689+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15973521466438086342 2023-10-21T04:45:33.726+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16162012140118277065 2023-10-21T04:45:33.812+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16337157693634799600 2023-10-21T04:45:33.931+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16483001637761719171 2023-10-21T04:45:33.996+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16706481854123898870 2023-10-21T04:45:34.030+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:34.110+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16961575949607441659 2023-10-21T04:45:34.182+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/17656144409930763141 2023-10-21T04:45:34.238+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/17797267771846799851 2023-10-21T04:45:34.298+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/17917116660857103172 2023-10-21T04:45:34.310+05:30 [Info] janitor: running cleanup. 2023-10-21T04:45:34.310+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[11619165464560280276 7045704526128519604 8378417662949994532 8669649359784409226 249887442666643345 14549775068266281786 7525085113233084253 17797267771846799851 14889401051926000176 12195715103909237684 12599498416578472779 4860549750398470299 4790214984019693894 1252359852432998131 12140458060295645238 3381524789655566900 14292583894719865678 4987105685507282285 8857269605176042330 6985049432225870291 17656144409930763141 15945117906390072636 5465835793930010953 4286621850367430051 16337157693634799600 15968368593314043432 4532813151040661705 13856704970496601958 12338036531800130039 16483001637761719171 4535197604682045916 16961575949607441659 18026660730857541767 16162012140118277065 17917116660857103172 3674078063353481907 7392796230534034390 2048006213723315482 14741949230702299196 11276338709617092948 15973521466438086342 18323460706027642575 12833670934079048728 8504686614261377811 10761885902580101111 18046371353728341033 15431181238515168303 12582561312368048736 16706481854123898870 10606939961365971246 13268712595133993155 8910459001587546151 573521267164877540 11365633301282701856 4138047918192025493 12824340686309573428 7147407900589480727 1301336828089581719 9665410041638025825 12027054384338714804]} 2023-10-21T04:45:34.401+05:30 [Info] Indexer::ReadMemstats Time Taken 2.693026ms 2023-10-21T04:45:34.497+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18026660730857541767 2023-10-21T04:45:34.577+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18046371353728341033 2023-10-21T04:45:34.632+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:45:34.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996998831031562, throttleDelayMs (new, old, change): (8, 3, 5) 2023-10-21T04:45:34.705+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18323460706027642575 2023-10-21T04:45:34.905+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/2048006213723315482 2023-10-21T04:45:35.039+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/249887442666643345 2023-10-21T04:45:35.074+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:35.074+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:35.097+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:45:35.178+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/3381524789655566900 2023-10-21T04:45:35.305+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/3674078063353481907 2023-10-21T04:45:35.366+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4138047918192025493 2023-10-21T04:45:35.400+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4286621850367430051 2023-10-21T04:45:35.503+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4532813151040661705 2023-10-21T04:45:35.606+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4535197604682045916 2023-10-21T04:45:35.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9998846584490497, throttleDelayMs (new, old, change): (10, 8, 2) 2023-10-21T04:45:35.786+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4790214984019693894 2023-10-21T04:45:35.963+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4860549750398470299 2023-10-21T04:45:36.054+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4987105685507282285 2023-10-21T04:45:36.165+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/5465835793930010953 2023-10-21T04:45:36.270+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/573521267164877540 2023-10-21T04:45:36.313+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/6985049432225870291 2023-10-21T04:45:36.398+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7045704526128519604 2023-10-21T04:45:36.527+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7147407900589480727 2023-10-21T04:45:36.587+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7392796230534034390 2023-10-21T04:45:36.611+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7525085113233084253 2023-10-21T04:45:36.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9999611274878408 2023-10-21T04:45:36.705+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8378417662949994532 2023-10-21T04:45:36.817+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8504686614261377811 2023-10-21T04:45:36.951+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8669649359784409226 2023-10-21T04:45:37.104+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8857269605176042330 2023-10-21T04:45:37.177+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8910459001587546151 2023-10-21T04:45:37.290+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/9665410041638025825 2023-10-21T04:45:37.379+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:45:37.379+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:45:37.379+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:45:37.379+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:45:37.379+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:45:37.379+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:46608->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:37.379+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:47068->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:37.379+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:55362->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:37.380+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:51738->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:37.380+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55362. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:37.380+05:30 [Info] messageListener.start(): message channel closed. Remove peer 9c:b4:ed:c8:47:16:44:e5 as follower. 2023-10-21T04:45:37.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9998676363170949 2023-10-21T04:45:38.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9994185590064473 2023-10-21T04:45:39.399+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55438. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:39.399+05:30 [Info] messageListener.start(): message channel closed. Remove peer 13:68:70:90:52:f:70:e2 as follower. 2023-10-21T04:45:39.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9992271056463178 2023-10-21T04:45:40.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8572980332314682, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:45:41.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6449693113491756, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:45:42.871+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:45:42.989+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:42.989+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:42.996+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:45:43.029+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:45:43.062+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:43.063+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:43.090+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:45:43.101+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:45:43.101+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:45:43.252+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:45:43.415+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:43.415+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:43.428+05:30 [Info] switched currmeta from 178 -> 178 force true 2023-10-21T04:45:44.618+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {ac5b1dc1660a058d991d594b1aa4a57f [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:45:44.620+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 7.191µs 2023-10-21T04:45:44.620+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:45:44.620+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:45:44.620+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:45:44.624+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:45:44.624+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:45:44.624+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 3.700105ms 2023-10-21T04:45:44.624+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:45:44.624+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:45:44.624+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 13] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:45:44.624+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 7.029µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:44.625+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 12] call. taskList: &{Rev:[0 0 0 0 0 0 0 13] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ac5b1dc1660a058d991d594b1aa4a57f Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ac5b1dc1660a058d991d594b1aa4a57f]}]} 2023-10-21T04:45:44.632+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 13] 2023-10-21T04:45:44.634+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 13] 2023-10-21T04:45:44.647+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {ac5b1dc1660a058d991d594b1aa4a57f [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:45:44.650+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:45:44.681+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"ac5b1dc1660a058d991d594b1aa4a57f","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:45:44.683+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{4325b30e7f9620a998388078247335a8 ac5b1dc1660a058d991d594b1aa4a57f Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:45:44.686+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 ac5b1dc1660a058d991d594b1aa4a57f Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:45:45.699+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:45:45.710+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:45:45.710+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:45:45.715+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:45:45.715+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{4325b30e7f9620a998388078247335a8 ac5b1dc1660a058d991d594b1aa4a57f Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:45:45.715+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:45:45.715+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.068292924s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:45:45.715+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{4325b30e7f9620a998388078247335a8 ac5b1dc1660a058d991d594b1aa4a57f Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:45:45.716+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 13] call. taskList: &{Rev:[0 0 0 0 0 0 0 14] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ac5b1dc1660a058d991d594b1aa4a57f Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ac5b1dc1660a058d991d594b1aa4a57f]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/ac5b1dc1660a058d991d594b1aa4a57f Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ac5b1dc1660a058d991d594b1aa4a57f]}]} 2023-10-21T04:45:45.716+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 14] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:45:45.716+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:45:45.716+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:45:45.716+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 14] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ac5b1dc1660a058d991d594b1aa4a57f Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ac5b1dc1660a058d991d594b1aa4a57f]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/ac5b1dc1660a058d991d594b1aa4a57f Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ac5b1dc1660a058d991d594b1aa4a57f]}]} 2023-10-21T04:45:45.717+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:45:45.717+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 14] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:45:45.720+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 14] 2023-10-21T04:45:45.720+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 14] 2023-10-21T04:45:45.765+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{d7ff2bfd429b21d9c24d822bc13e3acc d7ff2bfd429b21d9c24d822bc13e3acc plasma map[] [{45 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{9316375249999768625 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{3751526798537645569 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 0 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 5932387718257253322 [{16448845122371869558 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563 5748926106587204814] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{18137927630988528516 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [11114961608238406563] []}] 1 0 0 1 false plasma 0}]}]} {45 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{12606056968936758026 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{8303154871847919944 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 17476654447546601131 [{4904025283960472989 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{17465522551378543576 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5932387718257253322 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {9b27839d59b06374125d80a2b12a21b6 9b27839d59b06374125d80a2b12a21b6 plasma map[] [{45 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{16759912838651900043 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{4186965058349029465 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 0 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 5932387718257253322 [{2259746612664550306 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215 5629943257627822009] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{544693394183450628 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [17327901024118572215] []}] 1 0 0 0 false plasma 0}]}]} {45 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{4397940545762329850 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{12396348969790516872 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 17476654447546601131 [{12676698187551977291 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{10158339249566909515 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5932387718257253322 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {4325b30e7f9620a998388078247335a8 4325b30e7f9620a998388078247335a8 plasma map[] [{12 bucket_%2 _default c1 []}] [] 0 0 0 false} {ba6304988a6335d65281381d6cfc5e32 ba6304988a6335d65281381d6cfc5e32 plasma map[] [{3 bucket_1 _system _query []} {7 bucket_%2 _system _query []}] [] 0 0 0 false}] map[]} 2023-10-21T04:45:45.822+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:45:45.828+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:45:45.829+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:45:45.829+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:45:45.829+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:45:45.831+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:45:45.869+05:30 [Info] Planner::processDeleteToken: processing 60 tokens for node 127.0.0.1:9004 2023-10-21T04:45:45.869+05:30 [Info] Planner::processDeleteToken: processing 60 tokens for node 127.0.0.1:9002 2023-10-21T04:45:45.869+05:30 [Info] Planner::processDeleteToken: processing 60 tokens for node 127.0.0.1:9003 2023-10-21T04:45:45.869+05:30 [Info] Planner::processDeleteToken: processing 60 tokens for node 127.0.0.1:9001 2023-10-21T04:45:45.870+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:45:45.879+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:45:45.879+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:45:45.879+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:45:45.879+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:45:45.971+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:45:45.971+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:45:45.971+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [127.0.0.1:9004 127.0.0.1:9003] 2023-10-21T04:45:45.971+05:30 [Info] Planner::moveTenantsFromDeletedNodes selected newNodes for swap [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17476654447546601131:12676698187551977291:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 5277316367626246544:10158339249566909515:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 4390029557575378783:4397940545762329850:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 17703248484023971553:12396348969790516872:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 5932387718257253322:2259746612664550306:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 668551233566132751:544693394183450628:0 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 7790129686881525591:16759912838651900043:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:1 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:2 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.971+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:3 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:4 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:5 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:6 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:7 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:4186965058349029465:8 from source 127.0.0.1:9002 to dest 127.0.0.1:9003 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 4390029557575378783:12606056968936758026:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17703248484023971553:8303154871847919944:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 5932387718257253322:16448845122371869558:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 668551233566132751:18137927630988528516:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 7790129686881525591:9316375249999768625:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:1 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:2 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:3 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:4 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:5 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:6 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:7 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 16823504650100619022:3751526798537645569:8 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 17476654447546601131:4904025283960472989:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Moving index 5277316367626246544:17465522551378543576:0 from source 127.0.0.1:9001 to dest 127.0.0.1:9004 2023-10-21T04:45:45.972+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 17715624 Units 0 2023-10-21T04:45:45.972+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 17717762 Units 0 2023-10-21T04:45:45.972+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 283616 Units 16 Tenants 0 2023-10-21T04:45:45.972+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 285850 Units 8 Tenants 0 2023-10-21T04:45:45.972+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9004 127.0.0.1:9003]] 2023-10-21T04:45:45.972+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:45:45.972+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:45:45.972+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:45:45.973+05:30 [Info] ---------------------------------------- 2023-10-21T04:45:45.973+05:30 [Info] 2023-10-21T04:45:45.973+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:ba6304988a6335d65281381d6cfc5e32, useLiveData:false 2023-10-21T04:45:45.973+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:45:45.973+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.973+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.973+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.973+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.973+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:5932387718257253322, instId:16448845122371869558, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:18137927630988528516, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157294577 (150.008M), mem:7080 (6.91406K), overhead:157287497 (150.001M), min:157294577 (150.008M), data:7080 (6.91406K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.974+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.974+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.974+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.974+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.975+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.975+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.975+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.975+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.976+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:45:45.976+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.976+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.976+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.976+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:17476654447546601131, instId:4904025283960472989, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:45:45.976+05:30 [Info] Index total memory:157401900 (150.11M), mem:100000 (97.6562K), overhead:157301900 (150.015M), min:157401900 (150.11M), data:100000 (97.6562K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.976+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.976+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.976+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:17465522551378543576, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501], shardIdsInDefn: map[] 2023-10-21T04:45:45.976+05:30 [Info] Index total memory:157293156 (150.006M), mem:5850 (5.71289K), overhead:157287306 (150.001M), min:157293156 (150.006M), data:5850 (5.71289K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.976+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.976+05:30 [Info] 2023-10-21T04:45:45.976+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:4325b30e7f9620a998388078247335a8, useLiveData:false 2023-10-21T04:45:45.976+05:30 [Info] Indexer total memory:104857600 (100M), mem:0 (0), overhead:104857600 (100M), min:104857600 (100M), data:0 (0) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:45:45.976+05:30 [Info] Indexer isDeleted:false isNew:true exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:45:45.976+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.976+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17476654447546601131, instId:12676698187551977291, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.976+05:30 [Info] Index total memory:157401900 (150.11M), mem:100000 (97.6562K), overhead:157301900 (150.015M), min:157401900 (150.11M), data:100000 (97.6562K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.976+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.977+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.977+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:10158339249566909515, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589], shardIdsInDefn: map[] 2023-10-21T04:45:45.977+05:30 [Info] Index total memory:157293156 (150.006M), mem:5850 (5.71289K), overhead:157287306 (150.001M), min:157293156 (150.006M), data:5850 (5.71289K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.977+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.977+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.977+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.977+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.977+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.977+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.977+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.977+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.977+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.977+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.977+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.977+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.977+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.978+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.978+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.978+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.978+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.978+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.978+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.978+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.978+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.978+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.978+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.978+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.978+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.978+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.978+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.978+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.978+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.978+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.978+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.978+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.978+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.978+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.978+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.978+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.978+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.978+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.978+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.979+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.979+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.979+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.979+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.979+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.979+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.979+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.979+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.979+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.979+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.979+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.979+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.979+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.979+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.979+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.979+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.979+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.979+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.979+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.979+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.979+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.979+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.979+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.979+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:45:45.979+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.979+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.979+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.980+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:5932387718257253322, instId:2259746612664550306, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.980+05:30 [Info] Index total memory:157401900 (150.11M), mem:100000 (97.6562K), overhead:157301900 (150.015M), min:157401900 (150.11M), data:100000 (97.6562K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.980+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.980+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.980+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:544693394183450628, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215], shardIdsInDefn: map[] 2023-10-21T04:45:45.980+05:30 [Info] Index total memory:157293219 (150.007M), mem:5904 (5.76562K), overhead:157287315 (150.001M), min:157293219 (150.007M), data:5904 (5.76562K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.980+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.980+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.980+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.980+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.980+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.980+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.980+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.980+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.980+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.980+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.981+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.981+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.981+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.981+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.981+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.981+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.981+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.982+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.982+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.982+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.983+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.983+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.983+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.984+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.984+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.984+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.984+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.984+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.984+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.984+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.984+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.984+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.984+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.984+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.984+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.984+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.984+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.984+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.984+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.984+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.984+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.984+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.984+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.984+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.984+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.984+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.984+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.985+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.985+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.985+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.985+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.985+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.985+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.985+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.985+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.985+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.985+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.985+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.985+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.985+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.985+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.985+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:45:45.985+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:45:45.985+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:45:45.985+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:45:45.985+05:30 [Info] **************************************** 2023-10-21T04:45:45.986+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 283616 Units 16 Tenants 0 2023-10-21T04:45:45.986+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 285850 Units 8 Tenants 0 2023-10-21T04:45:45.986+05:30 [Info] Generating Shard Transfer Token (ShardTokenf:47:e3:1a:84:63:c:be) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: Region: InstId: 12676698187551977291 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 12676698187551977291 Defn: DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 2203301915252658598 RealInstId: 4397940545762329850 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 1863605407862672237 RealInstId: 12396348969790516872 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 10158339249566909515 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:45:45.986+05:30 [Info] Generating Shard Transfer Token (ShardTokene4:a6:e2:fc:d5:0:3f:66) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:79:d9:66:3b:76:f4:db Shards: [17327901024118572215 5629943257627822009] Destination: Region: InstId: 2259746612664550306 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2259746612664550306 Defn: DefnId: 5932387718257253322 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 9874972326106273392 RealInstId: 16759912838651900043 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 4743233249208554426 RealInstId: 4186965058349029465 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 544693394183450628 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:45:45.987+05:30 [Info] Generating Shard Transfer Token (ShardTokenf6:1a:ee:e4:7f:54:47:61) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf:47:e3:1a:84:63:c:be Shards: [10838601320854145501 11711860383628641160] Destination: Region: InstId: 1477850144207935465 RealInstId: 12606056968936758026 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 5672312913771963066 RealInstId: 8303154871847919944 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 4904025283960472989 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 4904025283960472989 Defn: DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17465522551378543576 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:45:45.988+05:30 [Info] Generating Shard Transfer Token (ShardToken9e:79:d9:66:3b:76:f4:db) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Shards: [11114961608238406563 5748926106587204814] Destination: Region: InstId: 16448845122371869558 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 16448845122371869558 Defn: DefnId: 5932387718257253322 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 18044291032990858536 RealInstId: 9316375249999768625 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 9316375249999768625 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 11368954339675856354 RealInstId: 3751526798537645569 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 3751526798537645569 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 18137927630988528516 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 18137927630988528516 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:45:45.989+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:45:45.989+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:45:45.993+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken9e:79:d9:66:3b:76:f4:db 2023-10-21T04:45:45.999+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenf:47:e3:1a:84:63:c:be 2023-10-21T04:45:46.016+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokene4:a6:e2:fc:d5:0:3f:66 2023-10-21T04:45:46.039+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenf6:1a:ee:e4:7f:54:47:61 2023-10-21T04:45:46.039+05:30 [Info] ShardRebalancer::observeRebalance &{4325b30e7f9620a998388078247335a8 ac5b1dc1660a058d991d594b1aa4a57f Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:45:46.052+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf6:1a:ee:e4:7f:54:47:61 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf:47:e3:1a:84:63:c:be Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1477850144207935465 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5672312913771963066 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4904025283960472989 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.054+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene4:a6:e2:fc:d5:0:3f:66 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:79:d9:66:3b:76:f4:db Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2259746612664550306 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9874972326106273392 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4743233249208554426 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.063+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf:47:e3:1a:84:63:c:be MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12676698187551977291 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2203301915252658598 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1863605407862672237 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.065+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:79:d9:66:3b:76:f4:db MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16448845122371869558 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18044291032990858536 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11368954339675856354 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.067+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"ac5b1dc1660a058d991d594b1aa4a57f","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:45:46.067+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:45:46.105+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:79:d9:66:3b:76:f4:db MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16448845122371869558 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18044291032990858536 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11368954339675856354 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.150+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:79:d9:66:3b:76:f4:db MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16448845122371869558 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18044291032990858536 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11368954339675856354 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.157+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:45:46.204+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf6:1a:ee:e4:7f:54:47:61 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf:47:e3:1a:84:63:c:be Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1477850144207935465 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5672312913771963066 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4904025283960472989 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.246+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf:47:e3:1a:84:63:c:be MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12676698187551977291 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2203301915252658598 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1863605407862672237 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.247+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:45:46.247+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:45:46.247+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:45:46.247+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.983µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:46.301+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf6:1a:ee:e4:7f:54:47:61 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf:47:e3:1a:84:63:c:be Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1477850144207935465 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5672312913771963066 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4904025283960472989 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.336+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:45:46.336+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf:47:e3:1a:84:63:c:be MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12676698187551977291 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2203301915252658598 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1863605407862672237 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.341+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:45:46.343+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene4:a6:e2:fc:d5:0:3f:66 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:79:d9:66:3b:76:f4:db Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2259746612664550306 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9874972326106273392 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4743233249208554426 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.344+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:45:46.344+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:45:46.344+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:45:46.344+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.445µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:46.369+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene4:a6:e2:fc:d5:0:3f:66 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:79:d9:66:3b:76:f4:db Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2259746612664550306 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9874972326106273392 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4743233249208554426 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.373+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:45:46.373+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:45:46.381+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 ac5b1dc1660a058d991d594b1aa4a57f Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:45:46.381+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 1 2023-10-21T04:45:46.404+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[9b27839d59b06374125d80a2b12a21b6:ShardTokenf:47:e3:1a:84:63:c:be d7ff2bfd429b21d9c24d822bc13e3acc:ShardTokenf6:1a:ee:e4:7f:54:47:61], currActiveTransfers: map[9b27839d59b06374125d80a2b12a21b6:1 d7ff2bfd429b21d9c24d822bc13e3acc:1] 2023-10-21T04:45:46.404+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"ac5b1dc1660a058d991d594b1aa4a57f","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:45:46.404+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:45:46.404+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:45:46.404+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:45:46.404+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:45:46.404+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:45:46.405+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf6:1a:ee:e4:7f:54:47:61 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf:47:e3:1a:84:63:c:be Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1477850144207935465 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5672312913771963066 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4904025283960472989 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.405+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf:47:e3:1a:84:63:c:be MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12676698187551977291 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2203301915252658598 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1863605407862672237 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.405+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 17.201µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:46.555+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf6:1a:ee:e4:7f:54:47:61 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf:47:e3:1a:84:63:c:be Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1477850144207935465 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5672312913771963066 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4904025283960472989 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.555+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:45:46.706+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf6:1a:ee:e4:7f:54:47:61 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf:47:e3:1a:84:63:c:be Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1477850144207935465 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5672312913771963066 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4904025283960472989 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.707+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:45:46.764+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf:47:e3:1a:84:63:c:be MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12676698187551977291 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2203301915252658598 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1863605407862672237 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.764+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:45:46.795+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[9b27839d59b06374125d80a2b12a21b6:ShardTokene4:a6:e2:fc:d5:0:3f:66 d7ff2bfd429b21d9c24d822bc13e3acc:ShardToken9e:79:d9:66:3b:76:f4:db], currActiveTransfers: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:1] 2023-10-21T04:45:46.796+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene4:a6:e2:fc:d5:0:3f:66 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:79:d9:66:3b:76:f4:db Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2259746612664550306 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9874972326106273392 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4743233249208554426 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.797+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:79:d9:66:3b:76:f4:db MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16448845122371869558 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18044291032990858536 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11368954339675856354 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.797+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:45:46.797+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:45:46.797+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:45:46.797+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 29.245µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:46.797+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[6455624821495495854:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/6455624821495495854/shards/shard6455624821495495854 18374611443167983589:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/18374611443167983589/shards/shard18374611443167983589] ac5b1dc1660a058d991d594b1aa4a57f ShardTokenf:47:e3:1a:84:63:c:be /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00805eb40 0xc00805eba0 map[] 0xc006fbb980 0xc0085fba40 false} 2023-10-21T04:45:46.797+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/6455624821495495854/shards/shard6455624821495495854 2023-10-21T04:45:46.798+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/18374611443167983589/shards/shard18374611443167983589 2023-10-21T04:45:46.802+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/18374611443167983589/shards/shard18374611443167983589/data: super block snapshot head offset :0 tail offset:1019904 sbgen :152 2023-10-21T04:45:46.802+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/18374611443167983589/shards/shard18374611443167983589/data/recovery: super block snapshot head offset :0 tail offset:913408 sbgen :152 2023-10-21T04:45:46.807+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/6455624821495495854/shards/shard6455624821495495854/data: super block snapshot head offset :0 tail offset:872448 sbgen :139 2023-10-21T04:45:46.807+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/6455624821495495854/shards/shard6455624821495495854/data/recovery: super block snapshot head offset :0 tail offset:737280 sbgen :139 2023-10-21T04:45:46.810+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/18374611443167983589/shards/shard18374611443167983589: metadata saved successfully 2023-10-21T04:45:46.811+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) added 2023-10-21T04:45:46.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:45:46.811+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 150199692 totalBytes: 1952596 numShards: 2 duration: 13.5969ms 2023-10-21T04:45:46.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/18374611443167983589/shards/shard18374611443167983589, newshardId 18374611443167983589, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, shardsize 1952596, duration 12.793742ms 2023-10-21T04:45:46.811+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 18374611443167983589, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, err: , elapsed(sec): 0.013861832 2023-10-21T04:45:46.811+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 18374611443167983589 bytesWritten: 1952596, totalBytes: 1952596, transferRate: 1.6271633333333334e+08 2023-10-21T04:45:46.812+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/6455624821495495854/shards/shard6455624821495495854: metadata saved successfully 2023-10-21T04:45:46.812+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) added 2023-10-21T04:45:46.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:45:46.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be/6455624821495495854/shards/shard6455624821495495854, newshardId 6455624821495495854, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, shardsize 1628849, duration 15.417145ms 2023-10-21T04:45:46.813+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 6455624821495495854, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, err: , elapsed(sec): 0.015710805 2023-10-21T04:45:46.813+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 6455624821495495854 bytesWritten: 1628849, totalBytes: 1628849, transferRate: 1.0858993333333333e+08 2023-10-21T04:45:46.813+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: ac5b1dc1660a058d991d594b1aa4a57f, transferId: ShardTokenf:47:e3:1a:84:63:c:be 2023-10-21T04:45:46.813+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: ac5b1dc1660a058d991d594b1aa4a57f, transferId: ShardTokenf:47:e3:1a:84:63:c:be, elapsed(sec): 0.000107555 2023-10-21T04:45:46.813+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[6455624821495495854: 18374611443167983589:], shardPaths: map[6455624821495495854:/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 18374611443167983589:/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.015873228 2023-10-21T04:45:46.813+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [18374611443167983589 6455624821495495854], ttid: ShardTokenf:47:e3:1a:84:63:c:be, elapsed(sec): 0.01622332 2023-10-21T04:45:46.861+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:45:46.882+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf:47:e3:1a:84:63:c:be MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12676698187551977291 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2203301915252658598 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1863605407862672237 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:46.887+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:1], batchSize: 1 2023-10-21T04:45:46.887+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardTokenf:47:e3:1a:84:63:c:be 2023-10-21T04:45:46.887+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[6455624821495495854:true 18374611443167983589:true] 2023-10-21T04:45:46.887+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.749µs 2023-10-21T04:45:46.887+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [18374611443167983589 6455624821495495854], isLockedForRecovery: true 2023-10-21T04:45:46.887+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [18374611443167983589 6455624821495495854], errMap: map[6455624821495495854: 18374611443167983589:], elapsed: 10.641µs 2023-10-21T04:45:46.888+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 17476654447546601131, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:45:46.888+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 17476654447546601131, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:45:46.959+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:46.959+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:46.962+05:30 [Info] switched currmeta from 178 -> 179 force true 2023-10-21T04:45:47.000+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12676698187551977291,realInstId: 0, indexDefn DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:45:47.001+05:30 [Info] Indexer::handleRecoverIndex InstId: 12676698187551977291 Defn: DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:45:47.001+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 365.014µs 2023-10-21T04:45:47.001+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 410.863µs 2023-10-21T04:45:47.001+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12676698187551977291, indexDefn DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:45:47.001+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 113.951806ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:47.001+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12676698187551977291 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[18374611443167983589 6455624821495495854]] 2023-10-21T04:45:47.003+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12676698187551977291 PartitionId: 0 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:45:47.004+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': added 2023-10-21T04:45:47.005+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCleaner initialized 2023-10-21T04:45:47.005+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:45:47.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:45:47.006+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCleaner initialized for recovery 2023-10-21T04:45:47.006+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:45:47.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery 2023-10-21T04:45:47.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:45:47.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true] 2023-10-21T04:45:47.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[913408] Data log Head [0] Tail [1019904] 2023-10-21T04:45:47.006+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [913408] 2023-10-21T04:45:47.008+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.008+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:45:47.008+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:45:47.008+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.008+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:45:47.008+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.008+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:45:47.008+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.008+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:45:47.009+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCleaner initialized 2023-10-21T04:45:47.009+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:45:47.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:45:47.009+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.009+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:45:47.010+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:45:47.010+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:45:47.010+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.011+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:45:47.011+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.011+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:45:47.012+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.012+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:45:47.012+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.012+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:45:47.012+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.013+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:45:47.013+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.013+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:45:47.013+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.013+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:45:47.013+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.014+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:45:47.015+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.015+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:45:47.014+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCleaner initialized for recovery 2023-10-21T04:45:47.016+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:45:47.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery 2023-10-21T04:45:47.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:45:47.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true] 2023-10-21T04:45:47.017+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.018+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:45:47.020+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.020+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex 2023-10-21T04:45:47.020+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : replayOffset: 1015808 (RP:991232) (CP:1015808) 2023-10-21T04:45:47.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1015808] took [17.88526ms] 2023-10-21T04:45:47.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[737280] Data log Head [0] Tail [872448] 2023-10-21T04:45:47.025+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [737280] 2023-10-21T04:45:47.024+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870818 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870774 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1059076, resident 1, memSz 70546, memSzIndex 351, numRecords 1000 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870774 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870730 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870730 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870686 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870642 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870642 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870598 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1055476, resident 1, memSz 1445, memSzIndex 0, numRecords 12 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870554 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.026+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27870510 controller mem 1486 2023-10-21T04:45:47.026+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1019904] replayOffset [1015808] 2023-10-21T04:45:47.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [20.653849ms] 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true]. Built [18] plasmas, took [21.349544ms] 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: after recovery: resident ratio 0 requestQuota 1059076 memSz 910 2023-10-21T04:45:47.027+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 15 count: 1000 metaSz: 2505 2023-10-21T04:45:47.028+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: doInit: data UsedSpace 110509 recovery UsedSpace 14206 2023-10-21T04:45:47.040+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex 2023-10-21T04:45:47.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:45:47.058+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:45:47.058+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper started 2023-10-21T04:45:47.062+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:45:47.062+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex 2023-10-21T04:45:47.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex started 2023-10-21T04:45:47.062+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:45:47.063+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:45:47.067+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.079+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:45:47.082+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:45:47.082+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.083+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:45:47.083+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.083+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:45:47.083+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.083+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:45:47.084+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.084+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:45:47.084+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.084+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:45:47.085+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.085+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:45:47.085+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.112+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:45:47.112+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.112+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:45:47.113+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.113+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:45:47.113+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.141+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:45:47.135+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:79:d9:66:3b:76:f4:db MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16448845122371869558 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18044291032990858536 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11368954339675856354 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:47.141+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:1 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:45:47.141+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.142+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:45:47.144+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.144+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:45:47.145+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.146+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:45:47.146+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.147+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:45:47.148+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:45:47.148+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex 2023-10-21T04:45:47.149+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : replayOffset: 868352 (RP:831488) (CP:868352) 2023-10-21T04:45:47.149+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [868352] took [124.320248ms] 2023-10-21T04:45:47.149+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.149+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654390 controller mem 4038 2023-10-21T04:45:47.149+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654346 controller mem 4038 2023-10-21T04:45:47.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654346 controller mem 4038 2023-10-21T04:45:47.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654346 controller mem 4038 2023-10-21T04:45:47.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.150+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654346 controller mem 4038 2023-10-21T04:45:47.180+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1056168, resident 1, memSz 68068, memSzIndex 872, numRecords 1000 2023-10-21T04:45:47.181+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654346 controller mem 4038 2023-10-21T04:45:47.181+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.180+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokene4:a6:e2:fc:d5:0:3f:66 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:79:d9:66:3b:76:f4:db Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2259746612664550306 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9874972326106273392 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4743233249208554426 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:47.181+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654302 controller mem 4038 2023-10-21T04:45:47.183+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.188+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654258 controller mem 4038 2023-10-21T04:45:47.183+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654258 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654258 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654258 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654258 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654214 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654170 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654126 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654082 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44654082 controller mem 4038 2023-10-21T04:45:47.189+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [872448] replayOffset [868352] 2023-10-21T04:45:47.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [164.703759ms] 2023-10-21T04:45:47.189+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:45:47.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true]. Built [17] plasmas, took [165.421082ms] 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: after recovery: resident ratio 0 requestQuota 1056168 memSz 2040 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 15 count: 1000 metaSz: 2505 2023-10-21T04:45:47.190+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: doInit: data UsedSpace 104054 recovery UsedSpace 16579 2023-10-21T04:45:47.191+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:45:47.191+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:45:47.191+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:45:47.191+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 31.212µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:47.191+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[5629943257627822009:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/5629943257627822009/shards/shard5629943257627822009 17327901024118572215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/17327901024118572215/shards/shard17327901024118572215] ac5b1dc1660a058d991d594b1aa4a57f ShardTokene4:a6:e2:fc:d5:0:3f:66 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00805eb40 0xc00805eba0 map[] 0xc009049260 0xc009078000 false} 2023-10-21T04:45:47.191+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/5629943257627822009/shards/shard5629943257627822009 2023-10-21T04:45:47.192+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/17327901024118572215/shards/shard17327901024118572215 2023-10-21T04:45:47.196+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/17327901024118572215/shards/shard17327901024118572215/data: super block snapshot head offset :0 tail offset:1007616 sbgen :152 2023-10-21T04:45:47.196+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/17327901024118572215/shards/shard17327901024118572215/data/recovery: super block snapshot head offset :0 tail offset:905216 sbgen :152 2023-10-21T04:45:47.197+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/5629943257627822009/shards/shard5629943257627822009/data: super block snapshot head offset :0 tail offset:868352 sbgen :139 2023-10-21T04:45:47.197+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/5629943257627822009/shards/shard5629943257627822009/data/recovery: super block snapshot head offset :0 tail offset:745472 sbgen :139 2023-10-21T04:45:47.205+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/17327901024118572215/shards/shard17327901024118572215: metadata saved successfully 2023-10-21T04:45:47.205+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/5629943257627822009/shards/shard5629943257627822009: metadata saved successfully 2023-10-21T04:45:47.206+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) added 2023-10-21T04:45:47.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:45:47.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/17327901024118572215/shards/shard17327901024118572215, newshardId 17327901024118572215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, shardsize 1932073, duration 14.306177ms 2023-10-21T04:45:47.206+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17327901024118572215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, err: , elapsed(sec): 0.015256692 2023-10-21T04:45:47.206+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17327901024118572215 bytesWritten: 1932073, totalBytes: 1932073, transferRate: 1.380052142857143e+08 2023-10-21T04:45:47.206+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:45:47.206+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 14] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ac5b1dc1660a058d991d594b1aa4a57f Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ac5b1dc1660a058d991d594b1aa4a57f]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/ac5b1dc1660a058d991d594b1aa4a57f Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ac5b1dc1660a058d991d594b1aa4a57f]}]} 2023-10-21T04:45:47.207+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) added 2023-10-21T04:45:47.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:45:47.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66/5629943257627822009/shards/shard5629943257627822009, newshardId 5629943257627822009, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, shardsize 1632908, duration 16.354415ms 2023-10-21T04:45:47.208+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 5629943257627822009, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, err: , elapsed(sec): 0.016670522 2023-10-21T04:45:47.208+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: ac5b1dc1660a058d991d594b1aa4a57f, transferId: ShardTokene4:a6:e2:fc:d5:0:3f:66 2023-10-21T04:45:47.208+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: ac5b1dc1660a058d991d594b1aa4a57f, transferId: ShardTokene4:a6:e2:fc:d5:0:3f:66, elapsed(sec): 0.00011932 2023-10-21T04:45:47.208+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[5629943257627822009: 17327901024118572215:], shardPaths: map[5629943257627822009:/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 17327901024118572215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.016851525 2023-10-21T04:45:47.208+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 5629943257627822009 bytesWritten: 1632908, totalBytes: 1632908, transferRate: 1.0205675e+08 2023-10-21T04:45:47.208+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17327901024118572215 5629943257627822009], ttid: ShardTokene4:a6:e2:fc:d5:0:3f:66, elapsed(sec): 0.017076597 2023-10-21T04:45:47.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 21 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex 2023-10-21T04:45:47.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:45:47.208+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:45:47.208+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex 2023-10-21T04:45:47.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex started 2023-10-21T04:45:47.209+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12676698187551977291 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:45:47.209+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:45:47.209+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:45:47.224+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12676698187551977291 PartitionId 0 Warmup took 220.097869ms 2023-10-21T04:45:47.224+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12676698187551977291 partitionId 0 WriterThreads 4 2023-10-21T04:45:47.224+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12676698187551977291 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index Index: 12676698187551977291 Partition: 0 2023-10-21T04:45:47.224+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12676698187551977291, shardIdMap: map[0:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:45:47.224+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12676698187551977291, partnShardIdMap: map[0:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:45:47.224+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12676698187551977291 Defn: DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:45:47.225+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12676698187551977291 2023-10-21T04:45:47.225+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12676698187551977291 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:45:47.226+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:45:47.226+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:45:47.226+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12676698187551977291 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:45:47.226+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12676698187551977291 Defn: DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:45:47.227+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12676698187551977291 Partitions [0] 2023-10-21T04:45:47.227+05:30 [Info] StorageMgr::openSnapshot IndexInst:12676698187551977291 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:45:47.232+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.397608ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:47.232+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 74.981µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:45:47.237+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:47.237+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:47.239+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 5, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:45:47.261+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken9e:79:d9:66:3b:76:f4:db MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16448845122371869558 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18044291032990858536 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 11368954339675856354 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:47.262+05:30 [Info] switched currmeta from 179 -> 181 force true 2023-10-21T04:45:47.265+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:45:47.265+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 14] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:45:47.274+05:30 [Info] GenericServiceManager::CancelTask: called with id: rebalance/ac5b1dc1660a058d991d594b1aa4a57f, rev: [] 2023-10-21T04:45:47.274+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: rebalance/ac5b1dc1660a058d991d594b1aa4a57f, rev: [] 2023-10-21T04:45:47.274+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:45:47.274+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:45:47.275+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress cancel received. progress collector stopped 2023-10-21T04:45:47.275+05:30 [Info] ShardRebalancer::updateProgress Cancel Received. progress reporter stapped 2023-10-21T04:45:47.285+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:45:47.286+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:45:47.549+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:47.549+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:47.583+05:30 [Info] switched currmeta from 181 -> 182 force true 2023-10-21T04:45:47.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9591415619581672, throttleDelayMs (new, old, change): (1, 0, 1) 2023-10-21T04:45:47.712+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:47.713+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:47.729+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:45:47.729+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:45:47.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:47.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:47.791+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:45:47.826+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:45:47.870+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:45:47.890+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:47.890+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:47.993+05:30 [Info] switched currmeta from 182 -> 182 force true 2023-10-21T04:45:48.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:48.462+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:48.466+05:30 [Info] switched currmeta from 182 -> 184 force true 2023-10-21T04:45:48.485+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:48.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:48.489+05:30 [Info] switched currmeta from 184 -> 184 force true 2023-10-21T04:45:48.504+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:45:48.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9993420539418257, throttleDelayMs (new, old, change): (6, 1, 5) 2023-10-21T04:45:48.710+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:45:49.004+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:45:49.009+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:49.009+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:49.011+05:30 [Info] switched currmeta from 184 -> 184 force true 2023-10-21T04:45:49.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7112685315160749, throttleDelayMs (new, old, change): (1, 6, -5) 2023-10-21T04:45:49.722+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:45:50.172+05:30 [Info] ShardRebalancer::waitForIndexState Cancel Received 2023-10-21T04:45:50.178+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:45:50.193+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {4325b30e7f9620a998388078247335a8 ac5b1dc1660a058d991d594b1aa4a57f Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:45:50.231+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:45:50.231+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenf:47:e3:1a:84:63:c:be MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardTokenf6:1a:ee:e4:7f:54:47:61 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 12676698187551977291 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2203301915252658598 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 1863605407862672237 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:50.231+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: ac5b1dc1660a058d991d594b1aa4a57f TransferTokenId: ShardTokenf:47:e3:1a:84:63:c:be Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:45:50.231+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000114024 2023-10-21T04:45:50.231+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenf:47:e3:1a:84:63:c:be, dropIndexes: true 2023-10-21T04:45:50.232+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be, shardId :0 2023-10-21T04:45:50.232+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokenf_47_e3_1a_84_63_c_be, shardId :0 2023-10-21T04:45:50.244+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12676698187551977291 &{1} 2023-10-21T04:45:50.244+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12676698187551977291 2023-10-21T04:45:50.244+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17476654447546601131, instId: 12676698187551977291 2023-10-21T04:45:50.244+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12676698187551977291] 2023-10-21T04:45:50.244+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12676698187551977291] 2023-10-21T04:45:50.244+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:45:50.244+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12676698187551977291] 2023-10-21T04:45:50.244+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 12676698187551977291 Defn: DefnId: 17476654447546601131 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:45:50.244+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 686.414µs 2023-10-21T04:45:50.244+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 734.433µs 2023-10-21T04:45:50.245+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12676698187551977291, PartitionId 0, IndexDefnId 17476654447546601131. 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:45:50.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex] 2023-10-21T04:45:50.245+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex] 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex] 2023-10-21T04:45:50.245+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex 2023-10-21T04:45:50.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex stopped 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:45:50.245+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:45:50.245+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12676698187551977291 2023-10-21T04:45:50.260+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:45:50.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex] 2023-10-21T04:45:50.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex closed 2023-10-21T04:45:50.260+05:30 [Info] bucket_%2/idx_secondary/Mainstore#12676698187551977291:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:45:50.260+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:45:50.260+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:45:50.260+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:45:50.260+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex] 2023-10-21T04:45:50.260+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex 2023-10-21T04:45:50.260+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex] 2023-10-21T04:45:50.277+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 45.229628ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.280+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 4390029557575378783 instance id 2203301915252658598 real instance id 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:45:50.280+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 161.758µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.280+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 17703248484023971553 instance id 1863605407862672237 real instance id 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:45:50.280+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 131.552µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.281+05:30 [Info] LifecycleMgr.DeleteIndex() : index 5277316367626246544 does not exist. 2023-10-21T04:45:50.281+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 75.805µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.281+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [18374611443167983589 6455624821495495854] 2023-10-21T04:45:50.294+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex] 2023-10-21T04:45:50.294+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex 2023-10-21T04:45:50.294+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex stopped 2023-10-21T04:45:50.295+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:45:50.295+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:45:50.295+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:45:50.296+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:45:50.305+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:45:50.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex] 2023-10-21T04:45:50.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex closed 2023-10-21T04:45:50.305+05:30 [Info] bucket_%2/idx_secondary/Backstore#12676698187551977291:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:45:50.305+05:30 [Info] Indexer::cleanupIndexData IndexInst 12676698187551977291 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:45:50.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex ... 2023-10-21T04:45:50.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex ... 2023-10-21T04:45:50.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:45:50.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/mainIndex sucessfully destroyed 2023-10-21T04:45:50.308+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex ... 2023-10-21T04:45:50.309+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 21 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex ... 2023-10-21T04:45:50.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:45:50.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_12676698187551977291_0.index destroyed 2023-10-21T04:45:50.311+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12676698187551977291, PartitionId 0, IndexDefnId 17476654447546601131. 2023-10-21T04:45:50.311+05:30 [Info] Indexer::cleanupIndexData IndexInst 12676698187551977291 Partition 0 Destroy Done 2023-10-21T04:45:50.327+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:50.327+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:50.341+05:30 [Info] switched currmeta from 184 -> 186 force true 2023-10-21T04:45:50.365+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:50.365+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:50.372+05:30 [Info] switched currmeta from 186 -> 186 force true 2023-10-21T04:45:50.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.4580883029265679, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:45:50.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 18374611443167983589 2023-10-21T04:45:50.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 6455624821495495854 2023-10-21T04:45:50.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 6455624821495495854 2023-10-21T04:45:50.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 18374611443167983589 2023-10-21T04:45:50.694+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [18374611443167983589 6455624821495495854], elapsed: 412.609729ms 2023-10-21T04:45:50.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:50.694+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:45:50.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper stopped 2023-10-21T04:45:50.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': stopped 2023-10-21T04:45:50.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:45:50.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:45:50.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:45:50.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:45:50.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:45:50.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:45:50.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:45:50.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:45:50.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:45:50.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:45:50.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:45:50.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:45:50.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:45:50.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:45:50.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:45:50.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:45:50.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:45:50.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:45:50.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shutdown completed 2023-10-21T04:45:50.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:50.699+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:45:50.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:45:50.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:45:50.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance destroyed 2023-10-21T04:45:50.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:45:50.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:50.701+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:45:50.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:45:50.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:45:50.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:45:50.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:45:50.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:45:50.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:45:50.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:45:50.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:45:50.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:45:50.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:45:50.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:45:50.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:45:50.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:45:50.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:45:50.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:45:50.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:45:50.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:45:50.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:45:50.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shutdown completed 2023-10-21T04:45:50.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:50.705+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:45:50.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:45:50.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:45:50.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance destroyed 2023-10-21T04:45:50.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:45:50.706+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Cleaned up Tenant with name:bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:45:50.706+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [18374611443167983589 6455624821495495854], elapsed(sec): 0.424682372 2023-10-21T04:45:50.709+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokene4:a6:e2:fc:d5:0:3f:66 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ac5b1dc1660a058d991d594b1aa4a57f ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken9e:79:d9:66:3b:76:f4:db Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 2259746612664550306 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9874972326106273392 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4743233249208554426 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:45:50.709+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: ac5b1dc1660a058d991d594b1aa4a57f TransferTokenId: ShardTokene4:a6:e2:fc:d5:0:3f:66 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:45:50.709+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 9.9276e-05 2023-10-21T04:45:50.709+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokene4:a6:e2:fc:d5:0:3f:66, dropIndexes: true 2023-10-21T04:45:50.709+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66, shardId :0 2023-10-21T04:45:50.709+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ac5b1dc1660a058d991d594b1aa4a57f_ShardTokene4_a6_e2_fc_d5_0_3f_66, shardId :0 2023-10-21T04:45:50.710+05:30 [Info] LifecycleMgr.DeleteIndex() : index 5932387718257253322 does not exist. 2023-10-21T04:45:50.710+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 76.51µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.710+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 7790129686881525591 instance id 9874972326106273392 real instance id 16759912838651900043 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:45:50.711+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 160.771µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.711+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16823504650100619022 instance id 4743233249208554426 real instance id 4186965058349029465 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:45:50.711+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 148.372µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.712+05:30 [Info] LifecycleMgr.DeleteIndex() : index 668551233566132751 does not exist. 2023-10-21T04:45:50.712+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 53.967µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:50.712+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17327901024118572215 5629943257627822009] 2023-10-21T04:45:50.728+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:45:51.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17327901024118572215 2023-10-21T04:45:51.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 5629943257627822009 2023-10-21T04:45:51.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 5629943257627822009 2023-10-21T04:45:51.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17327901024118572215 2023-10-21T04:45:51.694+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17327901024118572215 5629943257627822009], elapsed: 981.76986ms 2023-10-21T04:45:51.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:51.694+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:45:51.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:45:51.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shutdown completed 2023-10-21T04:45:51.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:51.694+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:45:51.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:45:51.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance destroyed 2023-10-21T04:45:51.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:45:51.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:51.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:45:51.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:45:51.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shutdown completed 2023-10-21T04:45:51.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:45:51.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:45:51.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:45:51.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance destroyed 2023-10-21T04:45:51.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:45:51.697+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17327901024118572215 5629943257627822009], elapsed(sec): 0.985030805 2023-10-21T04:45:51.704+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:45:51.704+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:45:51.704+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.605µs 2023-10-21T04:45:51.704+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:45:51.704+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:45:51.704+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:45:51.709+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:45:51.709+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:45:51.709+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:45:51.710+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:45:51.710+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 55.457µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:51.711+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:45:51.712+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:45:51.712+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced false, isMaster true, forceUnbalanced true, err: 2023-10-21T04:45:51.712+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 4.43750158s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:45:51.712+05:30 [Info] GenericServiceManager::CancelTask: return from id: rebalance/ac5b1dc1660a058d991d594b1aa4a57f, rev: [] call. err: 2023-10-21T04:45:51.712+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 14] call. taskList: &{Rev:[0 0 0 0 0 0 0 15] Tasks:[]} 2023-10-21T04:45:51.712+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:45:51.712+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 15] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:45:51.716+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/ac5b1dc1660a058d991d594b1aa4a57f, rev: [] 2023-10-21T04:45:51.716+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/ac5b1dc1660a058d991d594b1aa4a57f, rev: [] 2023-10-21T04:45:51.716+05:30 [Info] PauseServiceManager::endTask: called with err for taskId prepare/ac5b1dc1660a058d991d594b1aa4a57f 2023-10-21T04:45:51.716+05:30 [Info] PauseServiceManager::endTask task with ID prepare/ac5b1dc1660a058d991d594b1aa4a57f already cleaned up 2023-10-21T04:45:51.716+05:30 [Info] PauseServiceManager::PauseResumeCancelTask: removed all tasks with ID prepare/ac5b1dc1660a058d991d594b1aa4a57f 2023-10-21T04:45:51.716+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/ac5b1dc1660a058d991d594b1aa4a57f, rev: [] call. err: 2023-10-21T04:45:51.723+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:45:51.723+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 15] Tasks:[]} 2023-10-21T04:45:51.731+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:45:51.731+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 15] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:45:51.732+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 15] 2023-10-21T04:45:51.733+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 15] 2023-10-21T04:45:51.734+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:45:51.766+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9112 2023-10-21T04:45:51.766+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:45:51.774+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:45:51.811+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:45:51.811+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:45:51.811+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:45:51.818+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:55918 2023-10-21T04:45:51.818+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:45:51.833+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 12.043776ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:45:51.834+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:45:51.834+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:45:51.834+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:45:51.841+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:45:51.873+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:45:51.873+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9106 2023-10-21T04:45:51.873+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:45:51.880+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:45:51.911+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:45:51.912+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:45:51.912+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:45:51.912+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:45:51.912+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:45:51.915+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:45:51.915+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:45:51.915+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:45:51.915+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:45:51.915+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:45:51.915+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:45:51.915+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:45:51.915+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:47626->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:51.915+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:55918->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:51.916+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:52296->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:51.916+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:47178->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:45:51.916+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:55918. Reason = EOF. Cleaning Pipe. 2023-10-21T04:45:51.916+05:30 [Info] messageListener.start(): message channel closed. Remove peer 98:7a:51:82:54:4:72:da as follower. 2023-10-21T04:45:52.335+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:45:54.033+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:45:54.039+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:45:54.039+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:45:54.042+05:30 [Info] switched currmeta from 186 -> 186 force true 2023-10-21T04:46:02.781+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:46:03.121+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 244] aggregate storage info: map[bucket_%2:map[kv:2398400 index:11334127] bucket_1:map[kv:2468032 index:7549078]] 2023-10-21T04:46:05.537+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:05.537+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:05.567+05:30 [Info] switched currmeta from 186 -> 190 force true 2023-10-21T04:46:05.615+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:05.615+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:05.619+05:30 [Info] switched currmeta from 190 -> 190 force true 2023-10-21T04:46:05.815+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:05.815+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:05.820+05:30 [Info] switched currmeta from 190 -> 194 force true 2023-10-21T04:46:05.882+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:05.882+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:05.887+05:30 [Info] switched currmeta from 194 -> 194 force true 2023-10-21T04:46:09.179+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:09.278+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:09.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:09.323+05:30 [Info] switched currmeta from 194 -> 194 force true 2023-10-21T04:46:11.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9999318944441362, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:46:12.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9994354813708852, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:46:13.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7021012576936582, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:46:14.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.3287158654163574, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:46:21.733+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:46:21.733+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 15] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:46:21.734+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 15] call. err: operation_canceled 2023-10-21T04:46:21.734+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:46:21.734+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 15] Tasks:[]} 2023-10-21T04:46:21.736+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 15] 2023-10-21T04:46:21.737+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 15] 2023-10-21T04:46:26.899+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 16632040625897953619 2023-10-21T04:46:26.899+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 158.93µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:26.987+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41568 2023-10-21T04:46:26.997+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41568 2023-10-21T04:46:27.045+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41568 2023-10-21T04:46:27.113+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 16632040625897953619 2023-10-21T04:46:27.113+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 275.687µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:27.202+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:27.202+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:27.208+05:30 [Info] switched currmeta from 194 -> 199 force true 2023-10-21T04:46:27.242+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:27.242+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:27.272+05:30 [Info] switched currmeta from 199 -> 200 force true 2023-10-21T04:46:27.453+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:27.453+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:27.458+05:30 [Info] switched currmeta from 200 -> 203 force true 2023-10-21T04:46:27.474+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:27.474+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:27.475+05:30 [Info] switched currmeta from 203 -> 203 force true 2023-10-21T04:46:27.521+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56738 2023-10-21T04:46:27.540+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 14.948319ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:27.643+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:46:27.643+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:46:27.650+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:46:27.661+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 16632040625897953619. No update is performed. 2023-10-21T04:46:27.661+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 138.454µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:27.683+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:46:27.683+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:46:27.683+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:46:27.689+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:46:27.708+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:56738. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:27.709+05:30 [Info] messageListener.start(): message channel closed. Remove peer c5:9:83:8:28:29:84:4a as follower. 2023-10-21T04:46:27.736+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:46:27.736+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:46:27.736+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:46:27.773+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56776 2023-10-21T04:46:27.774+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:46:27.795+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.711128ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:27.796+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:46:27.796+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:46:27.796+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:46:27.810+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:46:27.823+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56784 2023-10-21T04:46:27.849+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 21.196227ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:27.853+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:46:27.853+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:46:27.854+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:46:27.854+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:46:27.854+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:46:27.856+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:46:27.856+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:46:27.856+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:46:27.857+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:46:27.857+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:46:27.857+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:56776->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:27.857+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:56776. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:27.858+05:30 [Info] messageListener.start(): message channel closed. Remove peer b:99:51:d8:fe:92:11:fa as follower. 2023-10-21T04:46:27.858+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:48480->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:27.858+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:53154->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:27.859+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:48014->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:27.884+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:56784. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:27.885+05:30 [Info] messageListener.start(): message channel closed. Remove peer e7:85:a7:ab:7e:ca:47:c5 as follower. 2023-10-21T04:46:28.043+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:28.296+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:28.341+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:56826 2023-10-21T04:46:28.353+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:28.376+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 14.71274ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:28.420+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:56826. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:28.420+05:30 [Info] messageListener.start(): message channel closed. Remove peer 68:a1:d4:5e:74:8c:77:42 as follower. 2023-10-21T04:46:28.794+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:28.794+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:28.797+05:30 [Info] switched currmeta from 203 -> 204 force true 2023-10-21T04:46:28.877+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:28.949+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:28.949+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:28.954+05:30 [Info] switched currmeta from 204 -> 205 force true 2023-10-21T04:46:29.195+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:29.199+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:29.199+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:29.201+05:30 [Info] switched currmeta from 205 -> 205 force true 2023-10-21T04:46:29.419+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:29.419+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:29.421+05:30 [Info] switched currmeta from 205 -> 206 force true 2023-10-21T04:46:29.590+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:29.590+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:29.593+05:30 [Info] switched currmeta from 206 -> 207 force true 2023-10-21T04:46:30.361+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14610256586894580199:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:46:30.361+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14610256586894580199:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:46:30.361+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14610256586894580199:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:46:30.361+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14610256586894580199:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:46:30.361+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14610256586894580199:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:46:30.469+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8859733702134432451:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:46:31.731+05:30 [Info] memstats {"Alloc":143072280, "TotalAlloc":429843912, "Sys":200379688, "Lookups":0, "Mallocs":3201390,"Frees":2869105, "HeapAlloc":143072280, "HeapSys":182091776, "HeapIdle":32382976, "HeapInuse":149708800,"HeapReleased":1048576, "HeapObjects":332285,"MSpanInuse":771680, "MSpanSys": 1191360, "StackInuse": 2457600,"GCSys":12285288, "LastGC":1697843701404002512,"PauseTotalNs":3444625, "PauseNs":[196791], "NumGC":9} 2023-10-21T04:46:32.786+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:46:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:46:32.787+05:30 [Info] client stats current counts: current: 14, not current: 0 2023-10-21T04:46:32.863+05:30 [Info] num concurrent scans {0} 2023-10-21T04:46:32.863+05:30 [Info] average scan response {0 ms} 2023-10-21T04:46:33.098+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:46:33.114+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:46:33.379+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:46:33.405+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:46:34.205+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:34.209+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:34.209+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:34.211+05:30 [Info] switched currmeta from 207 -> 207 force true 2023-10-21T04:46:34.311+05:30 [Info] janitor: running cleanup. 2023-10-21T04:46:34.311+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[10761885902580101111 18026660730857541767 7525085113233084253 12027054384338714804 8857269605176042330 8378417662949994532 12833670934079048728 5932387718257253322 4987105685507282285 14549775068266281786 14292583894719865678 16162012140118277065 9665410041638025825 18323460706027642575 15968368593314043432 7392796230534034390 4532813151040661705 16483001637761719171 3381524789655566900 12195715103909237684 13856704970496601958 3674078063353481907 8669649359784409226 15945117906390072636 11276338709617092948 15431181238515168303 4286621850367430051 573521267164877540 12599498416578472779 4138047918192025493 16961575949607441659 13268712595133993155 11365633301282701856 8504686614261377811 17656144409930763141 2048006213723315482 14889401051926000176 17797267771846799851 12824340686309573428 8910459001587546151 5465835793930010953 17917116660857103172 1252359852432998131 16706481854123898870 7045704526128519604 15973521466438086342 249887442666643345 17476654447546601131 1301336828089581719 16337157693634799600 12140458060295645238 6985049432225870291 12582561312368048736 14741949230702299196 18046371353728341033 4790214984019693894 10606939961365971246 7147407900589480727 12338036531800130039 4535197604682045916 11619165464560280276 4860549750398470299]} 2023-10-21T04:46:34.890+05:30 [Info] Indexer::ReadMemstats Time Taken 98.656µs 2023-10-21T04:46:35.726+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 128675491261474353 2023-10-21T04:46:35.726+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 142.865µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:36.040+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41568 2023-10-21T04:46:36.095+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41568 2023-10-21T04:46:36.277+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:41568 2023-10-21T04:46:36.644+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 128675491261474353 2023-10-21T04:46:36.644+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 265.204µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:36.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9995957554573013, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:46:36.859+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:36.859+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:36.875+05:30 [Info] switched currmeta from 207 -> 216 force true 2023-10-21T04:46:36.994+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:36.994+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:37.010+05:30 [Info] switched currmeta from 216 -> 216 force true 2023-10-21T04:46:37.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9997074351220824, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:46:37.786+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:46:37.786+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:46:37.875+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:46:37.909+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57074 2023-10-21T04:46:37.995+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 78.259081ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:38.043+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:46:38.043+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:46:38.043+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:46:38.121+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:46:38.238+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:46:38.239+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:46:38.239+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:46:38.312+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57120 2023-10-21T04:46:38.313+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:46:38.379+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 64.482772ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:38.379+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:46:38.379+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:46:38.379+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:46:38.418+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57128 2023-10-21T04:46:38.487+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:46:38.496+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:38.569+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 128675491261474353. No update is performed. 2023-10-21T04:46:38.569+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 129.449µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:38.603+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 180.152696ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:38.605+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57140 2023-10-21T04:46:38.619+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57074. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:38.619+05:30 [Info] messageListener.start(): message channel closed. Remove peer 57:22:4b:18:ec:67:dc:40 as follower. 2023-10-21T04:46:38.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9995800486498833 2023-10-21T04:46:38.692+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:46:38.692+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:46:38.692+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:46:38.692+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:46:38.692+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:46:38.692+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:46:38.692+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:46:38.692+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:46:38.692+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:46:38.692+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:46:38.692+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:53504->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:38.693+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:48322->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:38.693+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:48808->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:38.693+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:57120->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:46:38.693+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57120. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:38.693+05:30 [Info] messageListener.start(): message channel closed. Remove peer d2:cd:60:5c:f5:f0:ca:b5 as follower. 2023-10-21T04:46:38.729+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 121.472424ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:38.838+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57128. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:38.838+05:30 [Info] messageListener.start(): message channel closed. Remove peer 44:77:e4:ea:98:f0:8f:b6 as follower. 2023-10-21T04:46:38.880+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:39.001+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57140. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:39.001+05:30 [Info] messageListener.start(): message channel closed. Remove peer 51:aa:5:e5:48:e1:5:92 as follower. 2023-10-21T04:46:39.104+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:39.230+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:46:39.242+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:39.247+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:39.247+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:39.255+05:30 [Info] switched currmeta from 216 -> 216 force true 2023-10-21T04:46:39.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9505501483178066 2023-10-21T04:46:40.344+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:40.344+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:40.349+05:30 [Info] switched currmeta from 216 -> 217 force true 2023-10-21T04:46:40.376+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:40.376+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:40.379+05:30 [Info] switched currmeta from 217 -> 218 force true 2023-10-21T04:46:40.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7834349628053134, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:46:40.771+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:40.771+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:40.773+05:30 [Info] switched currmeta from 218 -> 219 force true 2023-10-21T04:46:40.807+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:40.808+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:40.827+05:30 [Info] switched currmeta from 219 -> 220 force true 2023-10-21T04:46:41.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.5807284528380409, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:46:41.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10381296292560770566:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:46:41.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10381296292560770566:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:46:41.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[7693136708512810836:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:46:44.240+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:44.251+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:44.251+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:44.253+05:30 [Info] switched currmeta from 220 -> 220 force true 2023-10-21T04:46:50.132+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:46:50.593+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:50.593+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:50.623+05:30 [Info] switched currmeta from 220 -> 220 force true 2023-10-21T04:46:50.650+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:46:50.740+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:46:50.740+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:46:50.768+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:50.891+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:50.891+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:50.894+05:30 [Info] switched currmeta from 220 -> 220 force true 2023-10-21T04:46:50.901+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:50.901+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:50.906+05:30 [Info] switched currmeta from 220 -> 220 force true 2023-10-21T04:46:51.737+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:46:51.737+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 15] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:46:51.738+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 15] 2023-10-21T04:46:51.738+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 15] call. err: operation_canceled 2023-10-21T04:46:51.738+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:46:51.738+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 15] Tasks:[]} 2023-10-21T04:46:51.744+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 15] 2023-10-21T04:46:52.074+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {ab759b18c08b0e81e45c6631c4e2d845 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:46:52.078+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 4.936µs 2023-10-21T04:46:52.078+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:46:52.078+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:46:52.078+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:46:52.080+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:46:52.080+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:52.081+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.463855ms 2023-10-21T04:46:52.081+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:46:52.081+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:46:52.081+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 16] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:46:52.081+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 8.304µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:52.081+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 15] call. taskList: &{Rev:[0 0 0 0 0 0 0 16] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ab759b18c08b0e81e45c6631c4e2d845 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ab759b18c08b0e81e45c6631c4e2d845]}]} 2023-10-21T04:46:52.089+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 16] 2023-10-21T04:46:52.091+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 16] 2023-10-21T04:46:53.160+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {ba6304988a6335d65281381d6cfc5e32 ab759b18c08b0e81e45c6631c4e2d845 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:46:53.164+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {ba6304988a6335d65281381d6cfc5e32 ab759b18c08b0e81e45c6631c4e2d845 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:46:53.164+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"ab759b18c08b0e81e45c6631c4e2d845","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:46:53.166+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{ba6304988a6335d65281381d6cfc5e32 ab759b18c08b0e81e45c6631c4e2d845 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:46:53.166+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{ba6304988a6335d65281381d6cfc5e32 ab759b18c08b0e81e45c6631c4e2d845 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:46:53.167+05:30 [Info] ShardRebalancer::observeRebalance &{ba6304988a6335d65281381d6cfc5e32 ab759b18c08b0e81e45c6631c4e2d845 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:46:53.173+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"ab759b18c08b0e81e45c6631c4e2d845","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:46:53.173+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:46:53.275+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:46:53.284+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:46:53.325+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:46:53.435+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:6a:aa:c2:29:3:d1:a5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3b:e7:9c:c8:dd:24:1e:45 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7756628172397162046 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7693136708512810836 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3851262100190137504 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.499+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbd:30:d7:e2:d2:4c:63:d7 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken76:ee:1e:24:89:be:a6:21 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8859733702134432451 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6438969491930702266 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13138923834221067684 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.541+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3b:e7:9c:c8:dd:24:1e:45 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:6a:aa:c2:29:3:d1:a5 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10381296292560770566 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2769351312389229321 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13496199801043700004 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.544+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken76:ee:1e:24:89:be:a6:21 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8379234217984719924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12957729777462041899 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14610256586894580199 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.545+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:6a:aa:c2:29:3:d1:a5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3b:e7:9c:c8:dd:24:1e:45 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7756628172397162046 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7693136708512810836 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3851262100190137504 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.547+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:46:53.547+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:46:53.547+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:46:53.547+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.913µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:53.567+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:6a:aa:c2:29:3:d1:a5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3b:e7:9c:c8:dd:24:1e:45 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7756628172397162046 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7693136708512810836 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3851262100190137504 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.604+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbd:30:d7:e2:d2:4c:63:d7 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken76:ee:1e:24:89:be:a6:21 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8859733702134432451 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6438969491930702266 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13138923834221067684 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.604+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:46:53.604+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:46:53.604+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:46:53.604+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 29.556µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:53.630+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbd:30:d7:e2:d2:4c:63:d7 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken76:ee:1e:24:89:be:a6:21 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8859733702134432451 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6438969491930702266 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13138923834221067684 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.688+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3b:e7:9c:c8:dd:24:1e:45 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:6a:aa:c2:29:3:d1:a5 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10381296292560770566 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2769351312389229321 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13496199801043700004 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.688+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken76:ee:1e:24:89:be:a6:21 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8379234217984719924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12957729777462041899 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14610256586894580199 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.735+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3b:e7:9c:c8:dd:24:1e:45 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:6a:aa:c2:29:3:d1:a5 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10381296292560770566 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2769351312389229321 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13496199801043700004 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.761+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken76:ee:1e:24:89:be:a6:21 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8379234217984719924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12957729777462041899 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14610256586894580199 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.801+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"ab759b18c08b0e81e45c6631c4e2d845","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:46:53.801+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:46:53.801+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:46:53.801+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:46:53.802+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:46:53.802+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:46:53.802+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 20.78µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:53.875+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3b:e7:9c:c8:dd:24:1e:45 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:6a:aa:c2:29:3:d1:a5 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10381296292560770566 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2769351312389229321 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13496199801043700004 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.881+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:6a:aa:c2:29:3:d1:a5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3b:e7:9c:c8:dd:24:1e:45 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7756628172397162046 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7693136708512810836 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3851262100190137504 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.954+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:6a:aa:c2:29:3:d1:a5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3b:e7:9c:c8:dd:24:1e:45 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7756628172397162046 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7693136708512810836 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3851262100190137504 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:53.954+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:46:53.954+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:46:53.954+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:46:53.955+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 30.333µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:53.955+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[6455624821495495854:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/6455624821495495854/shards/shard6455624821495495854 18374611443167983589:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/18374611443167983589/shards/shard18374611443167983589] ab759b18c08b0e81e45c6631c4e2d845 ShardToken85:6a:aa:c2:29:3:d1:a5 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc005092660 0xc0050926c0 map[] 0xc008640f00 0xc00908ca80 false} 2023-10-21T04:46:53.955+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/6455624821495495854/shards/shard6455624821495495854 2023-10-21T04:46:53.956+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/18374611443167983589/shards/shard18374611443167983589 2023-10-21T04:46:53.961+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/18374611443167983589/shards/shard18374611443167983589/data: super block snapshot head offset :0 tail offset:1196032 sbgen :174 2023-10-21T04:46:53.961+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/18374611443167983589/shards/shard18374611443167983589/data/recovery: super block snapshot head offset :0 tail offset:1073152 sbgen :174 2023-10-21T04:46:53.966+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/6455624821495495854/shards/shard6455624821495495854/data: super block snapshot head offset :0 tail offset:1011712 sbgen :159 2023-10-21T04:46:53.967+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/6455624821495495854/shards/shard6455624821495495854/data/recovery: super block snapshot head offset :0 tail offset:851968 sbgen :159 2023-10-21T04:46:53.970+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/18374611443167983589/shards/shard18374611443167983589: metadata saved successfully 2023-10-21T04:46:53.971+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) added 2023-10-21T04:46:53.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:46:53.971+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 152564400 totalBytes: 2288466 numShards: 2 duration: 15.92491ms 2023-10-21T04:46:53.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/18374611443167983589/shards/shard18374611443167983589, newshardId 18374611443167983589, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, shardsize 2288466, duration 15.236707ms 2023-10-21T04:46:53.971+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 18374611443167983589, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, err: , elapsed(sec): 0.016162917 2023-10-21T04:46:53.971+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 18374611443167983589 bytesWritten: 2288466, totalBytes: 2288466, transferRate: 1.525644e+08 2023-10-21T04:46:53.976+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/6455624821495495854/shards/shard6455624821495495854: metadata saved successfully 2023-10-21T04:46:53.977+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) added 2023-10-21T04:46:53.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:46:53.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5/6455624821495495854/shards/shard6455624821495495854, newshardId 6455624821495495854, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, shardsize 1882799, duration 22.066089ms 2023-10-21T04:46:53.977+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 6455624821495495854, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, err: , elapsed(sec): 0.022373052 2023-10-21T04:46:53.977+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: ab759b18c08b0e81e45c6631c4e2d845, transferId: ShardToken85:6a:aa:c2:29:3:d1:a5 2023-10-21T04:46:53.977+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: ab759b18c08b0e81e45c6631c4e2d845, transferId: ShardToken85:6a:aa:c2:29:3:d1:a5, elapsed(sec): 0.000121642 2023-10-21T04:46:53.977+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[6455624821495495854: 18374611443167983589:], shardPaths: map[6455624821495495854:/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 18374611443167983589:/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.022561391 2023-10-21T04:46:53.977+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 6455624821495495854 bytesWritten: 1882799, totalBytes: 1882799, transferRate: 8.558177272727273e+07 2023-10-21T04:46:53.977+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [18374611443167983589 6455624821495495854], ttid: ShardToken85:6a:aa:c2:29:3:d1:a5, elapsed(sec): 0.023031526 2023-10-21T04:46:53.983+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:46:54.005+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken85:6a:aa:c2:29:3:d1:a5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3b:e7:9c:c8:dd:24:1e:45 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7756628172397162046 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7693136708512810836 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3851262100190137504 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.005+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken85:6a:aa:c2:29:3:d1:a5 2023-10-21T04:46:54.005+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[6455624821495495854:true 18374611443167983589:true] 2023-10-21T04:46:54.005+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 30.217µs 2023-10-21T04:46:54.005+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [18374611443167983589 6455624821495495854], isLockedForRecovery: true 2023-10-21T04:46:54.005+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [18374611443167983589 6455624821495495854], errMap: map[6455624821495495854: 18374611443167983589:], elapsed: 12.393µs 2023-10-21T04:46:54.006+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:46:54.006+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:46:54.056+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12396348969790516872,realInstId: 0, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:46:54.057+05:30 [Info] Indexer::handleRecoverIndex InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:46:54.058+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 572.879µs 2023-10-21T04:46:54.058+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 736.666µs 2023-10-21T04:46:54.058+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.058+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 2 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.058+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': added 2023-10-21T04:46:54.059+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12396348969790516872, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:46:54.059+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 53.30802ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:54.061+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCleaner initialized 2023-10-21T04:46:54.061+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:46:54.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.066+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:46:54.074+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCleaner initialized for recovery 2023-10-21T04:46:54.074+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:46:54.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery 2023-10-21T04:46:54.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true] 2023-10-21T04:46:54.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1073152] Data log Head [0] Tail [1196032] 2023-10-21T04:46:54.074+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1073152] 2023-10-21T04:46:54.082+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{ba6304988a6335d65281381d6cfc5e32 ab759b18c08b0e81e45c6631c4e2d845 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:46:54.084+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.085+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:46:54.086+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:46:54.087+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.091+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:46:54.092+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.086+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCleaner initialized 2023-10-21T04:46:54.095+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:46:54.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.099+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:46:54.099+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.114+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:46:54.111+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:54.115+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:54.115+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.116+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:46:54.116+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.116+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:46:54.116+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.116+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:46:54.117+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.117+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:46:54.117+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.117+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:46:54.118+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.118+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.118+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.119+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.119+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.119+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.119+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.120+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.120+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.120+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:54.120+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.120+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.121+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.121+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.121+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.121+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.123+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.162+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex 2023-10-21T04:46:54.136+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3b:e7:9c:c8:dd:24:1e:45 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:6a:aa:c2:29:3:d1:a5 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10381296292560770566 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2769351312389229321 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13496199801043700004 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.140+05:30 [Info] switched currmeta from 220 -> 221 force true 2023-10-21T04:46:54.162+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : replayOffset: 1191936 (RP:1171456) (CP:1191936) 2023-10-21T04:46:54.162+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1191936] took [88.077165ms] 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867578 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867578 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867578 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867578 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867578 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1055476, resident 1, memSz 679, memSzIndex 0, numRecords 10 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867578 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867534 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867490 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867490 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867446 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867402 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867358 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1055832, resident 1, memSz 70174, memSzIndex 265, numRecords 1000 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867358 controller mem 1258 2023-10-21T04:46:54.163+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCleaner initialized for recovery 2023-10-21T04:46:54.163+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:46:54.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery 2023-10-21T04:46:54.163+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true] 2023-10-21T04:46:54.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[851968] Data log Head [0] Tail [1011712] 2023-10-21T04:46:54.171+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [851968] 2023-10-21T04:46:54.170+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867358 controller mem 1258 2023-10-21T04:46:54.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867314 controller mem 1258 2023-10-21T04:46:54.171+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867270 controller mem 1258 2023-10-21T04:46:54.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867270 controller mem 1258 2023-10-21T04:46:54.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.172+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 27867270 controller mem 1258 2023-10-21T04:46:54.172+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1196032] replayOffset [1191936] 2023-10-21T04:46:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.173+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:46:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.173+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:46:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.173+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:46:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.173+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:46:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.173+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:46:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.173+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:46:54.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.173+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:46:54.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.174+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:46:54.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.174+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.174+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.174+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.174+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.174+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.175+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.175+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:54.175+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.175+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.175+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.175+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.175+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.175+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.176+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.176+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex 2023-10-21T04:46:54.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [102.27334ms] 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true]. Built [18] plasmas, took [103.075069ms] 2023-10-21T04:46:54.178+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.178+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:46:54.180+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : replayOffset: 1007616 (RP:978944) (CP:1007616) 2023-10-21T04:46:54.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1007616] took [9.246462ms] 2023-10-21T04:46:54.180+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.180+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651284 controller mem 3878 2023-10-21T04:46:54.180+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.182+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651240 controller mem 3878 2023-10-21T04:46:54.182+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.183+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651196 controller mem 3878 2023-10-21T04:46:54.183+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.183+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651196 controller mem 3878 2023-10-21T04:46:54.183+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.183+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651196 controller mem 3878 2023-10-21T04:46:54.183+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.184+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651196 controller mem 3878 2023-10-21T04:46:54.184+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.184+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651196 controller mem 3878 2023-10-21T04:46:54.184+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651196 controller mem 3878 2023-10-21T04:46:54.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1056266, resident 1, memSz 69032, memSzIndex 874, numRecords 1000 2023-10-21T04:46:54.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651196 controller mem 3878 2023-10-21T04:46:54.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.185+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651152 controller mem 3878 2023-10-21T04:46:54.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.188+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper started 2023-10-21T04:46:54.188+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex started 2023-10-21T04:46:54.188+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.189+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.189+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.189+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651108 controller mem 3878 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651064 controller mem 3878 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651064 controller mem 3878 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651064 controller mem 3878 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651020 controller mem 3878 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44650976 controller mem 3878 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44650976 controller mem 3878 2023-10-21T04:46:54.192+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1011712] replayOffset [1007616] 2023-10-21T04:46:54.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [21.568038ms] 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.192+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:46:54.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true]. Built [17] plasmas, took [22.249489ms] 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.193+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:46:54.227+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbd:30:d7:e2:d2:4c:63:d7 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken76:ee:1e:24:89:be:a6:21 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8859733702134432451 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6438969491930702266 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13138923834221067684 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.272+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex started 2023-10-21T04:46:54.272+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:46:54.272+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:46:54.272+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:46:54.272+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex stopped 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.273+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.273+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.292+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken76:ee:1e:24:89:be:a6:21 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8379234217984719924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12957729777462041899 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14610256586894580199 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.298+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:46:54.298+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex closed 2023-10-21T04:46:54.298+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:54.299+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.299+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.299+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.299+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:46:54.299+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.299+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:46:54.350+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3b:e7:9c:c8:dd:24:1e:45 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken85:6a:aa:c2:29:3:d1:a5 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10381296292560770566 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2769351312389229321 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13496199801043700004 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.389+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:46:54.389+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.389+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex stopped 2023-10-21T04:46:54.390+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.390+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.390+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.390+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:46:54.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex closed 2023-10-21T04:46:54.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:54.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:46:54.408+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:46:54.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.414+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex sucessfully destroyed 2023-10-21T04:46:54.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:46:54.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:46:54.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index destroyed 2023-10-21T04:46:54.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.478+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.479+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.479+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.482+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.482+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.483+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.488+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.489+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:46:54.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex started 2023-10-21T04:46:54.489+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.489+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.490+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:46:54.490+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex started 2023-10-21T04:46:54.490+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Warmup took 431.624042ms 2023-10-21T04:46:54.490+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 2 WriterThreads 1 2023-10-21T04:46:54.491+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Stopped Billing WUs 2023-10-21T04:46:54.491+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index Index: 12396348969790516872 Partition: 2 2023-10-21T04:46:54.491+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.491+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 3 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.493+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.493+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:46:54.496+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.497+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.497+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.497+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.497+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.497+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:46:54.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.502+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.502+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex started 2023-10-21T04:46:54.503+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.503+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.515+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex started 2023-10-21T04:46:54.515+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:46:54.515+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:54.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:46:54.516+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.516+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:46:54.517+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:46:54.517+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex stopped 2023-10-21T04:46:54.517+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.517+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.517+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.517+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.515+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.515+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.523+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbd:30:d7:e2:d2:4c:63:d7 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken76:ee:1e:24:89:be:a6:21 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8859733702134432451 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6438969491930702266 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13138923834221067684 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.523+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:46:54.523+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:46:54.523+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:46:54.523+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 23.037µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:54.523+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[5629943257627822009:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/5629943257627822009/shards/shard5629943257627822009 17327901024118572215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/17327901024118572215/shards/shard17327901024118572215] ab759b18c08b0e81e45c6631c4e2d845 ShardTokenbd:30:d7:e2:d2:4c:63:d7 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc005092660 0xc0050926c0 map[] 0xc00a0586c0 0xc004c8b4a0 false} 2023-10-21T04:46:54.523+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/5629943257627822009/shards/shard5629943257627822009 2023-10-21T04:46:54.525+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/17327901024118572215/shards/shard17327901024118572215 2023-10-21T04:46:54.527+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/5629943257627822009/shards/shard5629943257627822009/data: super block snapshot head offset :0 tail offset:1011712 sbgen :159 2023-10-21T04:46:54.527+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/5629943257627822009/shards/shard5629943257627822009/data/recovery: super block snapshot head offset :0 tail offset:864256 sbgen :159 2023-10-21T04:46:54.528+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/17327901024118572215/shards/shard17327901024118572215/data: super block snapshot head offset :0 tail offset:1183744 sbgen :174 2023-10-21T04:46:54.529+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/17327901024118572215/shards/shard17327901024118572215/data/recovery: super block snapshot head offset :0 tail offset:1064960 sbgen :174 2023-10-21T04:46:54.533+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/5629943257627822009/shards/shard5629943257627822009: metadata saved successfully 2023-10-21T04:46:54.534+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) added 2023-10-21T04:46:54.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:46:54.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/5629943257627822009/shards/shard5629943257627822009, newshardId 5629943257627822009, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, shardsize 1895052, duration 10.726373ms 2023-10-21T04:46:54.534+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 5629943257627822009, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, err: , elapsed(sec): 0.011020424 2023-10-21T04:46:54.534+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 5629943257627822009 bytesWritten: 1895052, totalBytes: 1895052, transferRate: 1.895052e+08 2023-10-21T04:46:54.534+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/17327901024118572215/shards/shard17327901024118572215: metadata saved successfully 2023-10-21T04:46:54.535+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) added 2023-10-21T04:46:54.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:46:54.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7/17327901024118572215/shards/shard17327901024118572215, newshardId 17327901024118572215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, shardsize 2267945, duration 10.2447ms 2023-10-21T04:46:54.535+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17327901024118572215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, err: , elapsed(sec): 0.011863162 2023-10-21T04:46:54.535+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: ab759b18c08b0e81e45c6631c4e2d845, transferId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 2023-10-21T04:46:54.535+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: ab759b18c08b0e81e45c6631c4e2d845, transferId: ShardTokenbd:30:d7:e2:d2:4c:63:d7, elapsed(sec): 0.000117981 2023-10-21T04:46:54.535+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[5629943257627822009: 17327901024118572215:], shardPaths: map[5629943257627822009:/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 17327901024118572215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.012054424 2023-10-21T04:46:54.535+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17327901024118572215 bytesWritten: 2267945, totalBytes: 2267945, transferRate: 2.267945e+08 2023-10-21T04:46:54.535+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17327901024118572215 5629943257627822009], ttid: ShardTokenbd:30:d7:e2:d2:4c:63:d7, elapsed(sec): 0.012489234 2023-10-21T04:46:54.581+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:46:54.596+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:46:54.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex closed 2023-10-21T04:46:54.597+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:54.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.597+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbd:30:d7:e2:d2:4c:63:d7 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken76:ee:1e:24:89:be:a6:21 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8859733702134432451 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6438969491930702266 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13138923834221067684 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:46:54.599+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.599+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:46:54.599+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:46:54.599+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex stopped 2023-10-21T04:46:54.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.601+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.609+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:46:54.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex closed 2023-10-21T04:46:54.609+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:54.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:46:54.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:46:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex sucessfully destroyed 2023-10-21T04:46:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:46:54.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:46:54.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index destroyed 2023-10-21T04:46:54.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.618+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.618+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.621+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.621+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.625+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.626+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:46:54.626+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex started 2023-10-21T04:46:54.626+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.626+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.627+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.627+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:46:54.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex started 2023-10-21T04:46:54.627+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Warmup took 135.736103ms 2023-10-21T04:46:54.628+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 3 WriterThreads 1 2023-10-21T04:46:54.628+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Stopped Billing WUs 2023-10-21T04:46:54.628+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index Index: 12396348969790516872 Partition: 3 2023-10-21T04:46:54.628+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.628+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 4 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.629+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.629+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 43115 recovery UsedSpace 8236 2023-10-21T04:46:54.629+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.629+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.629+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.629+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:46:54.632+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:54.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:54.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.638+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex started 2023-10-21T04:46:54.638+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.638+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.639+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex started 2023-10-21T04:46:54.639+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:46:54.639+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:46:54.639+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:46:54.639+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex stopped 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.639+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.647+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:46:54.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex closed 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.648+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:46:54.648+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.648+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex stopped 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.648+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.650+05:30 [Info] switched currmeta from 221 -> 222 force true 2023-10-21T04:46:54.664+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:46:54.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex closed 2023-10-21T04:46:54.664+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:54.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:46:54.664+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:46:54.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex sucessfully destroyed 2023-10-21T04:46:54.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:46:54.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:46:54.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.681+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index destroyed 2023-10-21T04:46:54.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.684+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.684+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.684+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.691+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.692+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.699+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.699+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:46:54.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex started 2023-10-21T04:46:54.699+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.699+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.704+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.704+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:46:54.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex started 2023-10-21T04:46:54.704+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Warmup took 76.288663ms 2023-10-21T04:46:54.705+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 4 WriterThreads 1 2023-10-21T04:46:54.705+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Stopped Billing WUs 2023-10-21T04:46:54.705+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index Index: 12396348969790516872 Partition: 4 2023-10-21T04:46:54.705+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.705+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 5 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.705+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.705+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:46:54.706+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.707+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.707+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.707+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:46:54.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.712+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.712+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex started 2023-10-21T04:46:54.712+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.712+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.713+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex started 2023-10-21T04:46:54.713+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:46:54.714+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:46:54.714+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:46:54.714+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex stopped 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.714+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.715+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.718+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:46:54.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex closed 2023-10-21T04:46:54.718+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:54.718+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.718+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.718+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.718+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:46:54.718+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.718+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:46:54.718+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:46:54.718+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.719+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex stopped 2023-10-21T04:46:54.719+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.719+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.719+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.719+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.724+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:46:54.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex closed 2023-10-21T04:46:54.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:54.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:46:54.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:46:54.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex sucessfully destroyed 2023-10-21T04:46:54.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:46:54.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:46:54.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.731+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index destroyed 2023-10-21T04:46:54.733+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.734+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.734+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.738+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.738+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.740+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.740+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:46:54.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex started 2023-10-21T04:46:54.740+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.740+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.749+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.750+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:46:54.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex started 2023-10-21T04:46:54.750+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Warmup took 44.864444ms 2023-10-21T04:46:54.750+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 5 WriterThreads 1 2023-10-21T04:46:54.751+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Stopped Billing WUs 2023-10-21T04:46:54.751+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index Index: 12396348969790516872 Partition: 5 2023-10-21T04:46:54.751+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.750+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.750+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.752+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 6 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.752+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.752+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:46:54.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.753+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.753+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:46:54.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex started 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.776+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex started 2023-10-21T04:46:54.776+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:46:54.776+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:46:54.776+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:46:54.776+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex stopped 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.776+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.785+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:46:54.785+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex closed 2023-10-21T04:46:54.785+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:54.785+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.785+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.786+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:46:54.786+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.786+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:46:54.786+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:46:54.786+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex stopped 2023-10-21T04:46:54.787+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.787+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.787+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.787+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.796+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:46:54.796+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex closed 2023-10-21T04:46:54.796+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:54.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:46:54.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:46:54.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex sucessfully destroyed 2023-10-21T04:46:54.800+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:54.800+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:54.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:46:54.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:46:54.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index destroyed 2023-10-21T04:46:54.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.805+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.812+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.812+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.812+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:46:54.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex started 2023-10-21T04:46:54.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.817+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.817+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:46:54.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex started 2023-10-21T04:46:54.818+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Warmup took 66.08876ms 2023-10-21T04:46:54.818+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.818+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.819+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 6 WriterThreads 1 2023-10-21T04:46:54.819+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Stopped Billing WUs 2023-10-21T04:46:54.819+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index Index: 12396348969790516872 Partition: 6 2023-10-21T04:46:54.820+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.820+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 7 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.821+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.821+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:46:54.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.822+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.822+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:46:54.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.825+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex started 2023-10-21T04:46:54.825+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.825+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.828+05:30 [Info] switched currmeta from 222 -> 223 force true 2023-10-21T04:46:54.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.829+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex started 2023-10-21T04:46:54.830+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.830+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.830+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:46:54.830+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:54.831+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.831+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.831+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:46:54.831+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.832+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:46:54.833+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:46:54.834+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex stopped 2023-10-21T04:46:54.836+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.836+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.836+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.839+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.853+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:46:54.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex closed 2023-10-21T04:46:54.853+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:54.854+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.854+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.854+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:46:54.854+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.854+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:46:54.855+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:46:54.855+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex stopped 2023-10-21T04:46:54.855+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.855+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.855+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.855+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:46:54.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex closed 2023-10-21T04:46:54.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:54.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:46:54.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:46:54.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex sucessfully destroyed 2023-10-21T04:46:54.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:46:54.872+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken76:ee:1e:24:89:be:a6:21 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8379234217984719924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12957729777462041899 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14610256586894580199 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:46:54.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index destroyed 2023-10-21T04:46:54.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.890+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.890+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.895+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.895+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.908+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.909+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.909+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:46:54.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex started 2023-10-21T04:46:54.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.911+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.911+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:46:54.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex started 2023-10-21T04:46:54.911+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Warmup took 90.785431ms 2023-10-21T04:46:54.912+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 7 WriterThreads 1 2023-10-21T04:46:54.912+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Stopped Billing WUs 2023-10-21T04:46:54.912+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index Index: 12396348969790516872 Partition: 7 2023-10-21T04:46:54.912+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.912+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 8 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.913+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.913+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.913+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.913+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:46:54.914+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:46:54.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.919+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.919+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex started 2023-10-21T04:46:54.919+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.919+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.925+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex started 2023-10-21T04:46:54.925+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:46:54.926+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:54.926+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.926+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.926+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:46:54.927+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.927+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:46:54.927+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:46:54.927+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex stopped 2023-10-21T04:46:54.927+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.926+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.926+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.940+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:46:54.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex closed 2023-10-21T04:46:54.941+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:54.941+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:54.941+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:54.942+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:54.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:46:54.942+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.942+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:46:54.942+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:46:54.942+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex stopped 2023-10-21T04:46:54.942+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:54.943+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:54.942+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken76:ee:1e:24:89:be:a6:21 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8379234217984719924 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12957729777462041899 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14610256586894580199 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:46:54.943+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:54.943+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:54.956+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:54.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:46:54.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex closed 2023-10-21T04:46:54.956+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:54.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:46:54.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:46:54.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex sucessfully destroyed 2023-10-21T04:46:54.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:46:54.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:46:54.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index destroyed 2023-10-21T04:46:54.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.977+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:54.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.982+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:54.982+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:54.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:54.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:54.987+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.988+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:46:54.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex started 2023-10-21T04:46:54.988+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.989+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.989+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.989+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:46:54.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex started 2023-10-21T04:46:54.989+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Warmup took 76.568034ms 2023-10-21T04:46:54.989+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 8 WriterThreads 1 2023-10-21T04:46:54.989+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Stopped Billing WUs 2023-10-21T04:46:54.989+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index Index: 12396348969790516872 Partition: 8 2023-10-21T04:46:54.989+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:54.990+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 1 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:54.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:54.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 45511 recovery UsedSpace 8236 2023-10-21T04:46:54.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:54.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:46:54.991+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:46:54.992+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:54.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:54.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:54.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:54.999+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:54.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex started 2023-10-21T04:46:54.999+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:54.999+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:55.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:55.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:55.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:55.002+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:55.002+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex started 2023-10-21T04:46:55.003+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:46:55.003+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:55.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:46:55.003+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:46:55.003+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:55.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex stopped 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:55.003+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:55.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:46:55.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex closed 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:46:55.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:46:55.014+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:46:55.014+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:55.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex stopped 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:46:55.014+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:46:55.027+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:46:55.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:46:55.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex closed 2023-10-21T04:46:55.028+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:46:55.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:46:55.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:46:55.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:55.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex sucessfully destroyed 2023-10-21T04:46:55.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:46:55.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:46:55.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:55.037+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index destroyed 2023-10-21T04:46:55.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:46:55.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:55.043+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:55.043+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:55.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:46:55.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:55.048+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:46:55.049+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:46:55.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:55.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:55.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:55.053+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:55.054+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:46:55.054+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex started 2023-10-21T04:46:55.054+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:55.054+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:55.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:55.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:55.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:55.056+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:55.056+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:46:55.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex started 2023-10-21T04:46:55.056+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Warmup took 66.502593ms 2023-10-21T04:46:55.057+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 1 WriterThreads 1 2023-10-21T04:46:55.057+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Stopped Billing WUs 2023-10-21T04:46:55.057+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index Index: 12396348969790516872 Partition: 1 2023-10-21T04:46:55.057+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12396348969790516872, shardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:46:55.057+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12396348969790516872, partnShardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:46:55.057+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:55.057+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12396348969790516872 2023-10-21T04:46:55.057+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12396348969790516872 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:55.057+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:46:55.057+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:46:55.058+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12396348969790516872 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:55.058+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:55.058+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12396348969790516872 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:46:55.058+05:30 [Info] StorageMgr::openSnapshot IndexInst:12396348969790516872 Partition:7 No Snapshot Found. 2023-10-21T04:46:55.058+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12396348969790516872 Adding Nil Snapshot. 2023-10-21T04:46:55.058+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:55.058+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:55.099+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 40.549603ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:55.099+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 212.785µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:46:55.101+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[12396348969790516872:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:46:55.101+05:30 [Info] Indexer::handleMergePartition Source 7756628172397162046 Target 12396348969790516872 2023-10-21T04:46:55.101+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc009ec38c0 2023-10-21T04:46:55.101+05:30 [Info] MergePartition: sent async request to update index instance 12396348969790516872 rstate moved to ACTIVE 2023-10-21T04:46:55.109+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.896771ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:55.109+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 7756628172397162046, realInst: 12396348969790516872 2023-10-21T04:46:55.114+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 128675491261474353, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:46:55.115+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 128675491261474353, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:46:55.198+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 7693136708512810836,realInstId: 0, indexDefn DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:46:55.198+05:30 [Info] Indexer::handleRecoverIndex InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:46:55.198+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 232.379µs 2023-10-21T04:46:55.198+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 280.193µs 2023-10-21T04:46:55.200+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 7693136708512810836 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:55.200+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 7693136708512810836 PartitionId: 0 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:55.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:55.202+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1056266 memSz 2076 2023-10-21T04:46:55.202+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 18 count: 1000 metaSz: 2504 2023-10-21T04:46:55.203+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 7693136708512810836, indexDefn DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:46:55.203+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 88.8954ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:55.204+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 135258 recovery UsedSpace 16457 2023-10-21T04:46:55.207+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:55.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1055832 memSz 682 2023-10-21T04:46:55.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 18 count: 1000 metaSz: 2504 2023-10-21T04:46:55.207+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 136962 recovery UsedSpace 13752 2023-10-21T04:46:55.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex 2023-10-21T04:46:55.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:55.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:55.214+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex 2023-10-21T04:46:55.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex started 2023-10-21T04:46:55.214+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:55.215+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:55.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 22 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex 2023-10-21T04:46:55.220+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:55.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:55.221+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex 2023-10-21T04:46:55.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex started 2023-10-21T04:46:55.221+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7693136708512810836 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:46:55.222+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:55.222+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:55.246+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:46:55.251+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 7693136708512810836 PartitionId 0 Warmup took 50.924689ms 2023-10-21T04:46:55.252+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 7693136708512810836 partitionId 0 WriterThreads 4 2023-10-21T04:46:55.252+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 7693136708512810836 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index Index: 7693136708512810836 Partition: 0 2023-10-21T04:46:55.252+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 7693136708512810836, shardIdMap: map[0:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:46:55.252+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 7693136708512810836, partnShardIdMap: map[0:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:46:55.252+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:55.252+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 7693136708512810836 2023-10-21T04:46:55.252+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:46:55.252+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:46:55.252+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:46:55.253+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:46:55.253+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:55.253+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 7693136708512810836 Partitions [0] 2023-10-21T04:46:55.253+05:30 [Info] StorageMgr::openSnapshot IndexInst:7693136708512810836 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:46:55.269+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.934094ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:55.269+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 90.869µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:46:55.292+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:55.292+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:55.303+05:30 [Info] switched currmeta from 223 -> 227 force true 2023-10-21T04:46:55.310+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:55.310+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:55.314+05:30 [Info] switched currmeta from 227 -> 227 force true 2023-10-21T04:46:55.548+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:55.548+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:55.559+05:30 [Info] switched currmeta from 227 -> 228 force true 2023-10-21T04:46:55.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9984953192986419, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:46:55.829+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:55.829+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:55.834+05:30 [Info] switched currmeta from 228 -> 229 force true 2023-10-21T04:46:56.248+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[7693136708512810836:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:46:56.249+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:46:56.249+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:46:56.262+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4397940545762329850,realInstId: 0, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:46:56.262+05:30 [Info] Indexer::handleRecoverIndex InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:46:56.262+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 333.785µs 2023-10-21T04:46:56.262+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 381.254µs 2023-10-21T04:46:56.262+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.263+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 4 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.263+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4397940545762329850, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:46:56.263+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 13.929876ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:56.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.263+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.263+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.263+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.263+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.263+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 72388 recovery UsedSpace 103760 2023-10-21T04:46:56.264+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 76912 recovery UsedSpace 103760 2023-10-21T04:46:56.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:46:56.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.269+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:46:56.269+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex started 2023-10-21T04:46:56.269+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.269+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:46:56.271+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.272+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:46:56.272+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex started 2023-10-21T04:46:56.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.272+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:46:56.281+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:46:56.293+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:56.293+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:56.298+05:30 [Info] switched currmeta from 229 -> 230 force true 2023-10-21T04:46:56.313+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Warmup took 50.272546ms 2023-10-21T04:46:56.313+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 4 WriterThreads 1 2023-10-21T04:46:56.313+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Stopped Billing WUs 2023-10-21T04:46:56.313+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index Index: 4397940545762329850 Partition: 4 2023-10-21T04:46:56.314+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.314+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 5 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.314+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.314+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.314+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.314+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.314+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 72388 recovery UsedSpace 103760 2023-10-21T04:46:56.314+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 103760 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.323+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex started 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.323+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:46:56.323+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex started 2023-10-21T04:46:56.324+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:46:56.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.323+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.350+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Warmup took 35.674495ms 2023-10-21T04:46:56.350+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 5 WriterThreads 1 2023-10-21T04:46:56.350+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Stopped Billing WUs 2023-10-21T04:46:56.350+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index Index: 4397940545762329850 Partition: 5 2023-10-21T04:46:56.350+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.351+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 6 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2478 rp [1] sn: 22 count: 0 metaSz: 2479 2023-10-21T04:46:56.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2478 rp [1] sn: 22 count: 0 metaSz: 2479 2023-10-21T04:46:56.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 72388 recovery UsedSpace 103760 2023-10-21T04:46:56.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 103760 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.356+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex started 2023-10-21T04:46:56.356+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.356+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.356+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:46:56.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex started 2023-10-21T04:46:56.356+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:46:56.357+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.357+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.374+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Warmup took 23.779902ms 2023-10-21T04:46:56.375+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 6 WriterThreads 1 2023-10-21T04:46:56.375+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Stopped Billing WUs 2023-10-21T04:46:56.375+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index Index: 4397940545762329850 Partition: 6 2023-10-21T04:46:56.375+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.375+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 7 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.375+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.375+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.375+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2478 rp [1] sn: 22 count: 0 metaSz: 2479 2023-10-21T04:46:56.375+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 76912 recovery UsedSpace 103760 2023-10-21T04:46:56.376+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.376+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.376+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2478 rp [1] sn: 22 count: 0 metaSz: 2479 2023-10-21T04:46:56.376+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 103760 2023-10-21T04:46:56.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:46:56.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.383+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:46:56.383+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex started 2023-10-21T04:46:56.383+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.383+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:46:56.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.384+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:46:56.384+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex started 2023-10-21T04:46:56.384+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:46:56.384+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.384+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.406+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Warmup took 30.500691ms 2023-10-21T04:46:56.406+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 7 WriterThreads 1 2023-10-21T04:46:56.406+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Stopped Billing WUs 2023-10-21T04:46:56.406+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index Index: 4397940545762329850 Partition: 7 2023-10-21T04:46:56.406+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.406+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 8 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.406+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.406+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2478 rp [1] sn: 22 count: 0 metaSz: 2479 2023-10-21T04:46:56.406+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 76912 recovery UsedSpace 103760 2023-10-21T04:46:56.406+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2478 rp [1] sn: 22 count: 0 metaSz: 2479 2023-10-21T04:46:56.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 103760 2023-10-21T04:46:56.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:46:56.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.415+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:46:56.415+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex started 2023-10-21T04:46:56.415+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.415+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:46:56.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.418+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:46:56.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex started 2023-10-21T04:46:56.418+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:46:56.418+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.419+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.437+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Warmup took 31.247596ms 2023-10-21T04:46:56.438+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 8 WriterThreads 1 2023-10-21T04:46:56.438+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Stopped Billing WUs 2023-10-21T04:46:56.438+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index Index: 4397940545762329850 Partition: 8 2023-10-21T04:46:56.438+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.438+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 1 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 76912 recovery UsedSpace 103760 2023-10-21T04:46:56.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 103760 2023-10-21T04:46:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:46:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.456+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:46:56.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex started 2023-10-21T04:46:56.456+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.456+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:46:56.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.459+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:46:56.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex started 2023-10-21T04:46:56.459+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:46:56.459+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.459+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.482+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Warmup took 43.925506ms 2023-10-21T04:46:56.482+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 1 WriterThreads 1 2023-10-21T04:46:56.482+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Stopped Billing WUs 2023-10-21T04:46:56.482+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index Index: 4397940545762329850 Partition: 1 2023-10-21T04:46:56.482+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.483+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 2 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.483+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.483+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.483+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.483+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.483+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2478 rp [1] sn: 23 count: 0 metaSz: 2479 2023-10-21T04:46:56.483+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 76912 recovery UsedSpace 103760 2023-10-21T04:46:56.483+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 103760 2023-10-21T04:46:56.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:46:56.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.488+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:46:56.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex started 2023-10-21T04:46:56.488+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.488+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:46:56.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.488+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:46:56.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex started 2023-10-21T04:46:56.488+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:46:56.489+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.489+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.508+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Warmup took 25.385156ms 2023-10-21T04:46:56.508+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 2 WriterThreads 1 2023-10-21T04:46:56.508+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Stopped Billing WUs 2023-10-21T04:46:56.508+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index Index: 4397940545762329850 Partition: 2 2023-10-21T04:46:56.508+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:46:56.509+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 3 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:46:56.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:46:56.509+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.509+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 20 count: 0 metaSz: 2478 rp [1] sn: 21 count: 0 metaSz: 2479 2023-10-21T04:46:56.509+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 103760 2023-10-21T04:46:56.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:46:56.510+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:46:56.510+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: RP version: 13 numRPs: 2 rp [0] sn: 20 count: 0 metaSz: 2478 rp [1] sn: 21 count: 0 metaSz: 2479 2023-10-21T04:46:56.510+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 103760 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:46:56.516+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex started 2023-10-21T04:46:56.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.516+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:46:56.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex started 2023-10-21T04:46:56.516+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:46:56.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:46:56.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:46:56.529+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Warmup took 20.487213ms 2023-10-21T04:46:56.529+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 3 WriterThreads 1 2023-10-21T04:46:56.530+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Stopped Billing WUs 2023-10-21T04:46:56.530+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index Index: 4397940545762329850 Partition: 3 2023-10-21T04:46:56.530+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4397940545762329850, shardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:46:56.530+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4397940545762329850, partnShardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:46:56.530+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:56.531+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4397940545762329850 2023-10-21T04:46:56.531+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:56.531+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:46:56.531+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:46:56.531+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:56.531+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:56.532+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4397940545762329850 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:46:56.532+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:1 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.533+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:2 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.535+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:3 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.535+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:4 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.536+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:5 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.537+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:6 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.537+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:7 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.538+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:8 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:46:56.546+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.553685ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:56.546+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 99.067µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:46:56.554+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:56.554+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:56.558+05:30 [Info] switched currmeta from 230 -> 231 force true 2023-10-21T04:46:56.603+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:56.603+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:56.610+05:30 [Info] switched currmeta from 231 -> 232 force true 2023-10-21T04:46:56.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8574864158834902, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:46:57.120+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:57.120+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:57.125+05:30 [Info] switched currmeta from 232 -> 233 force true 2023-10-21T04:46:57.283+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4397940545762329850:INDEX_STATE_RECOVERED 7693136708512810836:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:46:57.283+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [128675491261474353 4390029557575378783] belonging to collectionId: c, ttid: ShardToken85:6a:aa:c2:29:3:d1:a5. Initiating build 2023-10-21T04:46:57.327+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [7693136708512810836 4397940545762329850] &{2} 2023-10-21T04:46:57.327+05:30 [Info] Indexer::handleBuildRecoveredIndexes [7693136708512810836 4397940545762329850] 2023-10-21T04:46:57.327+05:30 [Info] bulkUpdateRState: Index instance 7693136708512810836 rstate moved to PENDING 2023-10-21T04:46:57.327+05:30 [Info] bulkUpdateRState: Index instance 4397940545762329850 rstate moved to PENDING 2023-10-21T04:46:57.327+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [7693136708512810836 4397940545762329850] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:46:57.327+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:57.327+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:46:57.327+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:57.330+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:57.330+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:57.348+05:30 [Info] switched currmeta from 233 -> 235 force true 2023-10-21T04:46:57.356+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:57.366+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:57.368+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:46:57.368+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:46:57.368+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:46:57.368+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:46:57.368+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:46:57.368+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:46:57.368+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:46:57.368+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:46:57.368+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:46:57.368+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId 43323 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 28 0 28} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 21 0 21} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 36 0 36} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 25 0 25} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 27 0 27} 2023-10-21T04:46:57.369+05:30 [Info] Timekeeper::handleStreamOpen Stream INIT_STREAM State Changed to ACTIVE 2023-10-21T04:46:57.369+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:46:57.369+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:46:57.369+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 7693136708512810836 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:46:57.369+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[INIT_STREAM:map[bucket_%2:_default:c1:2]] 2023-10-21T04:46:57.369+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4397940545762329850 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:46:57.369+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:46:57.369+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=19,1=28,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=21,17=20,18=20,19=25,20=23,21=28,22=30,23=18,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=36,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=25,57=18,58=26,59=16,60=20,61=24,62=20,63=27] IndexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 43323 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 28 0 28} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 21 0 21} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 36 0 36} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 25 0 25} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 27 0 27} NumVBuckets: 64 2023-10-21T04:46:57.370+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:46:57.370+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:46:57.370+05:30 [Info] switched currmeta from 235 -> 235 force true 2023-10-21T04:46:57.371+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:57.371+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 44.203857ms 2023-10-21T04:46:57.371+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 44.253131ms 2023-10-21T04:46:57.371+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [7693136708512810836 4397940545762329850] 2023-10-21T04:46:57.371+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 83.562325ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:57.372+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4397940545762329850:true 7693136708512810836:true] 2023-10-21T04:46:57.380+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:46:57.382+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.931645ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:57.386+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:46:57.400+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.982876ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:57.409+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 6, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:46:57.421+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:46:57.422+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 16] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/ab759b18c08b0e81e45c6631c4e2d845 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:ab759b18c08b0e81e45c6631c4e2d845]}]} 2023-10-21T04:46:57.434+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:46:57.434+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 16] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:46:57.442+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:58466" doAuth() ... 2023-10-21T04:46:57.443+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:58466" auth successful 2023-10-21T04:46:57.450+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:58466" +1 2023-10-21T04:46:57.460+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/ab759b18c08b0e81e45c6631c4e2d845, rev: [] 2023-10-21T04:46:57.461+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/ab759b18c08b0e81e45c6631c4e2d845, rev: [] 2023-10-21T04:46:57.463+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:46:57.464+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:46:57.465+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:46:57.466+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:46:57.481+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:57.481+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:57.486+05:30 [Info] switched currmeta from 235 -> 237 force true 2023-10-21T04:46:57.633+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:57.633+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:57.663+05:30 [Info] switched currmeta from 237 -> 237 force true 2023-10-21T04:46:57.663+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:46:57.857+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:57.857+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:57.872+05:30 [Info] switched currmeta from 237 -> 241 force true 2023-10-21T04:46:58.079+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:46:58.087+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:58.179+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:58.179+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:58.200+05:30 [Info] switched currmeta from 241 -> 243 force true 2023-10-21T04:46:58.238+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:58.238+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:58.258+05:30 [Info] switched currmeta from 243 -> 243 force true 2023-10-21T04:46:58.280+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:58.280+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:58.350+05:30 [Info] switched currmeta from 243 -> 243 force true 2023-10-21T04:46:58.797+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 220432454230336 19 43323 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:58.800+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 14580309443106 28 43323 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:46:58.801+05:30 [Info] NewTsVbuuidCached: bucket_%2, 64 TsVbuuidPoolMap is updated from: map[] to: map[64:0xc00948eab0] 2023-10-21T04:46:58.802+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 135940091877708 21 43323 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:58.802+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 80577332946777 27 43323 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:58.805+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 247041243079227 24 43323 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:58.808+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 19010051182641 24 43323 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:58.808+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 172199206487624 16 43323 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:46:58.808+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 194695637011621 26 43323 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:58.811+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 182410957121870 32 43323 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:46:58.811+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 42756027613197 22 43323 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:58.814+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 121484100363817 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.816+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 68019130447759 21 43323 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:58.818+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 171853572917441 25 43323 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:58.820+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 107902021898438 23 43323 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:46:58.823+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 1542343912406 21 43323 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:58.823+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 59183965998125 22 43323 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:58.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 157626280229441 21 43323 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:58.827+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 73693878880740 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 67844364342499 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.830+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 170065152021297 25 43323 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:58.832+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 217154782530068 23 43323 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:46:58.835+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 194038385561911 28 43323 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:46:58.837+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 16864753682212 30 43323 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:46:58.839+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 206194254469959 18 43323 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:46:58.841+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 170842850732116 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.843+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 233111027052204 28 43323 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:46:58.845+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 58389099238647 24 43323 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:58.847+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 178705476290398 24 43323 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:58.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 252516046855767 27 43323 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:58.852+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 52334512457128 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.853+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 183084439776991 19 43323 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:58.856+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 270670782796252 22 43323 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:58.858+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 86344521159606 25 43323 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:58.863+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 264478627574156 22 43323 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:58.863+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 233119567174084 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.865+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 187961911866991 30 43323 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:46:58.867+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 94748374195527 19 43323 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:58.867+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 276412965411368 22 43323 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:58.869+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 30543404237888 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.872+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 66391235022367 25 43323 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:58.873+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 168721574304277 24 43323 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:58.875+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 50379500651629 21 43323 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:58.877+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 100004485169771 26 43323 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:58.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 148816420612283 27 43323 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:58.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 236414409908581 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 199348794242318 36 43323 172.31.5.112:9000. HWT [0-36,36]. 2023-10-21T04:46:58.884+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 199648020746400 23 43323 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:46:58.887+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 131255717944569 26 43323 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:58.887+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 84841900064407 22 43323 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:58.889+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 229485491476250 19 43323 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:58.891+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 28675906453951 21 43323 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:58.891+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 27094034037488 19 43323 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:58.895+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 268905923659105 24 43323 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:58.896+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 7777834518711 31 43323 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:46:58.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 70030494603811 21 43323 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:58.900+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 75978859756835 17 43323 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:46:58.903+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 235447613990794 25 43323 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:58.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 43207585340528 18 43323 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:46:58.907+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 47128363244154 26 43323 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:58.909+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 160015043890055 16 43323 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:46:58.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 65202412270069 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.914+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 170481067504432 20 43323 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:58.914+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 154979908447073 24 43323 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:58.917+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 14950536761368 27 43323 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:58.922+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 InstanceIds [4397940545762329850 7693136708512810836] 2023-10-21T04:46:58.922+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 43323 2023-10-21T04:46:58.922+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43323 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 4397940545762329850 INDEX_STATE_CATCHUP 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 7693136708512810836 INDEX_STATE_CATCHUP 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 4397940545762329850 true 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 7693136708512810836 true 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 7693136708512810836 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 43323 BuildTS: [0=19,1=28,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=21,17=20,18=20,19=25,20=23,21=28,22=30,23=18,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=36,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=25,57=18,58=26,59=16,60=20,61=24,62=20,63=27] 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 28 0 28} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 21 0 21} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 36 0 36} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 25 0 25} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 27 0 27} , fetchKVSeq false 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:46:58.922+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:46:58.922+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43323 STREAM_ACTIVE 2023-10-21T04:46:58.923+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 43323 2023-10-21T04:46:58.923+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2:_default:c1 43323 2023-10-21T04:46:58.923+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:58.923+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:46:58.923+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:46:58.923+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:46:58.923+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:46:58.923+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:46:58.924+05:30 [Info] send progress stats to clients 2023-10-21T04:46:58.924+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43323 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:46:58.924+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 7693136708512810836 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:46:58.924+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4397940545762329850 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:46:58.924+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:46:58.924+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:46:58.924+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:46:58.924+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43323 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:46:58.924+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:46:58.924+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:46:58.924+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:46:58.925+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:46:58.926+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:58466: use of closed network connection 2023-10-21T04:46:58.926+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:58466" exit: read tcp4 127.0.0.1:9121->127.0.0.1:58466: use of closed network connection 2023-10-21T04:46:58.926+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:58466" closed ! 2023-10-21T04:46:58.926+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:46:58.926+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:58466" already gone 2023-10-21T04:46:58.926+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2:_default:c1 2023-10-21T04:46:58.927+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:46:58.937+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.675068ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:58.941+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:46:58.941+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:46:58.941+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:46:58.941+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43323 2023-10-21T04:46:58.941+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:46:58.941+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:46:58.941+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:46:58.941+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:46:58.941+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:46:58.941+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:46:58.941+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:46:58.941+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:46:58.941+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:46:58.941+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:46:58.942+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 73678 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 28 0 28} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 21 0 21} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 36 0 36} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 25 0 25} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 27 0 27} 2023-10-21T04:46:58.942+05:30 [Info] Timekeeper::handleStreamOpen Stream MAINT_STREAM State Changed to ACTIVE 2023-10-21T04:46:58.942+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:46:58.942+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1] INIT_STREAM:map[]] 2023-10-21T04:46:58.942+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2] INIT_STREAM:map[]] 2023-10-21T04:46:58.942+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:46:58.942+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 73678 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 28 0 28} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 21 0 21} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 36 0 36} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 25 0 25} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 27 0 27} NumVBuckets: 64 2023-10-21T04:46:58.943+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:46:58.943+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:46:58.956+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:46:58.977+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 39.428302ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:58.992+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:58.992+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:59.002+05:30 [Info] switched currmeta from 243 -> 245 force true 2023-10-21T04:46:59.006+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:54292" doAuth() ... 2023-10-21T04:46:59.006+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:54292" auth successful 2023-10-21T04:46:59.010+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:59.010+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:59.012+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:54292" +1 2023-10-21T04:46:59.016+05:30 [Info] switched currmeta from 245 -> 245 force true 2023-10-21T04:46:59.252+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 220432454230336 19 73678 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:59.255+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 14580309443106 28 73678 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:46:59.256+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 80577332946777 27 73678 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:59.258+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 135940091877708 21 73678 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:59.259+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 247041243079227 24 73678 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:59.263+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 194695637011621 26 73678 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:59.263+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 172199206487624 16 73678 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:46:59.265+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 182410957121870 32 73678 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:46:59.265+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 19010051182641 24 73678 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:59.267+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 42756027613197 22 73678 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:59.272+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 68019130447759 21 73678 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:59.272+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 121484100363817 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.274+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 107902021898438 23 73678 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:46:59.274+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 171853572917441 25 73678 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:59.277+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 59183965998125 22 73678 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:59.293+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 1542343912406 21 73678 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:59.306+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 157626280229441 21 73678 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:59.318+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 16864753682212 30 73678 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:46:59.318+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 67844364342499 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.319+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 73693878880740 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.319+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 170065152021297 25 73678 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:59.319+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 217154782530068 23 73678 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:46:59.324+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 170842850732116 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.324+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 206194254469959 18 73678 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:46:59.329+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 58389099238647 24 73678 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:59.340+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 178705476290398 24 73678 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:59.342+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 252516046855767 27 73678 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:59.346+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 183084439776991 19 73678 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:59.346+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 233111027052204 28 73678 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:46:59.346+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 194038385561911 28 73678 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:46:59.349+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 52334512457128 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.349+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 270670782796252 22 73678 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:59.351+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 86344521159606 25 73678 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:59.355+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 233119567174084 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.355+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 264478627574156 22 73678 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:59.356+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 187961911866991 30 73678 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:46:59.365+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 94748374195527 19 73678 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:59.365+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 276412965411368 22 73678 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:59.368+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 148816420612283 27 73678 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:59.368+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 168721574304277 24 73678 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:59.368+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 30543404237888 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.368+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 100004485169771 26 73678 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:59.368+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 66391235022367 25 73678 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:59.373+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 199648020746400 23 73678 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:46:59.373+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 236414409908581 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.378+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 84841900064407 22 73678 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:46:59.378+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 199348794242318 36 73678 172.31.5.112:9000. HWT [0-36,36]. 2023-10-21T04:46:59.378+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 50379500651629 21 73678 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:59.381+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 131255717944569 26 73678 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:59.381+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 229485491476250 19 73678 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:59.384+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 27094034037488 19 73678 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:46:59.385+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 28675906453951 21 73678 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:59.407+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 268905923659105 24 73678 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:59.411+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 75978859756835 17 73678 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:46:59.411+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 7777834518711 31 73678 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:46:59.411+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 70030494603811 21 73678 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:46:59.414+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 43207585340528 18 73678 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:46:59.414+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 235447613990794 25 73678 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:46:59.420+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 47128363244154 26 73678 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:46:59.420+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 160015043890055 16 73678 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:46:59.422+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 65202412270069 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.425+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 154979908447073 24 73678 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:46:59.428+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 170481067504432 20 73678 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:46:59.434+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2 InstanceIds [7693136708512810836 4397940545762329850] 2023-10-21T04:46:59.434+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 73678 2023-10-21T04:46:59.434+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 73678 2023-10-21T04:46:59.434+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:46:59.434+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:46:59.434+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:46:59.434+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:46:59.434+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 73678 2023-10-21T04:46:59.434+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:46:59.434+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 73678 STREAM_ACTIVE 2023-10-21T04:46:59.434+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:46:59.434+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 14950536761368 27 73678 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:46:59.440+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:46:59.440+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:46:59.443+05:30 [Info] switched currmeta from 245 -> 245 force true 2023-10-21T04:46:59.481+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:57832 2023-10-21T04:46:59.496+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 10.599344ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:46:59.516+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:57832. Reason = EOF. Cleaning Pipe. 2023-10-21T04:46:59.516+05:30 [Info] messageListener.start(): message channel closed. Remove peer 40:d9:c8:f4:82:f6:2d:ff as follower. 2023-10-21T04:46:59.923+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:46:59.997+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:00.275+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:47:00.302+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:47:00.435+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:47:00.435+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:47:01.308+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:47:01.418+05:30 [Info] ShardRebalancer::waitForIndexState Cancel Received 2023-10-21T04:47:01.418+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken85:6a:aa:c2:29:3:d1:a5 2023-10-21T04:47:01.419+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:47:01.423+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:47:01.423+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken85:6a:aa:c2:29:3:d1:a5 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3b:e7:9c:c8:dd:24:1e:45 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 7756628172397162046 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 7693136708512810836 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3851262100190137504 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:47:01.423+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: ab759b18c08b0e81e45c6631c4e2d845 TransferTokenId: ShardToken85:6a:aa:c2:29:3:d1:a5 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:47:01.423+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000119454 2023-10-21T04:47:01.423+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken85:6a:aa:c2:29:3:d1:a5, dropIndexes: true 2023-10-21T04:47:01.423+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5, shardId :0 2023-10-21T04:47:01.423+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardToken85_6a_aa_c2_29_3_d1_a5, shardId :0 2023-10-21T04:47:01.424+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 17703248484023971553 instance id 7756628172397162046 real instance id 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:47:01.424+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 17703248484023971553 instance 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:47:01.424+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 17703248484023971553 instance id 12396348969790516872 2023-10-21T04:47:01.424+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 17703248484023971553 2023-10-21T04:47:01.431+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12396348969790516872 &{1} 2023-10-21T04:47:01.431+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12396348969790516872 2023-10-21T04:47:01.432+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17703248484023971553, instId: 12396348969790516872 2023-10-21T04:47:01.432+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12396348969790516872] 2023-10-21T04:47:01.432+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12396348969790516872] 2023-10-21T04:47:01.432+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:47:01.432+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12396348969790516872] 2023-10-21T04:47:01.433+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.433+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.434+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.434+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:47:01.434+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:47:01.434+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:47:01.434+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:47:01.435+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:47:01.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex stopped 2023-10-21T04:47:01.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.435+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:47:01.436+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:47:01.436+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:47:01.436+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:47:01.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex stopped 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.436+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.437+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:47:01.437+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:47:01.437+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:47:01.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex stopped 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.437+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:47:01.437+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:47:01.437+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:47:01.437+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex stopped 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.437+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.437+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:47:01.438+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 6.305893ms 2023-10-21T04:47:01.438+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 6.344344ms 2023-10-21T04:47:01.438+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:47:01.438+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:47:01.438+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:47:01.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex stopped 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.438+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:47:01.438+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:47:01.438+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:47:01.438+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex stopped 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.438+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.439+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12396348969790516872 2023-10-21T04:47:01.435+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.439+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.439+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.439+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:47:01.440+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:47:01.440+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:47:01.440+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex stopped 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.440+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:47:01.440+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:47:01.440+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex stopped 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.440+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.487+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:01.526+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:01.488+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 64.51186ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:01.490+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:47:01.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex closed 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:47:01.537+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:47:01.537+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:47:01.537+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex stopped 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.537+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.501+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:47:01.538+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex closed 2023-10-21T04:47:01.539+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.503+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.525+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.540+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.540+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:47:01.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex closed 2023-10-21T04:47:01.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:47:01.544+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:47:01.544+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:47:01.544+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:47:01.544+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:47:01.545+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex stopped 2023-10-21T04:47:01.545+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.545+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:47:01.543+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.545+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.546+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex closed 2023-10-21T04:47:01.546+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:47:01.547+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:47:01.547+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:47:01.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex stopped 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.547+05:30 [Info] switched currmeta from 245 -> 247 force true 2023-10-21T04:47:01.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:47:01.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex closed 2023-10-21T04:47:01.548+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:47:01.554+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.554+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 7693136708512810836 &{1} 2023-10-21T04:47:01.557+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:47:01.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex closed 2023-10-21T04:47:01.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.574+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.574+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:47:01.575+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:47:01.575+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:47:01.576+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:47:01.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex stopped 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.575+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.576+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.575+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:01.576+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:01.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex closed 2023-10-21T04:47:01.577+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.578+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.575+05:30 [Info] Indexer::handleDropIndex - IndexInstId 7693136708512810836 2023-10-21T04:47:01.578+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 128675491261474353, instId: 7693136708512810836 2023-10-21T04:47:01.578+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:47:01.578+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:47:01.579+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 7693136708512810836 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:47:01.579+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [7693136708512810836] 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:47:01.579+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:47:01.579+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:47:01.579+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex stopped 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.579+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex closed 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:47:01.580+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:47:01.580+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex stopped 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex closed 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:47:01.580+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:47:01.580+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex stopped 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:47:01.581+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:47:01.581+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [7693136708512810836] 2023-10-21T04:47:01.581+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:47:01.582+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [7693136708512810836] 2023-10-21T04:47:01.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:47:01.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:47:01.582+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73678 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:47:01.585+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.584+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 7693136708512810836 Defn: DefnId: 128675491261474353 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73678 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:47:01.576+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:47:01.587+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 7693136708512810836, PartitionId 0, IndexDefnId 128675491261474353. 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex] 2023-10-21T04:47:01.587+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex] 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex] 2023-10-21T04:47:01.587+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex 2023-10-21T04:47:01.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex stopped 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.587+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.588+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 12.703609ms 2023-10-21T04:47:01.588+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 7693136708512810836 2023-10-21T04:47:01.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.588+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index destroyed 2023-10-21T04:47:01.589+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.589+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 8 Destroy Done 2023-10-21T04:47:01.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:47:01.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex closed 2023-10-21T04:47:01.589+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.589+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:47:01.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:47:01.589+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [7693136708512810836] 2023-10-21T04:47:01.592+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:47:01.592+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:47:01.592+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:47:01.592+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:47:01.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex stopped 2023-10-21T04:47:01.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:47:01.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:47:01.594+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex] 2023-10-21T04:47:01.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex closed 2023-10-21T04:47:01.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.594+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.594+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.597+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.595+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:47:01.597+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73678 2023-10-21T04:47:01.598+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 9.976877ms 2023-10-21T04:47:01.598+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 22.966785ms 2023-10-21T04:47:01.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.598+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex] 2023-10-21T04:47:01.598+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index destroyed 2023-10-21T04:47:01.599+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.599+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 1 Destroy Done 2023-10-21T04:47:01.599+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex 2023-10-21T04:47:01.599+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex] 2023-10-21T04:47:01.599+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex] 2023-10-21T04:47:01.599+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex 2023-10-21T04:47:01.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex stopped 2023-10-21T04:47:01.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.599+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:47:01.604+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex closed 2023-10-21T04:47:01.605+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.605+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:47:01.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:47:01.606+05:30 [Info] switched currmeta from 247 -> 248 force true 2023-10-21T04:47:01.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:47:01.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:47:01.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.634+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index destroyed 2023-10-21T04:47:01.634+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.634+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 7 Destroy Done 2023-10-21T04:47:01.646+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:01.646+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:01.653+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.670+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:47:01.670+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex closed 2023-10-21T04:47:01.671+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.655+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 128.975475ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:01.656+05:30 [Info] switched currmeta from 248 -> 248 force true 2023-10-21T04:47:01.660+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.668+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.670+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.673+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:47:01.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:47:01.673+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.673+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:47:01.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex closed 2023-10-21T04:47:01.674+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 4390029557575378783 instance id 3851262100190137504 real instance id 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:47:01.674+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 4390029557575378783 instance 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:47:01.674+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 4390029557575378783 instance id 4397940545762329850 2023-10-21T04:47:01.676+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 4390029557575378783 2023-10-21T04:47:01.675+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex] 2023-10-21T04:47:01.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex closed 2023-10-21T04:47:01.676+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.676+05:30 [Info] Indexer::cleanupIndexData IndexInst 7693136708512810836 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:47:01.676+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex closed 2023-10-21T04:47:01.676+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.676+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:47:01.677+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex closed 2023-10-21T04:47:01.677+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.677+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:47:01.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:47:01.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex ... 2023-10-21T04:47:01.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex ... 2023-10-21T04:47:01.681+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:01.682+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:01.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:47:01.683+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:47:01.685+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.685+05:30 [Info] switched currmeta from 248 -> 249 force true 2023-10-21T04:47:01.685+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.686+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.686+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:47:01.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:47:01.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.689+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index destroyed 2023-10-21T04:47:01.692+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.692+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 2 Destroy Done 2023-10-21T04:47:01.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex ... 2023-10-21T04:47:01.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 22 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex ... 2023-10-21T04:47:01.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.697+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:47:01.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:47:01.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index/docIndex sucessfully destroyed 2023-10-21T04:47:01.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:47:01.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex closed 2023-10-21T04:47:01.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:47:01.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:47:01.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.703+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.704+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 3 Close Done All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_7693136708512810836_0.index destroyed 2023-10-21T04:47:01.705+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 7693136708512810836, PartitionId 0, IndexDefnId 128675491261474353. 2023-10-21T04:47:01.705+05:30 [Info] Indexer::cleanupIndexData IndexInst 7693136708512810836 Partition 0 Destroy Done 2023-10-21T04:47:01.708+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4397940545762329850 &{1} 2023-10-21T04:47:01.708+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4397940545762329850 2023-10-21T04:47:01.708+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 4390029557575378783, instId: 4397940545762329850 2023-10-21T04:47:01.708+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:47:01.708+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:47:01.708+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:47:01.708+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4397940545762329850] 2023-10-21T04:47:01.708+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4397940545762329850] 2023-10-21T04:47:01.708+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:47:01.708+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4397940545762329850] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:47:01.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:47:01.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index destroyed 2023-10-21T04:47:01.714+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.715+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 5 Destroy Done 2023-10-21T04:47:01.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:47:01.715+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:47:01.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.721+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index destroyed 2023-10-21T04:47:01.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:47:01.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:47:01.722+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.722+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 4 Destroy Done 2023-10-21T04:47:01.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.724+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index destroyed 2023-10-21T04:47:01.724+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.724+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 6 Destroy Done 2023-10-21T04:47:01.725+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:47:01.725+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:47:01.725+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:47:01.725+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:47:01.725+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.725+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.725+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:47:01.725+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:47:01.725+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:47:01.725+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73678 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:47:01.726+05:30 [Info] MutationStreamReader:Shutdown StreamReader MAINT_STREAM 2023-10-21T04:47:01.726+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:47:01.726+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:47:01.727+05:30 [Error] DATP[->dataport ":9123"] Accept() Error: accept tcp4 0.0.0.0:9123: use of closed network connection 2023-10-21T04:47:01.727+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9123->127.0.0.1:54292: use of closed network connection 2023-10-21T04:47:01.727+05:30 [Error] DATP[->dataport ":9123"] worker "127.0.0.1:54292" exit: read tcp4 127.0.0.1:9123->127.0.0.1:54292: use of closed network connection 2023-10-21T04:47:01.727+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:54292" closed ! 2023-10-21T04:47:01.727+05:30 [Info] DATP[->dataport ":9123"] ... stopped 2023-10-21T04:47:01.727+05:30 [Error] DATP[->dataport ":9123"] fatal remote "127.0.0.1:54292" already gone 2023-10-21T04:47:01.727+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:47:01.727+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:47:01.727+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.727+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.727+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:47:01.728+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73678 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:47:01.728+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:47:01.728+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:47:01.728+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:47:01.728+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 20.093359ms 2023-10-21T04:47:01.728+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4397940545762329850 2023-10-21T04:47:01.729+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2] 2023-10-21T04:47:01.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.736+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.737+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:47:01.737+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:47:01.737+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:47:01.737+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:47:01.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex stopped 2023-10-21T04:47:01.737+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.737+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.737+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.737+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.738+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:47:01.738+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:47:01.738+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:47:01.738+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:47:01.739+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex stopped 2023-10-21T04:47:01.739+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.739+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.739+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.739+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.740+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:47:01.740+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:47:01.772+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:47:01.774+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:47:01.773+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.774+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2]. Err genServer.closed 2023-10-21T04:47:01.774+05:30 [Error] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:47:01.774+05:30 [Info] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Treating genServer.closed As Success 2023-10-21T04:47:01.774+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73678 2023-10-21T04:47:01.774+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 46.709357ms 2023-10-21T04:47:01.774+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 66.874311ms 2023-10-21T04:47:01.774+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex stopped 2023-10-21T04:47:01.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:47:01.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex closed 2023-10-21T04:47:01.777+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.777+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.777+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.777+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:47:01.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:47:01.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:47:01.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:47:01.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex closed 2023-10-21T04:47:01.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.778+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.778+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:47:01.779+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:47:01.779+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:47:01.779+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:47:01.779+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:47:01.779+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex stopped 2023-10-21T04:47:01.779+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.779+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.780+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.780+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.780+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:47:01.780+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:47:01.781+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:47:01.781+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:47:01.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex stopped 2023-10-21T04:47:01.781+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.789+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.790+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.790+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.790+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:47:01.792+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:47:01.788+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:01.792+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:01.791+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.792+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:47:01.793+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:47:01.793+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex stopped 2023-10-21T04:47:01.793+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.792+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.793+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:47:01.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex sucessfully destroyed 2023-10-21T04:47:01.795+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.795+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:47:01.796+05:30 [Info] switched currmeta from 249 -> 250 force true 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:47:01.798+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:47:01.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex stopped 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index destroyed 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.798+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:47:01.798+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:47:01.798+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:47:01.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex stopped 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:47:01.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex closed 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:47:01.799+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:47:01.799+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:47:01.799+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:47:01.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex stopped 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.799+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:47:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex closed 2023-10-21T04:47:01.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:47:01.800+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:47:01.800+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:47:01.800+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:47:01.800+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:47:01.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex stopped 2023-10-21T04:47:01.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.801+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.801+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.801+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.799+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 3 Destroy Done 2023-10-21T04:47:01.801+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:47:01.805+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:47:01.805+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:47:01.805+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:47:01.805+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:47:01.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex stopped 2023-10-21T04:47:01.805+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.805+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.805+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.805+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex closed 2023-10-21T04:47:01.805+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:47:01.806+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:47:01.806+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:47:01.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex stopped 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.806+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:47:01.806+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:47:01.806+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex stopped 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.806+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:47:01.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex closed 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:47:01.811+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:47:01.811+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:47:01.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex stopped 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.811+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.812+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:47:01.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex closed 2023-10-21T04:47:01.813+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.813+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex ... 2023-10-21T04:47:01.814+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex ... 2023-10-21T04:47:01.813+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 139.164192ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:01.813+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.817+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.817+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:01.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:47:01.817+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:01.817+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex closed 2023-10-21T04:47:01.817+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.817+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.818+05:30 [Info] LifecycleMgr.DeleteIndex() : index 5277316367626246544 does not exist. 2023-10-21T04:47:01.818+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 72.383µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:01.818+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [18374611443167983589 6455624821495495854] 2023-10-21T04:47:01.821+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:47:01.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex closed 2023-10-21T04:47:01.822+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.822+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.823+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:47:01.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex closed 2023-10-21T04:47:01.823+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.823+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex ... 2023-10-21T04:47:01.825+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex ... 2023-10-21T04:47:01.825+05:30 [Info] switched currmeta from 250 -> 251 force true 2023-10-21T04:47:01.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:47:01.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex closed 2023-10-21T04:47:01.826+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex ... 2023-10-21T04:47:01.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex ... 2023-10-21T04:47:01.826+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.827+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.827+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.827+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:47:01.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index destroyed 2023-10-21T04:47:01.828+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.828+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 2 Destroy Done 2023-10-21T04:47:01.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:47:01.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex closed 2023-10-21T04:47:01.828+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.828+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.828+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:47:01.828+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:47:01.828+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:47:01.828+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:47:01.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex stopped 2023-10-21T04:47:01.828+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.828+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.829+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.829+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.829+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.829+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:47:01.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex closed 2023-10-21T04:47:01.830+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.831+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 1 Close Done 2023-10-21T04:47:01.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.833+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:47:01.835+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex closed 2023-10-21T04:47:01.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.835+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:01.836+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:47:01.837+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex ... 2023-10-21T04:47:01.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.838+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:01.839+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.839+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex closed 2023-10-21T04:47:01.841+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.841+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:47:01.842+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:47:01.842+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:47:01.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex ... 2023-10-21T04:47:01.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex ... 2023-10-21T04:47:01.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:47:01.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index destroyed 2023-10-21T04:47:01.843+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.843+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 3 Destroy Done 2023-10-21T04:47:01.843+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:47:01.844+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:47:01.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.849+05:30 [Info] switched currmeta from 251 -> 251 force true 2023-10-21T04:47:01.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex ... 2023-10-21T04:47:01.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex ... 2023-10-21T04:47:01.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex ... 2023-10-21T04:47:01.865+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex ... 2023-10-21T04:47:01.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex ... 2023-10-21T04:47:01.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex ... 2023-10-21T04:47:01.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex ... 2023-10-21T04:47:01.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex ... 2023-10-21T04:47:01.889+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:47:01.889+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:47:01.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex stopped 2023-10-21T04:47:01.889+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:47:01.889+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:47:01.889+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:47:01.889+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:47:01.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex ... 2023-10-21T04:47:01.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex ... 2023-10-21T04:47:01.892+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:47:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index destroyed 2023-10-21T04:47:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:47:01.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex closed 2023-10-21T04:47:01.894+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.894+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 6 Destroy Done 2023-10-21T04:47:01.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex ... 2023-10-21T04:47:01.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex ... 2023-10-21T04:47:01.895+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:47:01.895+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:47:01.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex ... 2023-10-21T04:47:01.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex ... 2023-10-21T04:47:01.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index destroyed 2023-10-21T04:47:01.900+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.900+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 5 Destroy Done 2023-10-21T04:47:01.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex ... 2023-10-21T04:47:01.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex ... 2023-10-21T04:47:01.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.900+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex sucessfully destroyed 2023-10-21T04:47:01.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index destroyed 2023-10-21T04:47:01.907+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.907+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 8 Destroy Done 2023-10-21T04:47:01.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex ... 2023-10-21T04:47:01.907+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex ... 2023-10-21T04:47:01.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index destroyed 2023-10-21T04:47:01.914+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.914+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 1 Destroy Done 2023-10-21T04:47:01.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex ... 2023-10-21T04:47:01.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex ... 2023-10-21T04:47:01.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index destroyed 2023-10-21T04:47:01.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex ... 2023-10-21T04:47:01.920+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex ... 2023-10-21T04:47:01.920+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 4 Destroy Done 2023-10-21T04:47:01.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:47:01.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index destroyed 2023-10-21T04:47:01.925+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:47:01.925+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 7 Destroy Done 2023-10-21T04:47:02.314+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:47:02.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 18374611443167983589 2023-10-21T04:47:02.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 6455624821495495854 2023-10-21T04:47:02.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 6455624821495495854 2023-10-21T04:47:02.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 18374611443167983589 2023-10-21T04:47:02.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [18374611443167983589 6455624821495495854], elapsed: 874.715234ms 2023-10-21T04:47:02.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:02.693+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:47:02.693+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper stopped 2023-10-21T04:47:02.693+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': stopped 2023-10-21T04:47:02.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:47:02.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:47:02.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:47:02.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:47:02.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:47:02.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:47:02.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:47:02.695+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:47:02.695+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:47:02.695+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:47:02.695+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:47:02.695+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:47:02.695+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:47:02.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:47:02.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:47:02.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:47:02.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:47:02.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:47:02.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shutdown completed 2023-10-21T04:47:02.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:02.696+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:47:02.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:47:02.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:47:02.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance destroyed 2023-10-21T04:47:02.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:47:02.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:02.698+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:47:02.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:47:02.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:47:02.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:47:02.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:47:02.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:47:02.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:47:02.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:47:02.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:47:02.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:47:02.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:47:02.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:47:02.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:47:02.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:47:02.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:47:02.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:47:02.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:47:02.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:47:02.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:47:02.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shutdown completed 2023-10-21T04:47:02.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:02.702+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:47:02.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:47:02.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:47:02.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance destroyed 2023-10-21T04:47:02.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:47:02.704+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Cleaned up Tenant with name:bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:47:02.704+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [18374611443167983589 6455624821495495854], elapsed(sec): 0.885380594 2023-10-21T04:47:02.706+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenbd:30:d7:e2:d2:4c:63:d7 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: ab759b18c08b0e81e45c6631c4e2d845 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken76:ee:1e:24:89:be:a6:21 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8859733702134432451 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6438969491930702266 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13138923834221067684 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:47:02.706+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: ab759b18c08b0e81e45c6631c4e2d845 TransferTokenId: ShardTokenbd:30:d7:e2:d2:4c:63:d7 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:47:02.706+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000125581 2023-10-21T04:47:02.706+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenbd:30:d7:e2:d2:4c:63:d7, dropIndexes: true 2023-10-21T04:47:02.707+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7, shardId :0 2023-10-21T04:47:02.707+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/ab759b18c08b0e81e45c6631c4e2d845_ShardTokenbd_30_d7_e2_d2_4c_63_d7, shardId :0 2023-10-21T04:47:02.707+05:30 [Info] LifecycleMgr.DeleteIndex() : index 16632040625897953619 does not exist. 2023-10-21T04:47:02.707+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 83.166µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:02.708+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 7790129686881525591 instance id 6438969491930702266 real instance id 16759912838651900043 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:47:02.708+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 159.854µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:02.708+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16823504650100619022 instance id 13138923834221067684 real instance id 4186965058349029465 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:47:02.708+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 240.739µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:02.715+05:30 [Info] LifecycleMgr.DeleteIndex() : index 668551233566132751 does not exist. 2023-10-21T04:47:02.715+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 121.737µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:02.715+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17327901024118572215 5629943257627822009] 2023-10-21T04:47:02.788+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:47:03.320+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:47:03.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17327901024118572215 2023-10-21T04:47:03.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 5629943257627822009 2023-10-21T04:47:03.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 5629943257627822009 2023-10-21T04:47:03.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17327901024118572215 2023-10-21T04:47:03.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17327901024118572215 5629943257627822009], elapsed: 977.80116ms 2023-10-21T04:47:03.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:03.693+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:47:03.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:47:03.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shutdown completed 2023-10-21T04:47:03.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:03.693+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:47:03.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:47:03.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance destroyed 2023-10-21T04:47:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:47:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:03.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:47:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:47:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shutdown completed 2023-10-21T04:47:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:47:03.695+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:47:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:47:03.695+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance destroyed 2023-10-21T04:47:03.696+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:47:03.696+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17327901024118572215 5629943257627822009], elapsed(sec): 0.980881375 2023-10-21T04:47:03.718+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:47:03.718+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:47:03.719+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.815µs 2023-10-21T04:47:03.719+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:47:03.719+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:47:03.719+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:47:03.722+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:47:03.722+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:47:03.722+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:47:03.722+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:47:03.723+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 32.447µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:03.729+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:47:03.729+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:47:03.729+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:47:03.730+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:47:03.730+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:47:03.733+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:47:03.733+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:47:03.733+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 34.824µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:03.737+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:47:03.738+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:47:03.738+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 6.275543608s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:47:03.739+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/ab759b18c08b0e81e45c6631c4e2d845, rev: [] call. err: 2023-10-21T04:47:03.738+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 16] call. taskList: &{Rev:[0 0 0 0 0 0 0 17] Tasks:[]} 2023-10-21T04:47:03.738+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 17] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:47:03.739+05:30 [Warn] util::traceLockLog: Long wait for svcMgrMu.Lock() 950.384033ms in RebalanceServiceManager::rebalanceJanitor 2023-10-21T04:47:03.741+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:47:03.741+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 17] Tasks:[]} 2023-10-21T04:47:03.746+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:47:03.746+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 17] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:47:03.751+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 17] 2023-10-21T04:47:03.752+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 17] 2023-10-21T04:47:04.316+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:47:04.323+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:04.323+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:04.325+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:47:04.326+05:30 [Info] switched currmeta from 251 -> 251 force true 2023-10-21T04:47:11.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9994060258694292, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:47:12.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7818578462560372, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:47:18.037+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:18.037+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:18.057+05:30 [Info] switched currmeta from 251 -> 255 force true 2023-10-21T04:47:18.081+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:18.081+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:18.091+05:30 [Info] switched currmeta from 255 -> 255 force true 2023-10-21T04:47:18.285+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:18.286+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:18.300+05:30 [Info] switched currmeta from 255 -> 259 force true 2023-10-21T04:47:18.314+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:18.314+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:18.319+05:30 [Info] switched currmeta from 259 -> 259 force true 2023-10-21T04:47:19.406+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:47:19.411+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:19.411+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:19.413+05:30 [Info] switched currmeta from 259 -> 259 force true 2023-10-21T04:47:31.731+05:30 [Info] memstats {"Alloc":108777296, "TotalAlloc":501283248, "Sys":227236152, "Lookups":0, "Mallocs":3817785,"Frees":3664390, "HeapAlloc":108777296, "HeapSys":207323136, "HeapIdle":85417984, "HeapInuse":121905152,"HeapReleased":3145728, "HeapObjects":153395,"MSpanInuse":595360, "MSpanSys": 1354560, "StackInuse": 2392064,"GCSys":13492984, "LastGC":1697843784100519606,"PauseTotalNs":3817745, "PauseNs":[373120], "NumGC":10} 2023-10-21T04:47:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:47:32.787+05:30 [Info] client stats current counts: current: 12, not current: 0 2023-10-21T04:47:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:47:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:47:33.142+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:47:33.161+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:47:33.433+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:47:33.455+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:47:33.746+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:47:33.752+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:47:33.752+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 17] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:47:33.752+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 17] call. err: operation_canceled 2023-10-21T04:47:33.752+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:47:33.752+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 17] Tasks:[]} 2023-10-21T04:47:33.754+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 17] 2023-10-21T04:47:33.754+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 17] 2023-10-21T04:47:34.311+05:30 [Info] janitor: running cleanup. 2023-10-21T04:47:34.311+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[12027054384338714804 16483001637761719171 11365633301282701856 3674078063353481907 10761885902580101111 4860549750398470299 8669649359784409226 13268712595133993155 17797267771846799851 2048006213723315482 17656144409930763141 9665410041638025825 8504686614261377811 17917116660857103172 18046371353728341033 8378417662949994532 15431181238515168303 16706481854123898870 128675491261474353 1301336828089581719 10606939961365971246 6985049432225870291 4790214984019693894 16162012140118277065 12824340686309573428 4286621850367430051 14549775068266281786 1252359852432998131 15968368593314043432 12599498416578472779 12195715103909237684 12140458060295645238 3381524789655566900 4138047918192025493 573521267164877540 12338036531800130039 249887442666643345 14292583894719865678 4535197604682045916 15973521466438086342 4987105685507282285 12833670934079048728 5465835793930010953 18323460706027642575 11276338709617092948 12582561312368048736 4532813151040661705 17476654447546601131 8857269605176042330 7147407900589480727 5932387718257253322 7045704526128519604 8910459001587546151 16632040625897953619 16961575949607441659 13856704970496601958 7392796230534034390 14889401051926000176 14741949230702299196 15945117906390072636 11619165464560280276 16337157693634799600 18026660730857541767 7525085113233084253]} 2023-10-21T04:47:35.389+05:30 [Info] Indexer::ReadMemstats Time Taken 151.85µs 2023-10-21T04:47:36.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9982418727229827, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:47:37.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9992895166459677, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:47:38.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8007371801764271, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:47:38.686+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 4841852412665776737 2023-10-21T04:47:38.686+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 140.112µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:38.761+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43668 2023-10-21T04:47:38.769+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43668 2023-10-21T04:47:38.821+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43668 2023-10-21T04:47:38.887+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 4841852412665776737 2023-10-21T04:47:38.887+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 284.63µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:38.983+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:38.983+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:38.989+05:30 [Info] switched currmeta from 259 -> 262 force true 2023-10-21T04:47:38.997+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:38.998+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:39.028+05:30 [Info] switched currmeta from 262 -> 265 force true 2023-10-21T04:47:39.129+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:39.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:39.151+05:30 [Info] switched currmeta from 265 -> 268 force true 2023-10-21T04:47:39.168+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:39.168+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:39.190+05:30 [Info] switched currmeta from 268 -> 268 force true 2023-10-21T04:47:39.270+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58836 2023-10-21T04:47:39.288+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 14.8371ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:39.421+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 4841852412665776737. No update is performed. 2023-10-21T04:47:39.421+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 158.766µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:39.436+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58836. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:39.436+05:30 [Info] messageListener.start(): message channel closed. Remove peer 22:38:cd:eb:ac:b6:b8:cd as follower. 2023-10-21T04:47:39.567+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58856 2023-10-21T04:47:39.596+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:47:39.615+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:39.615+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:39.618+05:30 [Info] switched currmeta from 268 -> 268 force true 2023-10-21T04:47:39.631+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 57.153264ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:39.656+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:47:39.656+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:47:39.664+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:47:39.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7660935342846871, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:47:39.686+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:47:39.687+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:47:39.687+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:47:39.704+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:47:39.724+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:47:39.724+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:47:39.724+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:47:39.733+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58856. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:39.733+05:30 [Info] messageListener.start(): message channel closed. Remove peer 15:df:ed:9f:4b:2c:db:eb as follower. 2023-10-21T04:47:39.746+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58882 2023-10-21T04:47:39.747+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:47:39.769+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 12.18704ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:39.769+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:47:39.769+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:47:39.769+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:47:39.781+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:47:39.789+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:39.804+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:47:39.804+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:47:39.804+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:47:39.804+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:47:39.805+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:47:39.805+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:47:39.805+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:47:39.805+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:47:39.805+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:47:39.805+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:47:39.805+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:50122->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:39.805+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:50588->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:39.805+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:58882->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:39.805+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:55258->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:39.806+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58882. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:39.806+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4b:e2:c2:a2:2d:52:e6:58 as follower. 2023-10-21T04:47:39.864+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:58890 2023-10-21T04:47:39.897+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 24.188764ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:39.903+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:58890. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:39.903+05:30 [Info] messageListener.start(): message channel closed. Remove peer 47:8d:c0:1b:83:e7:c4:36 as follower. 2023-10-21T04:47:40.132+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:40.272+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:40.321+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:40.321+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:40.323+05:30 [Info] switched currmeta from 268 -> 269 force true 2023-10-21T04:47:40.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:40.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:40.361+05:30 [Info] switched currmeta from 269 -> 270 force true 2023-10-21T04:47:40.398+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:40.800+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:40.800+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:40.802+05:30 [Info] switched currmeta from 270 -> 271 force true 2023-10-21T04:47:40.968+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:40.968+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:40.971+05:30 [Info] switched currmeta from 271 -> 272 force true 2023-10-21T04:47:41.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16515427720591431191:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:47:41.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16515427720591431191:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:47:41.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16515427720591431191:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:47:41.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16515427720591431191:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:47:41.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17044257022107709142:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:47:41.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17044257022107709142:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:47:44.507+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:47:44.512+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:44.512+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:44.514+05:30 [Info] switched currmeta from 272 -> 272 force true 2023-10-21T04:47:50.398+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 2608585389864151823 2023-10-21T04:47:50.398+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 135.881µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:50.462+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43668 2023-10-21T04:47:50.479+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43668 2023-10-21T04:47:50.526+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:43668 2023-10-21T04:47:50.582+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 2608585389864151823 2023-10-21T04:47:50.583+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 1.576144ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:50.626+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:50.626+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:50.628+05:30 [Info] switched currmeta from 272 -> 274 force true 2023-10-21T04:47:50.651+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:50.651+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:50.667+05:30 [Info] switched currmeta from 274 -> 276 force true 2023-10-21T04:47:50.680+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:50.680+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:50.692+05:30 [Info] switched currmeta from 276 -> 278 force true 2023-10-21T04:47:50.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:50.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:50.872+05:30 [Info] switched currmeta from 278 -> 281 force true 2023-10-21T04:47:50.903+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:50.903+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:50.915+05:30 [Info] switched currmeta from 281 -> 281 force true 2023-10-21T04:47:51.400+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:59194 2023-10-21T04:47:51.435+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 30.773418ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:51.528+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:59208 2023-10-21T04:47:51.553+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 2608585389864151823. No update is performed. 2023-10-21T04:47:51.553+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 131.773µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:51.556+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.601073ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:51.566+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:59194. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:51.567+05:30 [Info] messageListener.start(): message channel closed. Remove peer 64:b8:5e:bb:40:70:f3:2a as follower. 2023-10-21T04:47:51.635+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:59208. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:51.635+05:30 [Info] messageListener.start(): message channel closed. Remove peer bb:1c:d6:c9:b0:86:f8:9f as follower. 2023-10-21T04:47:51.668+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:47:51.668+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:47:51.680+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:47:51.713+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:47:51.713+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:47:51.713+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:47:51.714+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:59216 2023-10-21T04:47:51.724+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:47:51.742+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 22.34052ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:51.795+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:47:51.795+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:47:51.795+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:47:51.804+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:47:51.821+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:47:51.821+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:47:51.821+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:47:51.833+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:59230 2023-10-21T04:47:51.834+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:47:51.847+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:59216. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:51.847+05:30 [Info] messageListener.start(): message channel closed. Remove peer 40:c1:ee:61:92:da:82:11 as follower. 2023-10-21T04:47:51.858+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 22.484176ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:47:51.859+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:47:51.859+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:47:51.859+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:47:51.859+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:47:51.859+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:47:51.859+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:47:51.859+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:47:51.859+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:47:51.859+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:47:51.859+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:47:51.859+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:50936->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:51.859+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:59230->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:51.859+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:55588->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:51.860+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:50470->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:47:51.860+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:59230. Reason = EOF. Cleaning Pipe. 2023-10-21T04:47:51.860+05:30 [Info] messageListener.start(): message channel closed. Remove peer 74:a4:2f:e4:33:11:88:36 as follower. 2023-10-21T04:47:51.936+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:52.062+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:52.111+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:52.111+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:52.114+05:30 [Info] switched currmeta from 281 -> 282 force true 2023-10-21T04:47:52.171+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:52.171+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:52.177+05:30 [Info] switched currmeta from 282 -> 283 force true 2023-10-21T04:47:52.243+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:52.359+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:47:52.449+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:52.449+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:52.457+05:30 [Info] switched currmeta from 283 -> 284 force true 2023-10-21T04:47:52.505+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:52.505+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:52.511+05:30 [Info] switched currmeta from 284 -> 285 force true 2023-10-21T04:47:52.971+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15281531523624343311:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:47:52.971+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15281531523624343311:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:47:52.971+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15281531523624343311:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:47:52.971+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15281531523624343311:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:47:52.971+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[15281531523624343311:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:47:53.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14845378835615010356:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:47:54.527+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:47:54.532+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:47:54.533+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:47:54.534+05:30 [Info] switched currmeta from 285 -> 285 force true 2023-10-21T04:48:03.070+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 244] aggregate storage info: map[bucket_%2:map[kv:2447552 index:9359265] bucket_1:map[kv:2517184 index:10125466]] 2023-10-21T04:48:03.749+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:48:03.755+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:48:03.755+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 17] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:48:03.755+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 17] call. err: operation_canceled 2023-10-21T04:48:03.755+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:48:03.755+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 17] Tasks:[]} 2023-10-21T04:48:03.756+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 17] 2023-10-21T04:48:03.757+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 17] 2023-10-21T04:48:04.998+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:48:05.272+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:05.272+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:05.279+05:30 [Info] switched currmeta from 285 -> 285 force true 2023-10-21T04:48:05.381+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:48:05.381+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:48:05.482+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:48:05.482+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:48:05.699+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:05.699+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:05.702+05:30 [Info] switched currmeta from 285 -> 285 force true 2023-10-21T04:48:05.713+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:05.713+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:05.716+05:30 [Info] switched currmeta from 285 -> 285 force true 2023-10-21T04:48:06.876+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {85664c0432a8df82b3d3efd913473cb1 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:48:06.882+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 5.203µs 2023-10-21T04:48:06.882+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:48:06.882+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:48:06.882+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:48:06.885+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:48:06.885+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:06.885+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.63347ms 2023-10-21T04:48:06.885+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:48:06.885+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:48:06.885+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 18] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:48:06.885+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.711µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:06.885+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 17] call. taskList: &{Rev:[0 0 0 0 0 0 0 18] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/85664c0432a8df82b3d3efd913473cb1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:85664c0432a8df82b3d3efd913473cb1]}]} 2023-10-21T04:48:06.894+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 18] 2023-10-21T04:48:06.896+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 18] 2023-10-21T04:48:07.967+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {ba6304988a6335d65281381d6cfc5e32 85664c0432a8df82b3d3efd913473cb1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:48:07.970+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {ba6304988a6335d65281381d6cfc5e32 85664c0432a8df82b3d3efd913473cb1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:48:07.970+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"85664c0432a8df82b3d3efd913473cb1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:48:07.971+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{ba6304988a6335d65281381d6cfc5e32 85664c0432a8df82b3d3efd913473cb1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:48:07.972+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{ba6304988a6335d65281381d6cfc5e32 85664c0432a8df82b3d3efd913473cb1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:48:07.972+05:30 [Info] ShardRebalancer::observeRebalance &{ba6304988a6335d65281381d6cfc5e32 85664c0432a8df82b3d3efd913473cb1 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:48:07.981+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"85664c0432a8df82b3d3efd913473cb1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:48:07.981+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:48:08.072+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:48:08.079+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:48:08.120+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:48:08.273+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:68:e7:32:31:80:46:d9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16515427720591431191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2034303749683961252 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 52515533635989429 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.322+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:41:36:5d:f0:c3:a0:ee MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17044257022107709142 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6868744514623643769 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16667180614285973080 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.327+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:ea:70:79:ce:7f:27:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc0:4e:bd:51:d0:75:ef:9b Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1637471422672568438 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9725618697950995746 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14845378835615010356 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.327+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.369+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:68:e7:32:31:80:46:d9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16515427720591431191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2034303749683961252 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 52515533635989429 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.427+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:41:36:5d:f0:c3:a0:ee MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17044257022107709142 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6868744514623643769 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16667180614285973080 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.427+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:48:08.427+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:48:08.427+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:48:08.427+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.92µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:08.478+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.479+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:48:08.479+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:48:08.479+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:48:08.479+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 31.986µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:08.516+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:41:36:5d:f0:c3:a0:ee MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17044257022107709142 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6868744514623643769 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16667180614285973080 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.516+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.598+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:ea:70:79:ce:7f:27:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc0:4e:bd:51:d0:75:ef:9b Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1637471422672568438 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9725618697950995746 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14845378835615010356 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.626+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:ea:70:79:ce:7f:27:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc0:4e:bd:51:d0:75:ef:9b Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1637471422672568438 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9725618697950995746 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14845378835615010356 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.637+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:68:e7:32:31:80:46:d9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16515427720591431191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2034303749683961252 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 52515533635989429 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.696+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"85664c0432a8df82b3d3efd913473cb1","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:48:08.696+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:48:08.696+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:48:08.696+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:48:08.696+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:48:08.696+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:48:08.696+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 15.287µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:08.728+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:ea:70:79:ce:7f:27:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc0:4e:bd:51:d0:75:ef:9b Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1637471422672568438 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9725618697950995746 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14845378835615010356 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.747+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:08.886+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{ba6304988a6335d65281381d6cfc5e32 85664c0432a8df82b3d3efd913473cb1 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:48:08.953+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:ea:70:79:ce:7f:27:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc0:4e:bd:51:d0:75:ef:9b Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1637471422672568438 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9725618697950995746 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14845378835615010356 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.059+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokena7:ea:70:79:ce:7f:27:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc0:4e:bd:51:d0:75:ef:9b Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1637471422672568438 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9725618697950995746 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14845378835615010356 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.082+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.083+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:48:09.083+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:48:09.083+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:48:09.083+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.411µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:09.083+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[6455624821495495854:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/6455624821495495854/shards/shard6455624821495495854 18374611443167983589:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/18374611443167983589/shards/shard18374611443167983589] 85664c0432a8df82b3d3efd913473cb1 ShardTokenc0:4e:bd:51:d0:75:ef:9b /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc008297bc0 0xc008297c20 map[] 0xc00a15f1a0 0xc007d1b980 false} 2023-10-21T04:48:09.083+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/6455624821495495854/shards/shard6455624821495495854 2023-10-21T04:48:09.084+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/18374611443167983589/shards/shard18374611443167983589 2023-10-21T04:48:09.087+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/6455624821495495854/shards/shard6455624821495495854/data: super block snapshot head offset :0 tail offset:1155072 sbgen :179 2023-10-21T04:48:09.088+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/6455624821495495854/shards/shard6455624821495495854/data/recovery: super block snapshot head offset :0 tail offset:970752 sbgen :179 2023-10-21T04:48:09.091+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/18374611443167983589/shards/shard18374611443167983589/data: super block snapshot head offset :0 tail offset:1372160 sbgen :196 2023-10-21T04:48:09.092+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/18374611443167983589/shards/shard18374611443167983589/data/recovery: super block snapshot head offset :0 tail offset:1232896 sbgen :196 2023-10-21T04:48:09.098+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/6455624821495495854/shards/shard6455624821495495854: metadata saved successfully 2023-10-21T04:48:09.100+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) added 2023-10-21T04:48:09.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:48:09.100+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 126173235 totalBytes: 2144945 numShards: 2 duration: 17.081148ms 2023-10-21T04:48:09.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/6455624821495495854/shards/shard6455624821495495854, newshardId 6455624821495495854, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, shardsize 2144945, duration 17.400551ms 2023-10-21T04:48:09.101+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 6455624821495495854, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, err: , elapsed(sec): 0.01795563 2023-10-21T04:48:09.100+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/18374611443167983589/shards/shard18374611443167983589: metadata saved successfully 2023-10-21T04:48:09.100+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 6455624821495495854 bytesWritten: 2144945, totalBytes: 2144945, transferRate: 1.340590625e+08 2023-10-21T04:48:09.102+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) added 2023-10-21T04:48:09.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:48:09.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenc0_4e_bd_51_d0_75_ef_9b/18374611443167983589/shards/shard18374611443167983589, newshardId 18374611443167983589, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, shardsize 2624340, duration 18.408431ms 2023-10-21T04:48:09.103+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 18374611443167983589, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, err: , elapsed(sec): 0.019597354 2023-10-21T04:48:09.103+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 85664c0432a8df82b3d3efd913473cb1, transferId: ShardTokenc0:4e:bd:51:d0:75:ef:9b 2023-10-21T04:48:09.103+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 85664c0432a8df82b3d3efd913473cb1, transferId: ShardTokenc0:4e:bd:51:d0:75:ef:9b, elapsed(sec): 0.000189859 2023-10-21T04:48:09.103+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[6455624821495495854: 18374611443167983589:], shardPaths: map[6455624821495495854:/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 18374611443167983589:/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.020035448 2023-10-21T04:48:09.103+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 18374611443167983589 bytesWritten: 2624340, totalBytes: 2624340, transferRate: 1.4579666666666666e+08 2023-10-21T04:48:09.103+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [18374611443167983589 6455624821495495854], ttid: ShardTokenc0:4e:bd:51:d0:75:ef:9b, elapsed(sec): 0.020706314 2023-10-21T04:48:09.125+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:09.125+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:09.129+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:09.131+05:30 [Info] switched currmeta from 285 -> 286 force true 2023-10-21T04:48:09.148+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.148+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardTokenc0:4e:bd:51:d0:75:ef:9b 2023-10-21T04:48:09.148+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[6455624821495495854:true 18374611443167983589:true] 2023-10-21T04:48:09.148+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.605µs 2023-10-21T04:48:09.148+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [18374611443167983589 6455624821495495854], isLockedForRecovery: true 2023-10-21T04:48:09.148+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [18374611443167983589 6455624821495495854], errMap: map[6455624821495495854: 18374611443167983589:], elapsed: 26.983µs 2023-10-21T04:48:09.148+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:48:09.148+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:48:09.181+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12396348969790516872,realInstId: 0, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:09.182+05:30 [Info] Indexer::handleRecoverIndex InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:48:09.182+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 666.397µs 2023-10-21T04:48:09.183+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 904.887µs 2023-10-21T04:48:09.183+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:09.183+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12396348969790516872, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:48:09.184+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 35.220241ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:09.184+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 4 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:09.184+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': added 2023-10-21T04:48:09.188+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCleaner initialized 2023-10-21T04:48:09.188+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:09.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.191+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:09.192+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCleaner initialized for recovery 2023-10-21T04:48:09.192+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:09.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery 2023-10-21T04:48:09.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true] 2023-10-21T04:48:09.193+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1232896] Data log Head [0] Tail [1372160] 2023-10-21T04:48:09.193+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1232896] 2023-10-21T04:48:09.193+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCleaner initialized 2023-10-21T04:48:09.194+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:09.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.196+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.196+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:48:09.196+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:48:09.196+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.196+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:48:09.196+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.197+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:48:09.197+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.197+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:48:09.197+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.197+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:48:09.197+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.197+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:48:09.197+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.197+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:48:09.197+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.198+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:48:09.198+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.198+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:48:09.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCleaner initialized for recovery 2023-10-21T04:48:09.198+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:09.198+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.198+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.199+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.199+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.199+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.199+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery 2023-10-21T04:48:09.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true] 2023-10-21T04:48:09.201+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[970752] Data log Head [0] Tail [1155072] 2023-10-21T04:48:09.201+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [970752] 2023-10-21T04:48:09.204+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.237+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.237+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:48:09.237+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.237+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:48:09.237+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.237+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:48:09.237+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.238+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:48:09.238+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.238+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:48:09.238+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.238+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:48:09.238+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.239+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.239+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.240+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.240+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.240+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:09.240+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.240+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:09.240+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.240+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.243+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:48:09.243+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.243+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:48:09.244+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.244+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.244+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.244+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.244+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.245+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.245+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.245+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.245+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.245+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.245+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.246+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:09.248+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.248+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:09.250+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.252+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.269+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.269+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex 2023-10-21T04:48:09.269+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : replayOffset: 1368064 (RP:1347584) (CP:1368064) 2023-10-21T04:48:09.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1368064] took [93.352745ms] 2023-10-21T04:48:09.288+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1055476, resident 1, memSz 932, memSzIndex 0, numRecords 12 2023-10-21T04:48:09.288+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44648240 controller mem 1970 2023-10-21T04:48:09.292+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.293+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44648196 controller mem 1970 2023-10-21T04:48:09.294+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44648152 controller mem 1970 2023-10-21T04:48:09.296+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44648152 controller mem 1970 2023-10-21T04:48:09.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1058326, resident 1, memSz 65710, memSzIndex 385, numRecords 1000 2023-10-21T04:48:09.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44648152 controller mem 1970 2023-10-21T04:48:09.298+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.298+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44648152 controller mem 1970 2023-10-21T04:48:09.296+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.300+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.301+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44648108 controller mem 1970 2023-10-21T04:48:09.300+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex 2023-10-21T04:48:09.301+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.302+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45696640 controller mem 1970 2023-10-21T04:48:09.303+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.304+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45695481 controller mem 3658 2023-10-21T04:48:09.304+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.309+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45694790 controller mem 3900 2023-10-21T04:48:09.309+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.310+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702344 controller mem 3836 2023-10-21T04:48:09.310+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.309+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : replayOffset: 1150976 (RP:1118208) (CP:1150976) 2023-10-21T04:48:09.311+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1150976] took [109.821145ms] 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702300 controller mem 3836 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702300 controller mem 3836 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702300 controller mem 3836 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702300 controller mem 3836 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702256 controller mem 3836 2023-10-21T04:48:09.311+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.312+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702212 controller mem 3836 2023-10-21T04:48:09.312+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.312+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702168 controller mem 3836 2023-10-21T04:48:09.312+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.312+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702168 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702168 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702124 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702124 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702080 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45702036 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: request quota after header replay 1056164, resident 1, memSz 68710, memSzIndex 797, numRecords 1000 2023-10-21T04:48:09.313+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.313+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1155072] replayOffset [1150976] 2023-10-21T04:48:09.316+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:09.316+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:09.317+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.318+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [120.243612ms] 2023-10-21T04:48:09.322+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.322+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.323+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.324+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.325+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.325+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.325+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.326+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.326+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.326+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.326+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.327+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.327+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.327+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.327+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true]. Built [17] plasmas, took [126.076598ms] 2023-10-21T04:48:09.327+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.327+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 48188 recovery UsedSpace 8236 2023-10-21T04:48:09.328+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.329+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.332+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.332+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.333+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45701992 controller mem 3836 2023-10-21T04:48:09.333+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.333+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44653408 controller mem 3868 2023-10-21T04:48:09.334+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.335+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44653364 controller mem 3868 2023-10-21T04:48:09.335+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:09.335+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44653364 controller mem 3868 2023-10-21T04:48:09.335+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1372160] replayOffset [1368064] 2023-10-21T04:48:09.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [143.334366ms] 2023-10-21T04:48:09.338+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.338+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.343+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.345+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.345+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.344+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.341+05:30 [Info] switched currmeta from 286 -> 287 force true 2023-10-21T04:48:09.347+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.348+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.348+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.349+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.349+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.349+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.349+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.350+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.350+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.350+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.350+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.350+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:09.351+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true]. Built [18] plasmas, took [157.70448ms] 2023-10-21T04:48:09.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.351+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:48:09.348+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.352+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper started 2023-10-21T04:48:09.352+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex started 2023-10-21T04:48:09.353+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.353+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.369+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:41:36:5d:f0:c3:a0:ee MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17044257022107709142 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6868744514623643769 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16667180614285973080 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.370+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:68:e7:32:31:80:46:d9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16515427720591431191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2034303749683961252 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 52515533635989429 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.379+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex started 2023-10-21T04:48:09.379+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:48:09.380+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.406+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:09.380+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.406+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.406+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:09.407+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.407+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:09.407+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:09.407+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.407+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex stopped 2023-10-21T04:48:09.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.407+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.425+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:09.425+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex closed 2023-10-21T04:48:09.426+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:09.426+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.426+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.426+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.426+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:09.427+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.427+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:09.453+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:09.453+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex stopped 2023-10-21T04:48:09.453+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.453+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.453+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.453+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.474+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:09.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex closed 2023-10-21T04:48:09.474+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:09.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:48:09.474+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:48:09.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex sucessfully destroyed 2023-10-21T04:48:09.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:48:09.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:48:09.487+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index destroyed 2023-10-21T04:48:09.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.495+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.495+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.500+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.500+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.500+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.507+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.507+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:09.507+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex started 2023-10-21T04:48:09.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.507+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.515+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.515+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:09.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex started 2023-10-21T04:48:09.515+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Warmup took 331.082683ms 2023-10-21T04:48:09.515+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.515+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.515+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 4 WriterThreads 1 2023-10-21T04:48:09.515+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 4 Stopped Billing WUs 2023-10-21T04:48:09.515+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index Index: 12396348969790516872 Partition: 4 2023-10-21T04:48:09.515+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:09.516+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 5 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:09.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.516+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:48:09.516+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:48:09.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.542+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex started 2023-10-21T04:48:09.542+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.542+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.550+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex started 2023-10-21T04:48:09.550+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:48:09.550+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:09.550+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:09.550+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex stopped 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.550+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.551+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.573+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:09.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex closed 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:09.574+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:09.574+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.574+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex stopped 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.574+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:09.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex closed 2023-10-21T04:48:09.580+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:09.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:48:09.580+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:48:09.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex sucessfully destroyed 2023-10-21T04:48:09.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:48:09.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:48:09.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index destroyed 2023-10-21T04:48:09.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.593+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.593+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.600+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.600+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:09.600+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex started 2023-10-21T04:48:09.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.600+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.599+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.601+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.601+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.614+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.615+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:09.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex started 2023-10-21T04:48:09.615+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Warmup took 99.08784ms 2023-10-21T04:48:09.617+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 5 WriterThreads 1 2023-10-21T04:48:09.615+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.617+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 5 Stopped Billing WUs 2023-10-21T04:48:09.617+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index Index: 12396348969790516872 Partition: 5 2023-10-21T04:48:09.617+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:09.617+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 6 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:09.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.618+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.618+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:48:09.615+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.618+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.618+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:48:09.621+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:41:36:5d:f0:c3:a0:ee MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17044257022107709142 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6868744514623643769 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16667180614285973080 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.621+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:48:09.621+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:48:09.622+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:48:09.622+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.079µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:09.622+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[5629943257627822009:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/5629943257627822009/shards/shard5629943257627822009 17327901024118572215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/17327901024118572215/shards/shard17327901024118572215] 85664c0432a8df82b3d3efd913473cb1 ShardToken3a:41:36:5d:f0:c3:a0:ee /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc008297bc0 0xc008297c20 map[] 0xc00992de00 0xc0099030e0 false} 2023-10-21T04:48:09.622+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/5629943257627822009/shards/shard5629943257627822009 2023-10-21T04:48:09.623+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/17327901024118572215/shards/shard17327901024118572215 2023-10-21T04:48:09.626+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/5629943257627822009/shards/shard5629943257627822009/data: super block snapshot head offset :0 tail offset:1171456 sbgen :181 2023-10-21T04:48:09.627+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/5629943257627822009/shards/shard5629943257627822009/data/recovery: super block snapshot head offset :0 tail offset:995328 sbgen :181 2023-10-21T04:48:09.627+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/17327901024118572215/shards/shard17327901024118572215/data: super block snapshot head offset :0 tail offset:1368064 sbgen :198 2023-10-21T04:48:09.628+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/17327901024118572215/shards/shard17327901024118572215/data/recovery: super block snapshot head offset :0 tail offset:1232896 sbgen :198 2023-10-21T04:48:09.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.633+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex started 2023-10-21T04:48:09.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.633+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.634+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/17327901024118572215/shards/shard17327901024118572215: metadata saved successfully 2023-10-21T04:48:09.634+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/5629943257627822009/shards/shard5629943257627822009: metadata saved successfully 2023-10-21T04:48:09.635+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) added 2023-10-21T04:48:09.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:48:09.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/17327901024118572215/shards/shard17327901024118572215, newshardId 17327901024118572215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, shardsize 2620203, duration 11.996739ms 2023-10-21T04:48:09.635+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17327901024118572215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, err: , elapsed(sec): 0.01306785 2023-10-21T04:48:09.635+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17327901024118572215 bytesWritten: 2620203, totalBytes: 2620203, transferRate: 2.3820027272727272e+08 2023-10-21T04:48:09.637+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) added 2023-10-21T04:48:09.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:48:09.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee/5629943257627822009/shards/shard5629943257627822009, newshardId 5629943257627822009, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, shardsize 2185870, duration 15.001579ms 2023-10-21T04:48:09.637+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 5629943257627822009, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, err: , elapsed(sec): 0.015278345 2023-10-21T04:48:09.637+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 85664c0432a8df82b3d3efd913473cb1, transferId: ShardToken3a:41:36:5d:f0:c3:a0:ee 2023-10-21T04:48:09.637+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 85664c0432a8df82b3d3efd913473cb1, transferId: ShardToken3a:41:36:5d:f0:c3:a0:ee, elapsed(sec): 9.2082e-05 2023-10-21T04:48:09.637+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[5629943257627822009: 17327901024118572215:], shardPaths: map[5629943257627822009:/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 17327901024118572215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.015440865 2023-10-21T04:48:09.637+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 5629943257627822009 bytesWritten: 2185870, totalBytes: 2185870, transferRate: 1.5613357142857143e+08 2023-10-21T04:48:09.637+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17327901024118572215 5629943257627822009], ttid: ShardToken3a:41:36:5d:f0:c3:a0:ee, elapsed(sec): 0.015897594 2023-10-21T04:48:09.640+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:09.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.642+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex started 2023-10-21T04:48:09.642+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:48:09.642+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:09.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.642+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:09.643+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:09.643+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex stopped 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.643+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.651+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken3a:41:36:5d:f0:c3:a0:ee MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17044257022107709142 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6868744514623643769 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16667180614285973080 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.671+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:09.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex closed 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:09.672+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:09.672+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex stopped 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.672+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.705+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:09.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex closed 2023-10-21T04:48:09.706+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:09.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:48:09.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:48:09.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex sucessfully destroyed 2023-10-21T04:48:09.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:48:09.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:48:09.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index destroyed 2023-10-21T04:48:09.734+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.735+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.735+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.736+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.744+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.749+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.753+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:09.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.754+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.754+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:09.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex started 2023-10-21T04:48:09.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.755+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex started 2023-10-21T04:48:09.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.757+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Warmup took 140.214127ms 2023-10-21T04:48:09.758+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 6 WriterThreads 1 2023-10-21T04:48:09.758+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 6 Stopped Billing WUs 2023-10-21T04:48:09.758+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index Index: 12396348969790516872 Partition: 6 2023-10-21T04:48:09.758+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:09.758+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 7 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:09.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.758+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.758+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.758+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.758+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:48:09.759+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 54613 recovery UsedSpace 8236 2023-10-21T04:48:09.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.765+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex started 2023-10-21T04:48:09.765+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.765+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex started 2023-10-21T04:48:09.775+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:48:09.775+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:09.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:09.775+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex stopped 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.775+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:09.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex closed 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:09.794+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:09.794+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.794+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex stopped 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.794+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:09.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex closed 2023-10-21T04:48:09.816+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:09.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:48:09.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:48:09.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex sucessfully destroyed 2023-10-21T04:48:09.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:48:09.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:48:09.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.829+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index destroyed 2023-10-21T04:48:09.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.835+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.835+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.841+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.841+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.850+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.851+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.851+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:09.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex started 2023-10-21T04:48:09.851+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.851+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.855+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.855+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:09.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex started 2023-10-21T04:48:09.855+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Warmup took 96.64935ms 2023-10-21T04:48:09.855+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.855+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.855+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 7 WriterThreads 1 2023-10-21T04:48:09.855+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 7 Stopped Billing WUs 2023-10-21T04:48:09.855+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index Index: 12396348969790516872 Partition: 7 2023-10-21T04:48:09.855+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:09.855+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 8 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:09.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.856+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.856+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:48:09.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.856+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.856+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:48:09.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.860+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex started 2023-10-21T04:48:09.860+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.860+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.862+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex started 2023-10-21T04:48:09.862+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:48:09.862+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:09.862+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.862+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.862+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.862+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:09.862+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.862+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:09.862+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:09.862+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex stopped 2023-10-21T04:48:09.863+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.863+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.863+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.863+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:09.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex closed 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:09.867+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:09.867+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex stopped 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.873+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:09.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex closed 2023-10-21T04:48:09.873+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:09.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:48:09.873+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:48:09.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex sucessfully destroyed 2023-10-21T04:48:09.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:48:09.875+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:48:09.877+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.878+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index destroyed 2023-10-21T04:48:09.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.881+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.884+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.884+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.888+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.888+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:09.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex started 2023-10-21T04:48:09.888+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.888+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.894+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.895+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.895+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:09.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex started 2023-10-21T04:48:09.895+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Warmup took 39.290405ms 2023-10-21T04:48:09.895+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 8 WriterThreads 1 2023-10-21T04:48:09.895+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 8 Stopped Billing WUs 2023-10-21T04:48:09.895+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index Index: 12396348969790516872 Partition: 8 2023-10-21T04:48:09.895+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:09.895+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 1 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:09.895+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.896+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.896+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.896+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:48:09.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.896+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.896+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:48:09.896+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.900+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:09.900+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:09.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.905+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.905+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex started 2023-10-21T04:48:09.905+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.905+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.908+05:30 [Info] switched currmeta from 287 -> 288 force true 2023-10-21T04:48:09.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.910+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.910+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex started 2023-10-21T04:48:09.911+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:48:09.911+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:09.911+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:09.911+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex stopped 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.911+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.926+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:09.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex closed 2023-10-21T04:48:09.927+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:09.927+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:09.927+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:09.927+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:09.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:09.927+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.927+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:09.927+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:09.927+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.928+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex stopped 2023-10-21T04:48:09.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:09.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:09.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:09.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.939+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:09.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:09.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex closed 2023-10-21T04:48:09.940+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:09.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:48:09.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:48:09.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex sucessfully destroyed 2023-10-21T04:48:09.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:48:09.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:48:09.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index destroyed 2023-10-21T04:48:09.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.950+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.950+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:09.953+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.953+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:09.953+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:09.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.961+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.961+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:09.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex started 2023-10-21T04:48:09.962+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.962+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.963+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.963+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.963+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:09.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex started 2023-10-21T04:48:09.964+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Warmup took 68.680066ms 2023-10-21T04:48:09.965+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 1 WriterThreads 1 2023-10-21T04:48:09.966+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 1 Stopped Billing WUs 2023-10-21T04:48:09.964+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.964+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.966+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index Index: 12396348969790516872 Partition: 1 2023-10-21T04:48:09.967+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:09.968+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 2 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:09.968+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:09.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.968+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:48:09.969+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:09.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:09.969+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:48:09.975+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:68:e7:32:31:80:46:d9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16515427720591431191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2034303749683961252 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 52515533635989429 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:09.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:09.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:09.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:09.976+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:09.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex started 2023-10-21T04:48:09.976+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:09.976+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:09.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:09.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:09.986+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:09.986+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex started 2023-10-21T04:48:09.986+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:48:09.987+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:09.987+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:10.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:10.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:10.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:10.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex stopped 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:10.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:09.986+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:09.986+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:10.021+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:10.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:10.022+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex closed 2023-10-21T04:48:10.022+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:10.022+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:10.022+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:10.023+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:10.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:10.023+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:10.023+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:10.024+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:10.024+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:10.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex stopped 2023-10-21T04:48:10.024+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:10.024+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:10.025+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:10.025+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:10.034+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:10.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:10.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex closed 2023-10-21T04:48:10.035+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:10.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:48:10.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:48:10.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex sucessfully destroyed 2023-10-21T04:48:10.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:48:10.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:48:10.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index destroyed 2023-10-21T04:48:10.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:10.085+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:10.085+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:10.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:10.097+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:10.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:10.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:10.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:10.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:10.106+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:10.106+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:10.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex started 2023-10-21T04:48:10.106+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:10.106+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:10.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:10.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:10.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:10.108+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:10.108+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:10.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex started 2023-10-21T04:48:10.108+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Warmup took 139.835317ms 2023-10-21T04:48:10.108+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 2 WriterThreads 1 2023-10-21T04:48:10.108+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 2 Stopped Billing WUs 2023-10-21T04:48:10.108+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index Index: 12396348969790516872 Partition: 2 2023-10-21T04:48:10.108+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12396348969790516872 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:10.108+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12396348969790516872 PartitionId: 3 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:10.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:10.109+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:10.109+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 51200 recovery UsedSpace 8236 2023-10-21T04:48:10.110+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:10.110+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:10.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:10.111+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:48:10.111+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 58514 recovery UsedSpace 8236 2023-10-21T04:48:10.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:10.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:10.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:10.115+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:10.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex started 2023-10-21T04:48:10.115+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:10.115+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:10.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:10.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:10.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:10.131+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:10.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex started 2023-10-21T04:48:10.132+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:48:10.132+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:48:10.133+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:10.133+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:10.133+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:10.134+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:10.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:10.135+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:10.135+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:10.135+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:10.135+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:10.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex stopped 2023-10-21T04:48:10.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:10.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:10.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:10.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:10.140+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:10.141+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:10.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:10.142+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex closed 2023-10-21T04:48:10.142+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:10.142+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:10.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:10.143+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:10.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:10.143+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:10.143+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:10.143+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:10.143+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:10.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex stopped 2023-10-21T04:48:10.144+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:10.144+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:10.144+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:10.145+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:10.152+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenf5:68:e7:32:31:80:46:d9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16515427720591431191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2034303749683961252 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 52515533635989429 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:10.154+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:10.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:10.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex closed 2023-10-21T04:48:10.154+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:10.154+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:48:10.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:48:10.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex sucessfully destroyed 2023-10-21T04:48:10.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:48:10.158+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:48:10.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index destroyed 2023-10-21T04:48:10.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:10.166+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:10.166+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:10.176+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:10.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:10.177+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:10.177+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:48:10.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:10.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:10.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:10.199+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:10.199+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:10.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex started 2023-10-21T04:48:10.199+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:10.199+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:10.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:10.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:10.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:10.202+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:10.202+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:10.202+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex started 2023-10-21T04:48:10.202+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Warmup took 93.85979ms 2023-10-21T04:48:10.203+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12396348969790516872 partitionId 3 WriterThreads 1 2023-10-21T04:48:10.203+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 12396348969790516872 PartitionId 3 Stopped Billing WUs 2023-10-21T04:48:10.203+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12396348969790516872 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index Index: 12396348969790516872 Partition: 3 2023-10-21T04:48:10.203+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:10.203+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:10.203+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12396348969790516872, shardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:48:10.203+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12396348969790516872, partnShardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:48:10.203+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:10.204+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12396348969790516872 2023-10-21T04:48:10.205+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12396348969790516872 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:10.205+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:48:10.205+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:10.205+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12396348969790516872 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:10.205+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:10.206+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12396348969790516872 Partitions [4 5 6 7 8 1 2 3] 2023-10-21T04:48:10.206+05:30 [Info] StorageMgr::openSnapshot IndexInst:12396348969790516872 Partition:4 No Snapshot Found. 2023-10-21T04:48:10.206+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12396348969790516872 Adding Nil Snapshot. 2023-10-21T04:48:10.254+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 47.616887ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:10.254+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 214.062µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:48:10.294+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:10.294+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:10.299+05:30 [Info] switched currmeta from 288 -> 290 force true 2023-10-21T04:48:10.335+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:10.335+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:10.347+05:30 [Info] switched currmeta from 290 -> 290 force true 2023-10-21T04:48:10.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9998405023127165, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:48:10.966+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:10.966+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:10.995+05:30 [Info] switched currmeta from 290 -> 291 force true 2023-10-21T04:48:11.198+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[12396348969790516872:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:48:11.198+05:30 [Info] Indexer::handleMergePartition Source 3562146078059658886 Target 12396348969790516872 2023-10-21T04:48:11.198+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00995f680 2023-10-21T04:48:11.198+05:30 [Info] MergePartition: sent async request to update index instance 12396348969790516872 rstate moved to ACTIVE 2023-10-21T04:48:11.205+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.722445ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:11.206+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 3562146078059658886, realInst: 12396348969790516872 2023-10-21T04:48:11.217+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 2608585389864151823, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:48:11.217+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 2608585389864151823, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:48:11.264+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 15281531523624343311,realInstId: 0, indexDefn DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:48:11.265+05:30 [Info] Indexer::handleRecoverIndex InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:48:11.265+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 269.217µs 2023-10-21T04:48:11.265+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 313.341µs 2023-10-21T04:48:11.265+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 15281531523624343311 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.266+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 15281531523624343311 PartitionId: 0 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.266+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.267+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1056164 memSz 1866 2023-10-21T04:48:11.267+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 13 count: 1000 metaSz: 2505 2023-10-21T04:48:11.269+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 150553 recovery UsedSpace 16193 2023-10-21T04:48:11.268+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 15281531523624343311, indexDefn DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:48:11.271+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 53.947053ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:11.268+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1058326 memSz 882 2023-10-21T04:48:11.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 13 count: 1000 metaSz: 2505 2023-10-21T04:48:11.272+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 156778 recovery UsedSpace 14166 2023-10-21T04:48:11.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex 2023-10-21T04:48:11.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.276+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex 2023-10-21T04:48:11.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex started 2023-10-21T04:48:11.276+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.276+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex 2023-10-21T04:48:11.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.279+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex 2023-10-21T04:48:11.279+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex started 2023-10-21T04:48:11.279+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15281531523624343311 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:48:11.280+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.280+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.305+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 15281531523624343311 PartitionId 0 Warmup took 39.76986ms 2023-10-21T04:48:11.306+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 15281531523624343311 partitionId 0 WriterThreads 4 2023-10-21T04:48:11.306+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 15281531523624343311 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index Index: 15281531523624343311 Partition: 0 2023-10-21T04:48:11.307+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 15281531523624343311, shardIdMap: map[0:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:48:11.307+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 15281531523624343311, partnShardIdMap: map[0:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:48:11.307+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:11.323+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 15281531523624343311 2023-10-21T04:48:11.327+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:48:11.328+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:48:11.329+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:11.329+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:48:11.329+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:11.329+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 15281531523624343311 Partitions [0] 2023-10-21T04:48:11.330+05:30 [Info] StorageMgr::openSnapshot IndexInst:15281531523624343311 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:48:11.346+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 37.534818ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:11.346+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 94.982µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:48:11.368+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:11.369+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[15281531523624343311:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:48:11.369+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:48:11.369+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:48:11.455+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:11.455+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:11.486+05:30 [Info] switched currmeta from 291 -> 297 force true 2023-10-21T04:48:11.491+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4397940545762329850,realInstId: 0, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:11.491+05:30 [Info] Indexer::handleRecoverIndex InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:48:11.492+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 429.108µs 2023-10-21T04:48:11.492+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 489.224µs 2023-10-21T04:48:11.492+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.492+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 1 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.493+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.493+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.493+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 120330 2023-10-21T04:48:11.493+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4397940545762329850, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:48:11.493+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 124.367297ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:11.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.494+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.494+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.494+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 120330 2023-10-21T04:48:11.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:48:11.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.506+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:48:11.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex started 2023-10-21T04:48:11.506+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.506+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:48:11.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.510+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:48:11.510+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex started 2023-10-21T04:48:11.510+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:48:11.512+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.512+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.534+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Warmup took 41.958587ms 2023-10-21T04:48:11.536+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 1 WriterThreads 1 2023-10-21T04:48:11.536+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 1 Stopped Billing WUs 2023-10-21T04:48:11.536+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index Index: 4397940545762329850 Partition: 1 2023-10-21T04:48:11.537+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.535+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:11.540+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 2 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.542+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.542+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.542+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 120330 2023-10-21T04:48:11.546+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.547+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.548+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.548+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:48:11.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.549+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 120330 2023-10-21T04:48:11.552+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:48:11.553+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex started 2023-10-21T04:48:11.553+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.553+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:48:11.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.559+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.560+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:48:11.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex started 2023-10-21T04:48:11.560+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:48:11.560+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.560+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.597+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Warmup took 55.409452ms 2023-10-21T04:48:11.598+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 2 WriterThreads 1 2023-10-21T04:48:11.598+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 2 Stopped Billing WUs 2023-10-21T04:48:11.598+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index Index: 4397940545762329850 Partition: 2 2023-10-21T04:48:11.598+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.601+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 3 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.601+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.602+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.602+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2479 rp [1] sn: 22 count: 0 metaSz: 2480 2023-10-21T04:48:11.602+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 120330 2023-10-21T04:48:11.603+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 21 count: 0 metaSz: 2479 rp [1] sn: 22 count: 0 metaSz: 2480 2023-10-21T04:48:11.603+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 120330 2023-10-21T04:48:11.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:48:11.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.614+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:48:11.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex started 2023-10-21T04:48:11.614+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.614+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:48:11.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.625+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:48:11.625+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex started 2023-10-21T04:48:11.626+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.626+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.626+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:48:11.645+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:11.645+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:11.664+05:30 [Info] switched currmeta from 297 -> 302 force true 2023-10-21T04:48:11.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9999026364402183, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:48:11.674+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Warmup took 73.398138ms 2023-10-21T04:48:11.675+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 3 WriterThreads 1 2023-10-21T04:48:11.675+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 3 Stopped Billing WUs 2023-10-21T04:48:11.676+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index Index: 4397940545762329850 Partition: 3 2023-10-21T04:48:11.676+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.676+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 4 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.680+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.680+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.680+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 120330 2023-10-21T04:48:11.679+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.682+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.684+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 82040 recovery UsedSpace 120330 2023-10-21T04:48:11.687+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:48:11.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.688+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:48:11.688+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex started 2023-10-21T04:48:11.688+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.688+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.691+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:48:11.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.692+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.692+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:48:11.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex started 2023-10-21T04:48:11.694+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.694+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.695+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:48:11.721+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Warmup took 44.815564ms 2023-10-21T04:48:11.722+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 4 WriterThreads 1 2023-10-21T04:48:11.723+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 4 Stopped Billing WUs 2023-10-21T04:48:11.723+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index Index: 4397940545762329850 Partition: 4 2023-10-21T04:48:11.724+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.726+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 5 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.728+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.728+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.728+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.729+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 120330 2023-10-21T04:48:11.729+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.729+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.729+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 23 count: 0 metaSz: 2479 rp [1] sn: 24 count: 0 metaSz: 2480 2023-10-21T04:48:11.730+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 120330 2023-10-21T04:48:11.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:48:11.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.738+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:48:11.738+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex started 2023-10-21T04:48:11.738+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.738+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:48:11.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.742+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.742+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:48:11.743+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex started 2023-10-21T04:48:11.743+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:48:11.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.750+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:11.750+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:11.773+05:30 [Info] switched currmeta from 302 -> 302 force true 2023-10-21T04:48:11.779+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Warmup took 51.448966ms 2023-10-21T04:48:11.780+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 5 WriterThreads 1 2023-10-21T04:48:11.780+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 5 Stopped Billing WUs 2023-10-21T04:48:11.780+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index Index: 4397940545762329850 Partition: 5 2023-10-21T04:48:11.780+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.780+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 6 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.781+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.781+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2479 rp [1] sn: 23 count: 0 metaSz: 2480 2023-10-21T04:48:11.781+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 120330 2023-10-21T04:48:11.783+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.783+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.784+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2479 rp [1] sn: 23 count: 0 metaSz: 2480 2023-10-21T04:48:11.784+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 120330 2023-10-21T04:48:11.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:48:11.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.790+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:48:11.790+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex started 2023-10-21T04:48:11.790+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.791+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:48:11.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.799+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:48:11.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex started 2023-10-21T04:48:11.800+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:48:11.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.800+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.821+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Warmup took 39.868967ms 2023-10-21T04:48:11.821+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 6 WriterThreads 1 2023-10-21T04:48:11.821+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 6 Stopped Billing WUs 2023-10-21T04:48:11.822+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index Index: 4397940545762329850 Partition: 6 2023-10-21T04:48:11.822+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.822+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 7 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.823+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.823+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2479 rp [1] sn: 23 count: 0 metaSz: 2480 2023-10-21T04:48:11.823+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 120330 2023-10-21T04:48:11.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.824+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.824+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2479 rp [1] sn: 23 count: 0 metaSz: 2480 2023-10-21T04:48:11.824+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 120330 2023-10-21T04:48:11.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:48:11.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.827+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:48:11.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex started 2023-10-21T04:48:11.827+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.827+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:48:11.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.833+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:48:11.833+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex started 2023-10-21T04:48:11.833+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:48:11.833+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.833+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.857+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Warmup took 34.185553ms 2023-10-21T04:48:11.859+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 7 WriterThreads 1 2023-10-21T04:48:11.859+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 7 Stopped Billing WUs 2023-10-21T04:48:11.859+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index Index: 4397940545762329850 Partition: 7 2023-10-21T04:48:11.859+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4397940545762329850 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:11.859+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4397940545762329850 PartitionId: 8 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:11.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:48:11.859+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.859+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2479 rp [1] sn: 23 count: 0 metaSz: 2480 2023-10-21T04:48:11.860+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:11.860+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:48:11.860+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 22 count: 0 metaSz: 2479 rp [1] sn: 23 count: 0 metaSz: 2480 2023-10-21T04:48:11.860+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 120330 2023-10-21T04:48:11.860+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 120330 2023-10-21T04:48:11.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:48:11.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:11.870+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:48:11.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:48:11.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:48:11.871+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:48:11.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex started 2023-10-21T04:48:11.871+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:11.871+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:48:11.871+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.871+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex started 2023-10-21T04:48:11.871+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:48:11.872+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:11.872+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:11.913+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Warmup took 54.195387ms 2023-10-21T04:48:11.914+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4397940545762329850 partitionId 8 WriterThreads 1 2023-10-21T04:48:11.914+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4397940545762329850 PartitionId 8 Stopped Billing WUs 2023-10-21T04:48:11.914+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4397940545762329850 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index Index: 4397940545762329850 Partition: 8 2023-10-21T04:48:11.914+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4397940545762329850, shardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:48:11.914+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4397940545762329850, partnShardIdMap: map[1:[18374611443167983589 6455624821495495854] 2:[18374611443167983589 6455624821495495854] 3:[18374611443167983589 6455624821495495854] 4:[18374611443167983589 6455624821495495854] 5:[18374611443167983589 6455624821495495854] 6:[18374611443167983589 6455624821495495854] 7:[18374611443167983589 6455624821495495854] 8:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:48:11.914+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:11.914+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4397940545762329850 2023-10-21T04:48:11.914+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:11.915+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:48:11.915+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:11.915+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:11.915+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:11.915+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4397940545762329850 Partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:11.916+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:8 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.920+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:1 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.921+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:2 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.922+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:3 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.923+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:4 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.924+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:5 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.925+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:6 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.926+05:30 [Info] StorageMgr::openSnapshot IndexInst:4397940545762329850 Partition:7 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:48:11.929+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.611471ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:11.929+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 119.891µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:48:12.036+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:12.036+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:12.038+05:30 [Info] switched currmeta from 302 -> 303 force true 2023-10-21T04:48:12.540+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4397940545762329850:INDEX_STATE_RECOVERED 15281531523624343311:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:48:12.540+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [2608585389864151823 4390029557575378783] belonging to collectionId: c, ttid: ShardTokenc0:4e:bd:51:d0:75:ef:9b. Initiating build 2023-10-21T04:48:12.583+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [15281531523624343311 4397940545762329850] &{2} 2023-10-21T04:48:12.583+05:30 [Info] Indexer::handleBuildRecoveredIndexes [15281531523624343311 4397940545762329850] 2023-10-21T04:48:12.583+05:30 [Info] bulkUpdateRState: Index instance 15281531523624343311 rstate moved to PENDING 2023-10-21T04:48:12.583+05:30 [Info] bulkUpdateRState: Index instance 4397940545762329850 rstate moved to PENDING 2023-10-21T04:48:12.583+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [15281531523624343311 4397940545762329850] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:48:12.583+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:12.584+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:12.584+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:12.623+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:48:12.630+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43323 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:12.631+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:48:12.631+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:48:12.632+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:48:12.634+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43323 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:12.637+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:48:12.638+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:48:12.639+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:48:12.640+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId 43324 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 29 0 29} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 22 0 22} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 37 0 37} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 26 0 26} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 28 0 28} 2023-10-21T04:48:12.641+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:48:12.642+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:48:12.643+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 15281531523624343311 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:48:12.644+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_%2:_default:c1:2]] 2023-10-21T04:48:12.646+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 4397940545762329850 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:48:12.647+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:48:12.648+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=19,1=29,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=22,17=20,18=20,19=25,20=23,21=28,22=30,23=18,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=37,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=26,57=18,58=26,59=16,60=20,61=24,62=20,63=28] IndexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 43324 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 29 0 29} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 22 0 22} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 37 0 37} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 26 0 26} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 28 0 28} NumVBuckets: 64 2023-10-21T04:48:12.656+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:48:12.658+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:48:12.659+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:48:12.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:48:12.669+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:12.673+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 90.135707ms 2023-10-21T04:48:12.674+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [15281531523624343311 4397940545762329850] 2023-10-21T04:48:12.674+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 90.388166ms 2023-10-21T04:48:12.677+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 136.07755ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:12.677+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[4397940545762329850:true 15281531523624343311:true] 2023-10-21T04:48:12.694+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.730033ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:12.705+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:48:12.706+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:12.706+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:12.710+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 16.182541ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:12.738+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:12.750+05:30 [Info] switched currmeta from 303 -> 307 force true 2023-10-21T04:48:13.011+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60670" doAuth() ... 2023-10-21T04:48:13.011+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60670" auth successful 2023-10-21T04:48:13.011+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:60670" +1 2023-10-21T04:48:13.023+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:13.023+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:13.027+05:30 [Info] switched currmeta from 307 -> 307 force true 2023-10-21T04:48:13.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:48:13.749+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:14.670+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.997498808303383 2023-10-21T04:48:14.672+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:48:14.684+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:14.684+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:14.686+05:30 [Info] switched currmeta from 307 -> 307 force true 2023-10-21T04:48:14.753+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:15.467+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 14580309443106 29 43324 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:48:15.467+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 220432454230336 19 43324 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:15.470+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 135940091877708 21 43324 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:15.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 80577332946777 27 43324 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:15.473+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 247041243079227 24 43324 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:15.474+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 172199206487624 16 43324 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:48:15.477+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 182410957121870 32 43324 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:48:15.477+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 19010051182641 24 43324 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:15.477+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 194695637011621 26 43324 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:15.481+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 68019130447759 21 43324 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:15.481+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 42756027613197 22 43324 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:15.481+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 121484100363817 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.501+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 171853572917441 25 43324 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:15.501+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 107902021898438 23 43324 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:15.502+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 59183965998125 22 43324 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:15.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.517+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.521+05:30 [Info] switched currmeta from 307 -> 309 force true 2023-10-21T04:48:15.538+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.538+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.541+05:30 [Info] switched currmeta from 309 -> 309 force true 2023-10-21T04:48:15.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 1542343912406 21 43324 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:15.571+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 73693878880740 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.571+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 157626280229441 22 43324 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:15.577+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 67844364342499 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.582+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 170065152021297 25 43324 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:15.590+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 194038385561911 28 43324 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:15.590+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 217154782530068 23 43324 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:15.594+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 206194254469959 18 43324 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:48:15.594+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 16864753682212 30 43324 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:48:15.601+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 233111027052204 28 43324 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:15.601+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 170842850732116 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.606+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 58389099238647 24 43324 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:15.609+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 252516046855767 27 43324 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:15.609+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 178705476290398 24 43324 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:15.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 52334512457128 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.613+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 183084439776991 19 43324 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:15.617+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 86344521159606 25 43324 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:15.617+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 270670782796252 22 43324 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:15.618+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 233119567174084 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.621+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 264478627574156 22 43324 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:15.622+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 187961911866991 30 43324 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:48:15.624+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 276412965411368 22 43324 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:15.625+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 94748374195527 19 43324 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:15.628+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 30543404237888 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.631+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 66391235022367 25 43324 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:15.631+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 168721574304277 24 43324 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:15.634+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 100004485169771 26 43324 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:15.634+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 50379500651629 21 43324 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:15.638+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 148816420612283 27 43324 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:15.641+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 236414409908581 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 199648020746400 23 43324 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:15.647+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 199348794242318 37 43324 172.31.5.112:9000. HWT [0-37,37]. 2023-10-21T04:48:15.650+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 131255717944569 26 43324 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:15.650+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 84841900064407 22 43324 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:15.653+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 229485491476250 19 43324 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:15.656+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 27094034037488 19 43324 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:15.659+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 268905923659105 24 43324 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:15.659+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 28675906453951 21 43324 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:15.661+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 70030494603811 21 43324 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:15.661+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 7777834518711 31 43324 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:48:15.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9617792029574033 2023-10-21T04:48:15.668+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 75978859756835 17 43324 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:48:15.671+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 47128363244154 26 43324 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:15.671+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 43207585340528 18 43324 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:48:15.671+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 235447613990794 26 43324 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:15.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 160015043890055 16 43324 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:48:15.674+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 65202412270069 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.677+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 154979908447073 24 43324 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:15.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 170481067504432 20 43324 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:15.680+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 14950536761368 28 43324 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:15.684+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 InstanceIds [4397940545762329850 15281531523624343311] 2023-10-21T04:48:15.684+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 43324 2023-10-21T04:48:15.684+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43324 2023-10-21T04:48:15.684+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:48:15.684+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 15281531523624343311 INDEX_STATE_CATCHUP 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 4397940545762329850 INDEX_STATE_CATCHUP 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 15281531523624343311 true 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 4397940545762329850 true 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 4397940545762329850 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 43324 BuildTS: [0=19,1=29,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=22,17=20,18=20,19=25,20=23,21=28,22=30,23=18,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=37,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=26,57=18,58=26,59=16,60=20,61=24,62=20,63=28] 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 29 0 29} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 22 0 22} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 37 0 37} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 26 0 26} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 28 0 28} , fetchKVSeq false 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:48:15.685+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:48:15.685+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43324 STREAM_ACTIVE 2023-10-21T04:48:15.685+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 43324 2023-10-21T04:48:15.685+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2:_default:c1 43324 2023-10-21T04:48:15.685+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:15.686+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:15.686+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:15.686+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:15.686+05:30 [Info] send progress stats to clients 2023-10-21T04:48:15.686+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43324 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:15.686+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 15281531523624343311 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:48:15.686+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 4397940545762329850 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:48:15.686+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:48:15.686+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:48:15.686+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:48:15.687+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43324 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:15.687+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:48:15.687+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:48:15.687+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:48:15.687+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:48:15.687+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:60670: use of closed network connection 2023-10-21T04:48:15.687+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:60670" exit: read tcp4 127.0.0.1:9121->127.0.0.1:60670: use of closed network connection 2023-10-21T04:48:15.687+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60670" closed ! 2023-10-21T04:48:15.687+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:48:15.687+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:60670" already gone 2023-10-21T04:48:15.687+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:48:15.687+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:48:15.689+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:48:15.689+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2:_default:c1 2023-10-21T04:48:15.698+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.201753ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:15.713+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.440771ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:15.745+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.745+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.746+05:30 [Info] switched currmeta from 309 -> 311 force true 2023-10-21T04:48:15.762+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:15.762+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006852540 2023-10-21T04:48:15.762+05:30 [Info] handleUpdateIndexRState: Index instance 15281531523624343311 rstate moved to ACTIVE 2023-10-21T04:48:15.763+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:48:15.763+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:48:15.763+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:48:15.763+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43324 2023-10-21T04:48:15.763+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:48:15.763+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:48:15.763+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73678 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:15.764+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:48:15.764+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:48:15.764+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:48:15.764+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73678 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:15.764+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:48:15.764+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:48:15.764+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:48:15.764+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 73679 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 29 0 29} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 22 0 22} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 37 0 37} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 26 0 26} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 28 0 28} 2023-10-21T04:48:15.764+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:48:15.764+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1] INIT_STREAM:map[]] 2023-10-21T04:48:15.764+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2] INIT_STREAM:map[]] 2023-10-21T04:48:15.764+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:48:15.764+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 73679 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 29 0 29} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 22 0 22} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 37 0 37} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 26 0 26} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 28 0 28} NumVBuckets: 64 2023-10-21T04:48:15.767+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:48:15.767+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:48:15.770+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.084138ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:15.770+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 10158339249566909515. Retrying... 2023-10-21T04:48:15.771+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[4397940545762329850:true 15281531523624343311:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:48:15.771+05:30 [Info] Indexer::handleMergePartition Source 9584066495338256038 Target 4397940545762329850 2023-10-21T04:48:15.771+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc007b259e0 2023-10-21T04:48:15.771+05:30 [Info] MergePartition: sent async request to update index instance 4397940545762329850 rstate moved to ACTIVE 2023-10-21T04:48:15.771+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:48:15.773+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.773+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.777+05:30 [Info] switched currmeta from 311 -> 312 force true 2023-10-21T04:48:15.787+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.788+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.793+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 21.412496ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:15.793+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 9584066495338256038, realInst: 4397940545762329850 2023-10-21T04:48:15.798+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 5277316367626246544, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:48:15.799+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 5277316367626246544, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:48:15.817+05:30 [Info] switched currmeta from 312 -> 313 force true 2023-10-21T04:48:15.851+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 10158339249566909515,realInstId: 0, indexDefn DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:48:15.851+05:30 [Info] Indexer::handleRecoverIndex InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:48:15.851+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 228.312µs 2023-10-21T04:48:15.851+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 272.735µs 2023-10-21T04:48:15.852+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 10158339249566909515 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[18374611443167983589 6455624821495495854]] 2023-10-21T04:48:15.853+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 10158339249566909515 PartitionId: 0 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:48:15.853+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:48:15.853+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: after recovery: resident ratio 0 requestQuota 1055476 memSz 64 2023-10-21T04:48:15.853+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: RP version: 15 numRPs: 2 rp [0] sn: 16 count: 6 metaSz: 2482 rp [1] sn: 18 count: 6 metaSz: 2483 2023-10-21T04:48:15.854+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: doInit: data UsedSpace 96823 recovery UsedSpace 120678 2023-10-21T04:48:15.854+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 10158339249566909515, indexDefn DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:48:15.857+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 59.359861ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:15.860+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.861+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:48:15.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:48:15.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:48:15.864+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:48:15.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex started 2023-10-21T04:48:15.864+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10158339249566909515 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:48:15.867+05:30 [Info] switched currmeta from 313 -> 316 force true 2023-10-21T04:48:15.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:15.868+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:15.870+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:15.881+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.881+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.886+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 10158339249566909515 PartitionId 0 Warmup took 32.958748ms 2023-10-21T04:48:15.886+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 10158339249566909515 partitionId 0 WriterThreads 4 2023-10-21T04:48:15.886+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 10158339249566909515 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index Index: 10158339249566909515 Partition: 0 2023-10-21T04:48:15.886+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 10158339249566909515, shardIdMap: map[0:[18374611443167983589]] err: 2023-10-21T04:48:15.886+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 10158339249566909515, partnShardIdMap: map[0:[18374611443167983589]], err: 2023-10-21T04:48:15.886+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:15.889+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 10158339249566909515 2023-10-21T04:48:15.889+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:15.889+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:48:15.889+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:15.889+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:15.892+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:15.895+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 10158339249566909515 Partitions [0] 2023-10-21T04:48:15.895+05:30 [Info] StorageMgr::openSnapshot IndexInst:10158339249566909515 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:48:15.896+05:30 [Info] switched currmeta from 316 -> 316 force true 2023-10-21T04:48:15.906+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.95043ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:15.906+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 80.429µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:48:15.965+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:15.965+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:15.980+05:30 [Info] switched currmeta from 316 -> 320 force true 2023-10-21T04:48:16.005+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:16.005+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:16.008+05:30 [Info] switched currmeta from 320 -> 320 force true 2023-10-21T04:48:16.028+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:56492" doAuth() ... 2023-10-21T04:48:16.028+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:56492" auth successful 2023-10-21T04:48:16.029+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:56492" +1 2023-10-21T04:48:16.040+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:16.040+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:16.069+05:30 [Info] switched currmeta from 320 -> 321 force true 2023-10-21T04:48:16.543+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12606056968936758026:[7 8 1 2 3 4 5 6]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:48:16.543+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12606056968936758026:[7 8 1 2 3 4 5 6] 14845378835615010356:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:48:16.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9561955295973509 2023-10-21T04:48:16.688+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:48:16.690+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 220432454230336 19 73679 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:16.692+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 14580309443106 29 73679 172.31.5.112:9000. HWT [0-29,29]. 2023-10-21T04:48:16.694+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 135940091877708 21 73679 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:16.694+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 80577332946777 27 73679 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:16.697+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 247041243079227 24 73679 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:16.700+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 182410957121870 32 73679 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:48:16.700+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 172199206487624 16 73679 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:48:16.700+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 19010051182641 24 73679 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:16.700+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 194695637011621 26 73679 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:16.703+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 42756027613197 22 73679 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:16.704+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 121484100363817 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.707+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 68019130447759 21 73679 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:16.714+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 107902021898438 23 73679 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:16.714+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 171853572917441 25 73679 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:16.715+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 59183965998125 22 73679 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:16.715+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 1542343912406 21 73679 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:16.717+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 157626280229441 22 73679 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:16.718+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 73693878880740 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.720+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 67844364342499 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.722+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 170065152021297 25 73679 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:16.728+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 16864753682212 30 73679 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:48:16.728+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 217154782530068 23 73679 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:16.728+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 194038385561911 28 73679 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:16.731+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 206194254469959 18 73679 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:48:16.732+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 170842850732116 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.737+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 58389099238647 24 73679 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:16.737+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 233111027052204 28 73679 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:16.740+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 178705476290398 24 73679 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:16.740+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 252516046855767 27 73679 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:16.744+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 183084439776991 19 73679 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:16.750+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 86344521159606 25 73679 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:16.751+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 270670782796252 22 73679 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:16.753+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 52334512457128 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.756+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 233119567174084 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.758+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 187961911866991 30 73679 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:48:16.758+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 264478627574156 22 73679 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:16.760+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 276412965411368 22 73679 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:16.763+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 168721574304277 24 73679 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:16.763+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 66391235022367 25 73679 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:16.763+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 94748374195527 19 73679 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:16.763+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 30543404237888 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.765+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 50379500651629 21 73679 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:16.767+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 199348794242318 37 73679 172.31.5.112:9000. HWT [0-37,37]. 2023-10-21T04:48:16.767+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 148816420612283 27 73679 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:16.768+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 100004485169771 26 73679 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:16.769+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 236414409908581 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.769+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 199648020746400 23 73679 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:16.770+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 131255717944569 26 73679 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:16.770+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 84841900064407 22 73679 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:16.772+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 229485491476250 19 73679 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:16.776+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 27094034037488 19 73679 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:16.776+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 28675906453951 21 73679 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:16.778+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 7777834518711 31 73679 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:48:16.778+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 268905923659105 24 73679 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:16.780+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 70030494603811 21 73679 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:16.782+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 75978859756835 17 73679 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:48:16.784+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 235447613990794 26 73679 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:16.785+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 43207585340528 18 73679 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:48:16.786+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 47128363244154 26 73679 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:16.790+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 160015043890055 16 73679 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:48:16.790+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 65202412270069 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.794+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 170481067504432 20 73679 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:16.794+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 154979908447073 24 73679 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:16.797+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 14950536761368 28 73679 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:16.798+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2 InstanceIds [15281531523624343311 4397940545762329850] 2023-10-21T04:48:16.798+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 73679 2023-10-21T04:48:16.798+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 73679 2023-10-21T04:48:16.798+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:48:16.798+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:48:16.798+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:48:16.798+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:48:16.798+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 73679 2023-10-21T04:48:16.798+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 73679 STREAM_ACTIVE 2023-10-21T04:48:16.798+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:48:16.798+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:48:16.873+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[10158339249566909515:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:48:16.873+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [5277316367626246544] belonging to collectionId: 9, ttid: ShardTokenc0:4e:bd:51:d0:75:ef:9b. Initiating build 2023-10-21T04:48:16.885+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [10158339249566909515] &{2} 2023-10-21T04:48:16.885+05:30 [Info] Indexer::handleBuildRecoveredIndexes [10158339249566909515] 2023-10-21T04:48:16.885+05:30 [Info] bulkUpdateRState: Index instance 10158339249566909515 rstate moved to PENDING 2023-10-21T04:48:16.885+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [10158339249566909515] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:48:16.885+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:16.885+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:16.885+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:16.886+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:48:16.886+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:48:16.886+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:48:16.887+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:16.887+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:48:16.887+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:48:16.887+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 7 New 0 2023-10-21T04:48:16.887+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:16.887+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:48:16.887+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:48:16.887+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:48:16.887+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_system:_query reset bucket state to NIL 2023-10-21T04:48:16.887+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:48:16.887+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId 969 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 29 29 29} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 22 22 22} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 37 37 37} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 26 0 26} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} 2023-10-21T04:48:16.888+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:48:16.888+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:48:16.888+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:48:16.888+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:48:16.888+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 10158339249566909515 stream INIT_STREAM keyspaceId bucket_%2:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:48:16.888+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:48:16.888+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=19,1=29,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=22,17=20,18=20,19=25,20=23,21=28,22=30,23=18,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=37,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=26,57=18,58=26,59=16,60=20,61=24,62=20,63=28] IndexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 969 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 29 29 29} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 22 22 22} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 18 0 18} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 37 37 37} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 26 0 26} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} NumVBuckets: 64 2023-10-21T04:48:16.888+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:48:16.888+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:48:16.889+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:16.889+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 3.841326ms 2023-10-21T04:48:16.889+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 3.880986ms 2023-10-21T04:48:16.889+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [10158339249566909515] 2023-10-21T04:48:16.889+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 15.683239ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:16.901+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[10158339249566909515:true] 2023-10-21T04:48:16.901+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_system:_query Check pending stream merge. 2023-10-21T04:48:16.902+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.798009ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:16.908+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:16.928+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:16.930+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:16.949+05:30 [Info] switched currmeta from 321 -> 323 force true 2023-10-21T04:48:16.959+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:16.959+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:16.966+05:30 [Info] switched currmeta from 323 -> 323 force true 2023-10-21T04:48:16.967+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:48:16.967+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[6 7 8 1 2 3 4 5] 15281531523624343311:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:48:17.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8278318720304529, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:48:17.799+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:48:17.799+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:48:17.893+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:48:17.900+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60830" doAuth() ... 2023-10-21T04:48:17.900+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60830" auth successful 2023-10-21T04:48:17.901+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:60830" +1 2023-10-21T04:48:17.912+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:18.182+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 0 220432454230336 19 969 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:18.182+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 1 14580309443106 29 969 172.31.5.112:9000. HWT [29-29,29]. 2023-10-21T04:48:18.185+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 2 80577332946777 27 969 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:18.188+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 6 194695637011621 26 969 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:18.188+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 5 172199206487624 16 969 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:48:18.188+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 3 135940091877708 21 969 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:18.189+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_system:_query setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:48:18.189+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 4 247041243079227 24 969 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:18.192+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 7 19010051182641 24 969 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:18.195+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 9 42756027613197 22 969 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:18.196+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 8 182410957121870 32 969 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:48:18.196+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 10158339249566909515 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:48:18.197+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 10158339249566909515 PartitionId: 0 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:6 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.211µs SnapOpenDur 84.605µs 2023-10-21T04:48:18.197+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 10158339249566909515, PartitionId 0 SnapshotId 2 Creating recovery point ... 2023-10-21T04:48:18.198+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 10 121484100363817 20 969 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:18.200+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 11 68019130447759 21 969 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:18.203+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 12 171853572917441 25 969 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:18.203+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 13 107902021898438 23 969 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:18.205+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 14 59183965998125 22 969 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:18.208+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 10158339249566909515, PartitionId 0 SnapshotId 2 Created recovery point (took 11.059195ms) 2023-10-21T04:48:18.209+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 10158339249566909515, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:48:18.209+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 15 1542343912406 21 969 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:18.211+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 17 73693878880740 20 969 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:18.211+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 16 157626280229441 22 969 172.31.5.112:9000. HWT [22-22,22]. 2023-10-21T04:48:18.214+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 18 67844364342499 20 969 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:48:18.216+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 19 170065152021297 25 969 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:18.219+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 21 194038385561911 28 969 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:18.219+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 20 217154782530068 23 969 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:18.220+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 22 16864753682212 30 969 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:48:18.222+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 23 206194254469959 18 969 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:48:18.225+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 25 233111027052204 28 969 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:48:18.226+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 27 178705476290398 24 969 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:18.226+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 26 58389099238647 24 969 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:18.226+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 24 170842850732116 20 969 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:18.228+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 28 252516046855767 27 969 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:18.230+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 29 52334512457128 20 969 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:18.232+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 31 270670782796252 22 969 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:18.232+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 30 183084439776991 19 969 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:18.236+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 33 233119567174084 20 969 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:18.237+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 32 86344521159606 25 969 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:18.238+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 35 187961911866991 30 969 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:48:18.238+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 34 264478627574156 22 969 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:18.242+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 38 30543404237888 20 969 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:18.242+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 36 276412965411368 22 969 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:18.242+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 37 94748374195527 19 969 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:18.245+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 39 66391235022367 25 969 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:48:18.247+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 41 50379500651629 21 969 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:18.247+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 40 168721574304277 24 969 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:18.251+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 42 100004485169771 26 969 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:18.255+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 44 236414409908581 20 969 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:48:18.255+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 43 148816420612283 27 969 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:48:18.258+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 45 199348794242318 37 969 172.31.5.112:9000. HWT [37-37,37]. 2023-10-21T04:48:18.265+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 48 131255717944569 26 969 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:18.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 46 199648020746400 23 969 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:48:18.266+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 47 84841900064407 22 969 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:48:18.267+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 50 27094034037488 19 969 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:18.267+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 49 229485491476250 19 969 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:48:18.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 52 268905923659105 24 969 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:18.270+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 51 28675906453951 21 969 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:18.277+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 54 70030494603811 21 969 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:48:18.277+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 53 7777834518711 31 969 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:48:18.278+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 55 75978859756835 17 969 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:48:18.284+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 58 47128363244154 26 969 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:18.284+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 57 43207585340528 18 969 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:48:18.286+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 59 160015043890055 16 969 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:48:18.287+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 56 235447613990794 26 969 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:48:18.296+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 60 65202412270069 20 969 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:48:18.296+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 62 170481067504432 20 969 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:48:18.296+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 61 154979908447073 24 969 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:48:18.307+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 63 14950536761368 28 969 172.31.5.112:9000. HWT [28-28,28]. 2023-10-21T04:48:18.310+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_system:_query InstanceIds [10158339249566909515] 2023-10-21T04:48:18.310+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query Cid 9 SessionId 969 2023-10-21T04:48:18.310+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 969 2023-10-21T04:48:18.310+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:48:18.310+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:48:18.311+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 10158339249566909515 INDEX_STATE_CATCHUP 2023-10-21T04:48:18.311+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 10158339249566909515 true 2023-10-21T04:48:18.312+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 10158339249566909515 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query Session: 969 BuildTS: [0=19,1=29,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=22,17=20,18=20,19=25,20=23,21=28,22=30,23=18,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=37,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=26,57=18,58=26,59=16,60=20,61=24,62=20,63=28] 2023-10-21T04:48:18.312+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_system:_query, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 30 29 30} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 23 22 23} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 19 0 19} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 38 37 38} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 0 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} , fetchKVSeq false 2023-10-21T04:48:18.313+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:48:18.313+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:48:18.313+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 969 STREAM_ACTIVE 2023-10-21T04:48:18.313+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=4,1=30,2=4,3=4,4=4,5=4,6=4,7=4,8=4,9=4,10=4,11=4,12=4,13=4,14=4,15=4,16=23,17=4,18=20,19=4,20=4,21=4,22=4,23=19,24=20,25=4,26=4,27=4,28=4,29=4,30=4,31=4,32=4,33=4,34=4,35=30,36=4,37=4,38=4,39=4,40=4,41=4,42=4,43=4,44=4,45=38,46=4,47=4,48=4,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=27,57=4,58=4,59=4,60=20,61=4,62=20,63=28] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:18.314+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 969 2023-10-21T04:48:18.314+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_system:_query 969 2023-10-21T04:48:18.314+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:18.315+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:18.314+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:48:18.315+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:48:18.316+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:18.316+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:18.317+05:30 [Info] send progress stats to clients 2023-10-21T04:48:18.317+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:48:18.317+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:48:18.319+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:48:18.323+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 969 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:48:18.326+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:48:18.327+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.134663ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:18.335+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_%2 2023-10-21T04:48:18.335+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 969 2023-10-21T04:48:18.335+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 969 2023-10-21T04:48:18.335+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:48:18.335+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:48:18.335+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 10158339249566909515 false 2023-10-21T04:48:18.335+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_system:_query 10158339249566909515 2023-10-21T04:48:18.335+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:48:18.335+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 969 2023-10-21T04:48:18.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:18.413+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:18.415+05:30 [Info] switched currmeta from 323 -> 324 force true 2023-10-21T04:48:18.532+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:18.533+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:18.540+05:30 [Info] switched currmeta from 324 -> 325 force true 2023-10-21T04:48:18.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7055256590138074, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:48:18.667+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is enabled 2023-10-21T04:48:18.915+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:19.315+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:48:19.315+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_system:_query 2023-10-21T04:48:19.315+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 10158339249566909515 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId: 969, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 30 29 30} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 24 0 24} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 23 22 23} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 19 0 19} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 38 37 38} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 0 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} 2023-10-21T04:48:19.315+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 10158339249566909515 INDEX_STATE_ACTIVE 2023-10-21T04:48:19.315+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query State Changed to INACTIVE 2023-10-21T04:48:19.315+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:48:19.315+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:48:19.315+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 7 New 0 2023-10-21T04:48:19.315+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 969 2023-10-21T04:48:19.315+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:19.316+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080] 2023-10-21T04:48:19.316+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:48:19.316+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 969 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:19.316+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 10158339249566909515 from stream INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:48:19.316+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:48:19.316+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:48:19.316+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 7 New 0 2023-10-21T04:48:19.316+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 969 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:19.316+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:48:19.316+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query total received key-versions: map[0:0 1:1 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:1 17:0 18:0 19:0 20:0 21:0 22:0 23:1 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:1 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:1 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:48:19.316+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query latest sequence numbers: map[0:0 1:30 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:23 17:0 18:0 19:0 20:0 21:0 22:0 23:19 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:38 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:27 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:48:19.316+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:48:19.317+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:60830: use of closed network connection 2023-10-21T04:48:19.317+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:60830" exit: read tcp4 127.0.0.1:9121->127.0.0.1:60830: use of closed network connection 2023-10-21T04:48:19.317+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:60830" closed ! 2023-10-21T04:48:19.317+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:48:19.317+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:60830" already gone 2023-10-21T04:48:19.317+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:48:19.317+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:48:19.317+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:48:19.317+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:48:19.317+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:48:19.317+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:19.317+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId 969 2023-10-21T04:48:19.318+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:48:19.335+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.865403ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:19.371+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_system:_query]. Err genServer.closed 2023-10-21T04:48:19.374+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:48:19.375+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Treating genServer.closed As Success 2023-10-21T04:48:19.377+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 969 2023-10-21T04:48:19.379+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 969 2023-10-21T04:48:19.380+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:48:19.402+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:19.403+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:19.410+05:30 [Info] switched currmeta from 325 -> 326 force true 2023-10-21T04:48:19.544+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:19.544+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:19.547+05:30 [Info] switched currmeta from 326 -> 327 force true 2023-10-21T04:48:19.699+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:48:19.707+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:19.707+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:19.710+05:30 [Info] switched currmeta from 327 -> 327 force true 2023-10-21T04:48:19.918+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:19.919+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc005e7e900 2023-10-21T04:48:19.919+05:30 [Info] handleUpdateIndexRState: Index instance 10158339249566909515 rstate moved to ACTIVE 2023-10-21T04:48:19.927+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.219468ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:19.927+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[10158339249566909515:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:48:19.927+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardTokenc0:4e:bd:51:d0:75:ef:9b, elapsed(sec): 10.779286407 2023-10-21T04:48:19.930+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardTokenc0:4e:bd:51:d0:75:ef:9b 2023-10-21T04:48:19.930+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken3a:41:36:5d:f0:c3:a0:ee 2023-10-21T04:48:19.930+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[5629943257627822009:true 17327901024118572215:true] 2023-10-21T04:48:19.930+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.522µs 2023-10-21T04:48:19.930+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [17327901024118572215 5629943257627822009], isLockedForRecovery: true 2023-10-21T04:48:19.930+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [17327901024118572215 5629943257627822009], errMap: map[5629943257627822009: 17327901024118572215:], elapsed: 11.026µs 2023-10-21T04:48:19.938+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:19.939+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 4841852412665776737, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:48:19.939+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 4841852412665776737, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:48:19.978+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17044257022107709142,realInstId: 0, indexDefn DefnId: 4841852412665776737 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:48:19.985+05:30 [Info] Indexer::handleRecoverIndex InstId: 17044257022107709142 Defn: DefnId: 4841852412665776737 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:48:19.986+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 329.592µs 2023-10-21T04:48:19.986+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 375.302µs 2023-10-21T04:48:19.986+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17044257022107709142 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[17327901024118572215 5629943257627822009]] 2023-10-21T04:48:19.986+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17044257022107709142 PartitionId: 0 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:48:19.986+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': added 2023-10-21T04:48:19.987+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17044257022107709142, indexDefn DefnId: 4841852412665776737 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:48:19.987+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 48.427336ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:19.990+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCleaner initialized 2023-10-21T04:48:19.990+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:19.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:48:19.991+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCleaner initialized for recovery 2023-10-21T04:48:19.991+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:19.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery 2023-10-21T04:48:19.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:48:19.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data], Shared [true] 2023-10-21T04:48:19.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1232896] Data log Head [0] Tail [1368064] 2023-10-21T04:48:19.991+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1232896] 2023-10-21T04:48:19.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:19.993+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex 2023-10-21T04:48:19.993+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swap is disabled 2023-10-21T04:48:19.993+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:19.993+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex 2023-10-21T04:48:19.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:19.994+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex 2023-10-21T04:48:19.994+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:19.994+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex 2023-10-21T04:48:19.995+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:19.995+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex 2023-10-21T04:48:19.996+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:19.997+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex 2023-10-21T04:48:19.998+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:19.998+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex 2023-10-21T04:48:19.999+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.000+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex 2023-10-21T04:48:20.000+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.000+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex 2023-10-21T04:48:20.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.001+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:48:20.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.001+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:48:20.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.001+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:48:20.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.001+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:48:20.001+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.001+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:48:20.002+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.002+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:48:20.002+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.002+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:48:20.002+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.002+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:48:20.004+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.004+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex 2023-10-21T04:48:20.006+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCleaner initialized 2023-10-21T04:48:20.006+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:20.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:48:20.008+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : replayOffset: 1363968 (RP:1343488) (CP:1363968) 2023-10-21T04:48:20.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1363968] took [17.359103ms] 2023-10-21T04:48:20.008+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1055476, resident 1, memSz 933, memSzIndex 0, numRecords 12 2023-10-21T04:48:20.008+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889914 controller mem 1252 2023-10-21T04:48:20.009+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.013+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889870 controller mem 1252 2023-10-21T04:48:20.013+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.013+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889826 controller mem 1252 2023-10-21T04:48:20.014+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.014+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889782 controller mem 1252 2023-10-21T04:48:20.014+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.014+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889738 controller mem 1252 2023-10-21T04:48:20.014+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.015+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889694 controller mem 1252 2023-10-21T04:48:20.015+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.015+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889650 controller mem 1252 2023-10-21T04:48:20.015+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.015+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889650 controller mem 1252 2023-10-21T04:48:20.015+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.016+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889606 controller mem 1252 2023-10-21T04:48:20.016+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.016+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889606 controller mem 1252 2023-10-21T04:48:20.016+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.016+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889606 controller mem 1252 2023-10-21T04:48:20.016+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.016+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889606 controller mem 1252 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1056310, resident 1, memSz 74987, memSzIndex 262, numRecords 1000 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889606 controller mem 1252 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889562 controller mem 1252 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889562 controller mem 1252 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889562 controller mem 1252 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889562 controller mem 1252 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 18889562 controller mem 1252 2023-10-21T04:48:20.017+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1368064] replayOffset [1363968] 2023-10-21T04:48:20.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [26.227044ms] 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.017+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data], Shared [true]. Built [18] plasmas, took [26.952379ms] 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: after recovery: resident ratio 0 requestQuota 1056310 memSz 676 2023-10-21T04:48:20.034+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCleaner initialized for recovery 2023-10-21T04:48:20.034+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:48:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery 2023-10-21T04:48:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:48:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data], Shared [true] 2023-10-21T04:48:20.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[995328] Data log Head [0] Tail [1171456] 2023-10-21T04:48:20.035+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [995328] 2023-10-21T04:48:20.037+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.037+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex 2023-10-21T04:48:20.037+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.037+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex 2023-10-21T04:48:20.037+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.054+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex 2023-10-21T04:48:20.018+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 23 count: 1000 metaSz: 2507 2023-10-21T04:48:20.045+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 7, actionOnNode: 127.0.0.1:9003, currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:48:20.047+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:20.055+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.055+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: doInit: data UsedSpace 148480 recovery UsedSpace 13740 2023-10-21T04:48:20.058+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:20.058+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex 2023-10-21T04:48:20.062+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.062+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex 2023-10-21T04:48:20.063+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.063+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex 2023-10-21T04:48:20.063+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.064+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex 2023-10-21T04:48:20.064+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.072+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:48:20.072+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 18] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/85664c0432a8df82b3d3efd913473cb1 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:85664c0432a8df82b3d3efd913473cb1]}]} 2023-10-21T04:48:20.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex 2023-10-21T04:48:20.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:48:20.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:48:20.073+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swapper started 2023-10-21T04:48:20.073+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex 2023-10-21T04:48:20.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex started 2023-10-21T04:48:20.073+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:20.073+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:20.074+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex 2023-10-21T04:48:20.091+05:30 [Info] switched currmeta from 327 -> 330 force true 2023-10-21T04:48:20.095+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.096+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:48:20.095+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:48:20.097+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 18] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:48:20.111+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/85664c0432a8df82b3d3efd913473cb1, rev: [] 2023-10-21T04:48:20.111+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/85664c0432a8df82b3d3efd913473cb1, rev: [] 2023-10-21T04:48:20.111+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:48:20.111+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:48:20.111+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:48:20.112+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:48:20.116+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:20.116+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:20.117+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.117+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:48:20.118+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.120+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:48:20.122+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.124+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:48:20.125+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.126+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:48:20.126+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.126+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:48:20.127+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.128+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:48:20.129+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.130+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:48:20.133+05:30 [Info] switched currmeta from 330 -> 330 force true 2023-10-21T04:48:20.142+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:48:20.145+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex 2023-10-21T04:48:20.146+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : replayOffset: 1167360 (RP:1134592) (CP:1167360) 2023-10-21T04:48:20.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1167360] took [111.268634ms] 2023-10-21T04:48:20.146+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.146+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674816 controller mem 3642 2023-10-21T04:48:20.147+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.147+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674816 controller mem 3642 2023-10-21T04:48:20.147+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.147+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674816 controller mem 3642 2023-10-21T04:48:20.147+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.147+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674816 controller mem 3642 2023-10-21T04:48:20.147+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1056280, resident 1, memSz 69741, memSzIndex 811, numRecords 1000 2023-10-21T04:48:20.148+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674816 controller mem 3642 2023-10-21T04:48:20.148+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.155+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674772 controller mem 3642 2023-10-21T04:48:20.155+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.155+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674728 controller mem 3642 2023-10-21T04:48:20.155+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674728 controller mem 3642 2023-10-21T04:48:20.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674728 controller mem 3642 2023-10-21T04:48:20.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674728 controller mem 3642 2023-10-21T04:48:20.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.157+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674728 controller mem 3642 2023-10-21T04:48:20.157+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.157+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674684 controller mem 3642 2023-10-21T04:48:20.157+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.157+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674640 controller mem 3642 2023-10-21T04:48:20.158+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.158+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674596 controller mem 3642 2023-10-21T04:48:20.158+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.158+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674552 controller mem 3642 2023-10-21T04:48:20.158+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.158+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674552 controller mem 3642 2023-10-21T04:48:20.159+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:48:20.159+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 35674508 controller mem 3642 2023-10-21T04:48:20.159+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1171456] replayOffset [1167360] 2023-10-21T04:48:20.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [126.16615ms] 2023-10-21T04:48:20.162+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.162+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.163+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.163+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.164+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.164+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.164+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.164+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.164+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.165+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.165+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.165+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.165+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.166+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.166+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.166+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.166+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:48:20.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data], Shared [true]. Built [17] plasmas, took [131.714001ms] 2023-10-21T04:48:20.166+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: after recovery: resident ratio 0 requestQuota 1056280 memSz 1878 2023-10-21T04:48:20.167+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: RP version: 2 numRPs: 1 rp [0] sn: 23 count: 1000 metaSz: 2507 2023-10-21T04:48:20.167+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: doInit: data UsedSpace 144825 recovery UsedSpace 16218 2023-10-21T04:48:20.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 23 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex 2023-10-21T04:48:20.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:48:20.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:48:20.175+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex 2023-10-21T04:48:20.175+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex started 2023-10-21T04:48:20.176+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17044257022107709142 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:48:20.176+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:48:20.176+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:48:20.234+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17044257022107709142 PartitionId 0 Warmup took 247.883963ms 2023-10-21T04:48:20.235+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17044257022107709142 partitionId 0 WriterThreads 4 2023-10-21T04:48:20.235+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17044257022107709142 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index Index: 17044257022107709142 Partition: 0 2023-10-21T04:48:20.235+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17044257022107709142, shardIdMap: map[0:[17327901024118572215 5629943257627822009]] err: 2023-10-21T04:48:20.235+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17044257022107709142, partnShardIdMap: map[0:[17327901024118572215 5629943257627822009]], err: 2023-10-21T04:48:20.235+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17044257022107709142 Defn: DefnId: 4841852412665776737 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:20.235+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17044257022107709142 2023-10-21T04:48:20.235+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17044257022107709142 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:48:20.235+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:48:20.236+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:20.236+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17044257022107709142 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:48:20.236+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17044257022107709142 Defn: DefnId: 4841852412665776737 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:48:20.236+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17044257022107709142 Partitions [0] 2023-10-21T04:48:20.237+05:30 [Info] StorageMgr::openSnapshot IndexInst:17044257022107709142 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:48:20.250+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.375806ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:20.250+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 87.242µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:48:20.323+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:20.323+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:20.329+05:30 [Info] switched currmeta from 330 -> 331 force true 2023-10-21T04:48:20.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:48:20.437+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:48:20.616+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:20.617+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:20.622+05:30 [Info] switched currmeta from 331 -> 331 force true 2023-10-21T04:48:20.703+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:48:20.763+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:20.763+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:20.773+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:48:20.777+05:30 [Info] switched currmeta from 331 -> 331 force true 2023-10-21T04:48:21.045+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:48:21.058+05:30 [Info] ShardRebalancer::waitForIndexState Cancel Received 2023-10-21T04:48:21.069+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken3a:41:36:5d:f0:c3:a0:ee 2023-10-21T04:48:21.069+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:48:21.093+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno cache..., established connections: map[172.31.5.112:49660:172.31.5.112:12000] 2023-10-21T04:48:21.114+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:48:21.114+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken3a:41:36:5d:f0:c3:a0:ee MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17044257022107709142 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 6868744514623643769 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 16667180614285973080 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:21.114+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 85664c0432a8df82b3d3efd913473cb1 TransferTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:48:21.114+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000102299 2023-10-21T04:48:21.114+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardToken3a:41:36:5d:f0:c3:a0:ee, dropIndexes: true 2023-10-21T04:48:21.116+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee, shardId :0 2023-10-21T04:48:21.116+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardToken3a_41_36_5d_f0_c3_a0_ee, shardId :0 2023-10-21T04:48:21.125+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 17044257022107709142 &{1} 2023-10-21T04:48:21.125+05:30 [Info] Indexer::handleDropIndex - IndexInstId 17044257022107709142 2023-10-21T04:48:21.125+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 4841852412665776737, instId: 17044257022107709142 2023-10-21T04:48:21.125+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [17044257022107709142] 2023-10-21T04:48:21.125+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [17044257022107709142] 2023-10-21T04:48:21.125+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:21.125+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [17044257022107709142] 2023-10-21T04:48:21.125+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17044257022107709142, PartitionId 0, IndexDefnId 4841852412665776737. 2023-10-21T04:48:21.125+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:21.125+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:21.126+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:21.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex] 2023-10-21T04:48:21.125+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 17044257022107709142 Defn: DefnId: 4841852412665776737 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:48:21.126+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex 2023-10-21T04:48:21.126+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 818.68µs 2023-10-21T04:48:21.126+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex] 2023-10-21T04:48:21.126+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 854.266µs 2023-10-21T04:48:21.126+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex] 2023-10-21T04:48:21.126+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 17044257022107709142 2023-10-21T04:48:21.126+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex 2023-10-21T04:48:21.126+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex stopped 2023-10-21T04:48:21.126+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:21.126+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:21.126+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:21.126+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:21.130+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:21.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex] 2023-10-21T04:48:21.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex closed 2023-10-21T04:48:21.130+05:30 [Info] bucket_1/idx_secondary/Mainstore#17044257022107709142:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:21.130+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:21.130+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:21.130+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:21.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex] 2023-10-21T04:48:21.130+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex 2023-10-21T04:48:21.130+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex] 2023-10-21T04:48:21.159+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 44.378826ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:21.162+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 7790129686881525591 instance id 6868744514623643769 real instance id 16759912838651900043 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:21.162+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 327.119µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:21.163+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 16823504650100619022 instance id 16667180614285973080 real instance id 4186965058349029465 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:21.163+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 163.371µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:21.164+05:30 [Info] LifecycleMgr.DeleteIndex() : index 668551233566132751 does not exist. 2023-10-21T04:48:21.164+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 107.616µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:21.164+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [17327901024118572215 5629943257627822009] 2023-10-21T04:48:21.181+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex] 2023-10-21T04:48:21.181+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex 2023-10-21T04:48:21.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex stopped 2023-10-21T04:48:21.182+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:21.182+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:21.182+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:21.182+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:21.188+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:21.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex] 2023-10-21T04:48:21.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex closed 2023-10-21T04:48:21.190+05:30 [Info] bucket_1/idx_secondary/Backstore#17044257022107709142:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:21.190+05:30 [Info] Indexer::cleanupIndexData IndexInst 17044257022107709142 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:21.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex ... 2023-10-21T04:48:21.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex ... 2023-10-21T04:48:21.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:48:21.205+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/mainIndex sucessfully destroyed 2023-10-21T04:48:21.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex ... 2023-10-21T04:48:21.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex ... 2023-10-21T04:48:21.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:48:21.210+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17044257022107709142_0.index destroyed 2023-10-21T04:48:21.213+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17044257022107709142, PartitionId 0, IndexDefnId 4841852412665776737. 2023-10-21T04:48:21.213+05:30 [Info] Indexer::cleanupIndexData IndexInst 17044257022107709142 Partition 0 Destroy Done 2023-10-21T04:48:21.262+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:21.262+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:21.270+05:30 [Info] switched currmeta from 331 -> 333 force true 2023-10-21T04:48:21.324+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:21.324+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:21.328+05:30 [Info] switched currmeta from 333 -> 333 force true 2023-10-21T04:48:21.355+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49690:172.31.5.112:12000] 2023-10-21T04:48:21.372+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49696:172.31.5.112:12000] 2023-10-21T04:48:21.403+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49698:172.31.5.112:12000] 2023-10-21T04:48:21.431+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49694:172.31.5.112:12000] 2023-10-21T04:48:21.441+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49692:172.31.5.112:12000] 2023-10-21T04:48:21.467+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49700:172.31.5.112:12000] 2023-10-21T04:48:21.489+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49702:172.31.5.112:12000] 2023-10-21T04:48:21.506+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49704:172.31.5.112:12000] 2023-10-21T04:48:21.525+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49706:172.31.5.112:12000] 2023-10-21T04:48:21.536+05:30 [Info] {bucket,feeds} "bucket_1" created for dcp_seqno worker cache..., established connections: map[172.31.5.112:49708:172.31.5.112:12000] 2023-10-21T04:48:21.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 17327901024118572215 2023-10-21T04:48:21.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 5629943257627822009 2023-10-21T04:48:21.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 5629943257627822009 2023-10-21T04:48:21.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 17327901024118572215 2023-10-21T04:48:21.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [17327901024118572215 5629943257627822009], elapsed: 529.072587ms 2023-10-21T04:48:21.693+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:21.693+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:48:21.693+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swapper stopped 2023-10-21T04:48:21.693+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': stopped 2023-10-21T04:48:21.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:21.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:21.693+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:21.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:21.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All daemons stopped 2023-10-21T04:48:21.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:21.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.Close: Begin close 2023-10-21T04:48:21.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.Close: Done close 2023-10-21T04:48:21.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:21.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:21.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:21.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:21.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.Close: Begin close 2023-10-21T04:48:21.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.Close: Done close 2023-10-21T04:48:21.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:21.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:21.703+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:21.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:48:21.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shutdown completed 2023-10-21T04:48:21.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:21.703+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) removed 2023-10-21T04:48:21.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All daemons stopped 2023-10-21T04:48:21.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance closed 2023-10-21T04:48:21.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : All instance destroyed 2023-10-21T04:48:21.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:48:21.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:21.705+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:48:21.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:21.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:21.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:21.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:21.705+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All daemons stopped 2023-10-21T04:48:21.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:21.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.Close: Begin close 2023-10-21T04:48:21.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.Close: Done close 2023-10-21T04:48:21.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:21.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:21.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:21.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:21.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.Close: Begin close 2023-10-21T04:48:21.709+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.Close: Done close 2023-10-21T04:48:21.709+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:21.709+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:21.709+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:21.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:48:21.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shutdown completed 2023-10-21T04:48:21.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:21.709+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) removed 2023-10-21T04:48:21.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All daemons stopped 2023-10-21T04:48:21.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance closed 2023-10-21T04:48:21.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : All instance destroyed 2023-10-21T04:48:21.711+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:48:21.711+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': Cleaned up Tenant with name:bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:48:21.711+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [17327901024118572215 5629943257627822009], elapsed(sec): 0.546450037 2023-10-21T04:48:21.725+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokena7:ea:70:79:ce:7f:27:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenc0:4e:bd:51:d0:75:ef:9b Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 1637471422672568438 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9725618697950995746 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14845378835615010356 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:21.726+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenc0:4e:bd:51:d0:75:ef:9b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokena7:ea:70:79:ce:7f:27:c9 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 3562146078059658886 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15281531523624343311 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 9584066495338256038 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:21.726+05:30 [Info] RebalanceServiceManager::cleanupShardTokenForDest Fetching transfer tokens again from metaKV before attempting to drop token on destination 2023-10-21T04:48:24.709+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:48:24.718+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:24.718+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:24.720+05:30 [Info] switched currmeta from 333 -> 333 force true 2023-10-21T04:48:25.076+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:25.106+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:26.114+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:26.731+05:30 [Info] RebalanceServiceManager::updateShardTokenMap Found 3 tokens. Cleaning up. 2023-10-21T04:48:26.731+05:30 [Info] RebalanceServiceManager::cleanupShardTokenForDest Cleaning up token: ShardTokenc0:4e:bd:51:d0:75:ef:9b on dest as ShardTokenDropOnSource is not posted for this token 2023-10-21T04:48:26.731+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenc0:4e:bd:51:d0:75:ef:9b, dropIndexes: true 2023-10-21T04:48:26.731+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 17703248484023971553 instance id 3562146078059658886 real instance id 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:26.731+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 17703248484023971553 instance 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:26.731+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 17703248484023971553 instance id 12396348969790516872 2023-10-21T04:48:26.732+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 17703248484023971553 2023-10-21T04:48:26.738+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12396348969790516872 &{1} 2023-10-21T04:48:26.738+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12396348969790516872 2023-10-21T04:48:26.738+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 17703248484023971553, instId: 12396348969790516872 2023-10-21T04:48:26.741+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12396348969790516872] 2023-10-21T04:48:26.741+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12396348969790516872] 2023-10-21T04:48:26.742+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:26.742+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12396348969790516872] 2023-10-21T04:48:26.744+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.744+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.744+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.744+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:26.744+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:48:26.745+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 6.389528ms 2023-10-21T04:48:26.745+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 6.432515ms 2023-10-21T04:48:26.745+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex stopped 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.744+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.745+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12396348969790516872 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.745+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:48:26.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex stopped 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:26.754+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:26.754+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:48:26.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex stopped 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.754+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:26.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex] 2023-10-21T04:48:26.744+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex closed 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:26.745+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.756+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:26.756+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:48:26.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex stopped 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.756+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:26.757+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:26.757+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:48:26.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex stopped 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.757+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.744+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12396348969790516872, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.759+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.759+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:26.759+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:26.760+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:48:26.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex stopped 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.760+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:26.760+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:48:26.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex stopped 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.760+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:26.760+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:48:26.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex stopped 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.760+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.761+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:26.761+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:26.761+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:26.761+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:48:26.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex stopped 2023-10-21T04:48:26.761+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.761+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.761+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.761+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.777+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 45.736613ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:26.781+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex] 2023-10-21T04:48:26.795+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex closed 2023-10-21T04:48:26.795+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.795+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.796+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.793+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex] 2023-10-21T04:48:26.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex closed 2023-10-21T04:48:26.801+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.801+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:26.802+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:26.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:48:26.805+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex ... 2023-10-21T04:48:26.807+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex] 2023-10-21T04:48:26.808+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex closed 2023-10-21T04:48:26.808+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.812+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.813+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:26.810+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 15281531523624343311 &{1} 2023-10-21T04:48:26.813+05:30 [Info] Indexer::handleDropIndex - IndexInstId 15281531523624343311 2023-10-21T04:48:26.813+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 2608585389864151823, instId: 15281531523624343311 2023-10-21T04:48:26.813+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:48:26.813+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:26.813+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 15281531523624343311 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:48:26.814+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [15281531523624343311] 2023-10-21T04:48:26.814+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [15281531523624343311] 2023-10-21T04:48:26.814+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:26.814+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [15281531523624343311] 2023-10-21T04:48:26.814+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:26.814+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 15281531523624343311 Defn: DefnId: 2608585389864151823 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:26.815+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.657599ms 2023-10-21T04:48:26.815+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 15281531523624343311, PartitionId 0, IndexDefnId 2608585389864151823. 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex] 2023-10-21T04:48:26.815+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex] 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex] 2023-10-21T04:48:26.815+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex 2023-10-21T04:48:26.815+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex stopped 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.815+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.815+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 15281531523624343311 2023-10-21T04:48:26.816+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex] 2023-10-21T04:48:26.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex closed 2023-10-21T04:48:26.816+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.816+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.816+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.816+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:26.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex sucessfully destroyed 2023-10-21T04:48:26.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:48:26.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex ... 2023-10-21T04:48:26.818+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:26.818+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:26.818+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:26.818+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:48:26.818+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex stopped 2023-10-21T04:48:26.818+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.818+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.818+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.818+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.818+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:26.819+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:48:26.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex stopped 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.819+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:26.819+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:48:26.819+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex stopped 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.819+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.826+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.827+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [15281531523624343311] 2023-10-21T04:48:26.845+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.845+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex] 2023-10-21T04:48:26.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex closed 2023-10-21T04:48:26.846+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.829+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:26.830+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.839+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.842+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.844+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index destroyed 2023-10-21T04:48:26.848+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.848+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 5 Destroy Done 2023-10-21T04:48:26.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex] 2023-10-21T04:48:26.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex closed 2023-10-21T04:48:26.848+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.849+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.849+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.849+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.849+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:26.849+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:26.849+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:26.849+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:26.851+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:48:26.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex] 2023-10-21T04:48:26.851+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 3 Close Done 2023-10-21T04:48:26.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:26.848+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:48:26.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex stopped 2023-10-21T04:48:26.855+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.856+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.856+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.856+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:26.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:48:26.856+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex ... 2023-10-21T04:48:26.857+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.857+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73679 2023-10-21T04:48:26.857+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 42.488366ms 2023-10-21T04:48:26.857+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 44.217936ms 2023-10-21T04:48:26.855+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex closed 2023-10-21T04:48:26.858+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.858+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.858+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.859+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:26.859+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex] 2023-10-21T04:48:26.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex closed 2023-10-21T04:48:26.861+05:30 [Info] switched currmeta from 333 -> 336 force true 2023-10-21T04:48:26.861+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.863+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.863+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.863+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:26.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex] 2023-10-21T04:48:26.863+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex closed 2023-10-21T04:48:26.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex sucessfully destroyed 2023-10-21T04:48:26.864+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.865+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.865+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.867+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex] 2023-10-21T04:48:26.864+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex] 2023-10-21T04:48:26.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex closed 2023-10-21T04:48:26.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:26.877+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:26.877+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:26.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.881+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.881+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:48:26.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex ... 2023-10-21T04:48:26.868+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex closed 2023-10-21T04:48:26.883+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.883+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:26.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:48:26.884+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex ... 2023-10-21T04:48:26.881+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex] 2023-10-21T04:48:26.885+05:30 [Info] switched currmeta from 336 -> 336 force true 2023-10-21T04:48:26.886+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 97.59447ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:26.911+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 4390029557575378783 instance id 9584066495338256038 real instance id 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:26.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.903+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.908+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.909+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.912+05:30 [Info] LifecycleMgr.PruneIndexPartition() : index defnId 4390029557575378783 instance 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:48:26.913+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : index defnId 4390029557575378783 instance id 4397940545762329850 2023-10-21T04:48:26.913+05:30 [Info] LifecycleMgr.DeleteIndexInstance() : there is only a single instance. Delete index 4390029557575378783 2023-10-21T04:48:26.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex sucessfully destroyed 2023-10-21T04:48:26.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index destroyed 2023-10-21T04:48:26.914+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:26.914+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:26.915+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:26.915+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:48:26.914+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:48:26.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex ... 2023-10-21T04:48:26.915+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.916+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 3 Destroy Done 2023-10-21T04:48:26.915+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:26.915+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex stopped 2023-10-21T04:48:26.917+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.918+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.917+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:26.918+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.919+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.919+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:26.920+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:48:26.920+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:26.920+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:26.920+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:26.920+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:48:26.920+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex 2023-10-21T04:48:26.920+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex] 2023-10-21T04:48:26.920+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex] 2023-10-21T04:48:26.920+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex 2023-10-21T04:48:26.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex] 2023-10-21T04:48:26.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex closed 2023-10-21T04:48:26.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex] 2023-10-21T04:48:26.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex closed 2023-10-21T04:48:26.921+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.921+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:26.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:48:26.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex ... 2023-10-21T04:48:26.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex stopped 2023-10-21T04:48:26.921+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.921+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.921+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.921+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.921+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex stopped 2023-10-21T04:48:26.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex stopped 2023-10-21T04:48:26.922+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.922+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.922+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.922+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.922+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:26.922+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:26.923+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:26.923+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:26.922+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index destroyed 2023-10-21T04:48:26.925+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.928+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:26.928+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex sucessfully destroyed 2023-10-21T04:48:26.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:48:26.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex ... 2023-10-21T04:48:26.930+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.930+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 1 Destroy Done 2023-10-21T04:48:26.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:48:26.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex ... 2023-10-21T04:48:26.934+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index destroyed 2023-10-21T04:48:26.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex sucessfully destroyed 2023-10-21T04:48:26.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex] 2023-10-21T04:48:26.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex closed 2023-10-21T04:48:26.950+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.950+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:26.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:48:26.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex ... 2023-10-21T04:48:26.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:48:26.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex ... 2023-10-21T04:48:26.952+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4397940545762329850 &{1} 2023-10-21T04:48:26.953+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4397940545762329850 2023-10-21T04:48:26.954+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 4390029557575378783, instId: 4397940545762329850 2023-10-21T04:48:26.954+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:48:26.955+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:26.956+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4397940545762329850 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:48:26.956+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4397940545762329850] 2023-10-21T04:48:26.956+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4397940545762329850] 2023-10-21T04:48:26.957+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:26.957+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4397940545762329850] 2023-10-21T04:48:26.952+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:48:26.958+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 2 Destroy Done 2023-10-21T04:48:26.958+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:48:26.958+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:48:26.958+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:48:26.958+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.958+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:48:26.958+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.958+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:48:26.959+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:26.959+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:26.959+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.959+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 5.770003ms 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.959+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.959+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4397940545762329850, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:48:26.959+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4397940545762329850 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:26.959+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:26.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:48:26.959+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:48:26.960+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [4397940545762329850] 2023-10-21T04:48:26.963+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:48:26.964+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73679 2023-10-21T04:48:26.964+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.886874ms 2023-10-21T04:48:26.993+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 39.583166ms 2023-10-21T04:48:26.972+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:26.994+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:26.984+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:26.985+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:26.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex sucessfully destroyed 2023-10-21T04:48:26.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex] 2023-10-21T04:48:26.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex closed 2023-10-21T04:48:26.996+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:26.999+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:26.999+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:48:26.993+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index destroyed 2023-10-21T04:48:27.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:48:27.010+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:48:27.010+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:48:27.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:48:27.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex stopped 2023-10-21T04:48:27.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.010+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.011+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:48:27.011+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:48:27.011+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:48:27.011+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:48:27.011+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex stopped 2023-10-21T04:48:27.015+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.015+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.015+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.016+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.016+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:48:27.016+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:48:27.016+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:48:27.016+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:48:27.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex stopped 2023-10-21T04:48:27.017+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.017+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.017+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.012+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:48:27.012+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.019+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:48:27.021+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex ... 2023-10-21T04:48:27.023+05:30 [Info] switched currmeta from 336 -> 338 force true 2023-10-21T04:48:27.023+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.024+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 8 Destroy Done 2023-10-21T04:48:27.025+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:48:27.025+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:48:27.026+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:48:27.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex stopped 2023-10-21T04:48:27.026+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.026+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.026+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.026+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.031+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:48:27.031+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[6 7 8 1 2 3 4 5]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex stopped 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex stopped 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex stopped 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:48:27.032+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:48:27.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex stopped 2023-10-21T04:48:27.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:48:27.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex ... 2023-10-21T04:48:27.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex] 2023-10-21T04:48:27.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex closed 2023-10-21T04:48:27.034+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.035+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:48:27.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex ... 2023-10-21T04:48:27.038+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index destroyed 2023-10-21T04:48:27.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex] 2023-10-21T04:48:27.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex closed 2023-10-21T04:48:27.041+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.041+05:30 [Info] Indexer::cleanupIndexData IndexInst 15281531523624343311 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex ... 2023-10-21T04:48:27.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 23 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex ... 2023-10-21T04:48:27.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex] 2023-10-21T04:48:27.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex closed 2023-10-21T04:48:27.042+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:48:27.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:48:27.042+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex ... 2023-10-21T04:48:27.042+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 4 Destroy Done 2023-10-21T04:48:27.043+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.042+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.043+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.043+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:48:27.048+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.051+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.051+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.052+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:48:27.069+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:48:27.052+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 141.565388ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:27.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex sucessfully destroyed 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex] 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex closed 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex] 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex closed 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex] 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex closed 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex] 2023-10-21T04:48:27.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex closed 2023-10-21T04:48:27.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex ... 2023-10-21T04:48:27.071+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex ... 2023-10-21T04:48:27.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:27.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:27.071+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:48:27.072+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:48:27.072+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex stopped 2023-10-21T04:48:27.072+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.073+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.073+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.075+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index destroyed 2023-10-21T04:48:27.077+05:30 [Info] switched currmeta from 338 -> 339 force true 2023-10-21T04:48:27.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.077+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.077+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.078+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.078+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:48:27.078+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:48:27.078+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:48:27.078+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:48:27.079+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:48:27.077+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:48:27.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.079+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.080+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.080+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.080+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:48:27.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.080+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.081+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.081+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.081+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:48:27.078+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.082+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex stopped 2023-10-21T04:48:27.082+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.082+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.083+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.083+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.083+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_15281531523624343311_0.index destroyed 2023-10-21T04:48:27.084+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 15281531523624343311, PartitionId 0, IndexDefnId 2608585389864151823. 2023-10-21T04:48:27.084+05:30 [Info] Indexer::cleanupIndexData IndexInst 15281531523624343311 Partition 0 Destroy Done 2023-10-21T04:48:27.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:48:27.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex ... 2023-10-21T04:48:27.083+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 7 Destroy Done 2023-10-21T04:48:27.083+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.085+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.085+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:48:27.083+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:48:27.085+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:48:27.086+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:48:27.086+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:48:27.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex stopped 2023-10-21T04:48:27.086+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.086+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.086+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.086+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.087+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:48:27.087+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:48:27.087+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:48:27.087+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:48:27.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex stopped 2023-10-21T04:48:27.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex] 2023-10-21T04:48:27.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex closed 2023-10-21T04:48:27.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.088+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex ... 2023-10-21T04:48:27.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex ... 2023-10-21T04:48:27.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.089+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index destroyed 2023-10-21T04:48:27.089+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12396348969790516872, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:48:27.089+05:30 [Info] Indexer::cleanupIndexData IndexInst 12396348969790516872 Partition 6 Destroy Done 2023-10-21T04:48:27.093+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex] 2023-10-21T04:48:27.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex closed 2023-10-21T04:48:27.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.094+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:48:27.096+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:27.096+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:27.097+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.100+05:30 [Info] switched currmeta from 339 -> 339 force true 2023-10-21T04:48:27.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex ... 2023-10-21T04:48:27.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex ... 2023-10-21T04:48:27.101+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex] 2023-10-21T04:48:27.102+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex closed 2023-10-21T04:48:27.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.102+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex] 2023-10-21T04:48:27.108+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex closed 2023-10-21T04:48:27.103+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 10158339249566909515 &{1} 2023-10-21T04:48:27.108+05:30 [Info] Indexer::handleDropIndex - IndexInstId 10158339249566909515 2023-10-21T04:48:27.109+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 5277316367626246544, instId: 10158339249566909515 2023-10-21T04:48:27.109+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:48:27.110+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:27.110+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 10158339249566909515 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:48:27.110+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [10158339249566909515] 2023-10-21T04:48:27.110+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [10158339249566909515] 2023-10-21T04:48:27.110+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:48:27.111+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [10158339249566909515] 2023-10-21T04:48:27.111+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:48:27.111+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:48:27.111+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:27.112+05:30 [Info] MutationStreamReader:Shutdown StreamReader MAINT_STREAM 2023-10-21T04:48:27.112+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:48:27.113+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:48:27.113+05:30 [Error] DATP[->dataport ":9123"] Accept() Error: accept tcp4 0.0.0.0:9123: use of closed network connection 2023-10-21T04:48:27.108+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.113+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9123->127.0.0.1:56492: use of closed network connection 2023-10-21T04:48:27.115+05:30 [Error] DATP[->dataport ":9123"] worker "127.0.0.1:56492" exit: read tcp4 127.0.0.1:9123->127.0.0.1:56492: use of closed network connection 2023-10-21T04:48:27.108+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:48:27.113+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:56492" closed ! 2023-10-21T04:48:27.114+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 10158339249566909515, PartitionId 0, IndexDefnId 5277316367626246544. 2023-10-21T04:48:27.115+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index destroyed 2023-10-21T04:48:27.116+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.116+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.116+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:48:27.116+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.116+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:48:27.117+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:48:27.117+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:48:27.116+05:30 [Info] DATP[->dataport ":9123"] ... stopped 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.117+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex] 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex closed 2023-10-21T04:48:27.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex] 2023-10-21T04:48:27.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex ... 2023-10-21T04:48:27.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex ... 2023-10-21T04:48:27.117+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex] 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex] 2023-10-21T04:48:27.117+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:48:27.117+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex stopped 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.117+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.117+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:27.117+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:27.118+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:48:27.116+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 1 Destroy Done 2023-10-21T04:48:27.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex stopped 2023-10-21T04:48:27.118+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.119+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex] 2023-10-21T04:48:27.119+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex closed 2023-10-21T04:48:27.118+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex stopped 2023-10-21T04:48:27.118+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:48:27.119+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.119+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.119+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex] 2023-10-21T04:48:27.120+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:48:27.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.120+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 4 Close Done 2023-10-21T04:48:27.121+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 5 Close Done 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex closed 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:48:27.122+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:48:27.122+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex] 2023-10-21T04:48:27.123+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex closed 2023-10-21T04:48:27.123+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.123+05:30 [Info] Indexer::cleanupIndexData IndexInst 10158339249566909515 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.123+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:48:27.123+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 7 New 0 2023-10-21T04:48:27.122+05:30 [Info] switched currmeta from 339 -> 340 force true 2023-10-21T04:48:27.122+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:27.123+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.123+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.128+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.131+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.124+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.125+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex ... 2023-10-21T04:48:27.132+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:48:27.134+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:48:27.131+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 22.530554ms 2023-10-21T04:48:27.131+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 10158339249566909515 2023-10-21T04:48:27.132+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2] 2023-10-21T04:48:27.132+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex ... 2023-10-21T04:48:27.134+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:48:27.135+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:48:27.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex stopped 2023-10-21T04:48:27.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex ... 2023-10-21T04:48:27.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex ... 2023-10-21T04:48:27.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.145+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.145+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:48:27.145+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:48:27.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.145+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index destroyed 2023-10-21T04:48:27.147+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.147+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 2 Destroy Done 2023-10-21T04:48:27.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex ... 2023-10-21T04:48:27.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex ... 2023-10-21T04:48:27.148+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.152+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2]. Err genServer.closed 2023-10-21T04:48:27.152+05:30 [Error] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:48:27.152+05:30 [Info] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Treating genServer.closed As Success 2023-10-21T04:48:27.152+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73679 2023-10-21T04:48:27.152+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 21.29779ms 2023-10-21T04:48:27.152+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 43.900194ms 2023-10-21T04:48:27.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index destroyed 2023-10-21T04:48:27.156+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.156+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 4 Destroy Done 2023-10-21T04:48:27.169+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:48:27.170+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:48:27.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex] 2023-10-21T04:48:27.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex closed 2023-10-21T04:48:27.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex] 2023-10-21T04:48:27.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex closed 2023-10-21T04:48:27.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex ... 2023-10-21T04:48:27.172+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex ... 2023-10-21T04:48:27.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex stopped 2023-10-21T04:48:27.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:48:27.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:48:27.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:48:27.174+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:48:27.174+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.174+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.173+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.176+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 3 Close Done 2023-10-21T04:48:27.176+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:48:27.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex ... 2023-10-21T04:48:27.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex] 2023-10-21T04:48:27.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex closed 2023-10-21T04:48:27.180+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:27.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex] 2023-10-21T04:48:27.180+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.180+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.180+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:27.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex closed 2023-10-21T04:48:27.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex ... 2023-10-21T04:48:27.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex ... 2023-10-21T04:48:27.181+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#12396348969790516872:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:48:27.182+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:48:27.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index destroyed 2023-10-21T04:48:27.186+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 10158339249566909515, PartitionId 0, IndexDefnId 5277316367626246544. 2023-10-21T04:48:27.186+05:30 [Info] Indexer::cleanupIndexData IndexInst 10158339249566909515 Partition 0 Destroy Done 2023-10-21T04:48:27.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex ... 2023-10-21T04:48:27.186+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex ... 2023-10-21T04:48:27.188+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 102.622678ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:27.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index destroyed 2023-10-21T04:48:27.201+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [18374611443167983589 6455624821495495854] 2023-10-21T04:48:27.202+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.202+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 5 Destroy Done 2023-10-21T04:48:27.206+05:30 [Info] switched currmeta from 340 -> 341 force true 2023-10-21T04:48:27.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex ... 2023-10-21T04:48:27.211+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex ... 2023-10-21T04:48:27.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex ... 2023-10-21T04:48:27.212+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex ... 2023-10-21T04:48:27.218+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:27.218+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:27.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex ... 2023-10-21T04:48:27.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex ... 2023-10-21T04:48:27.221+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index destroyed 2023-10-21T04:48:27.223+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.223+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 7 Destroy Done 2023-10-21T04:48:27.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex ... 2023-10-21T04:48:27.223+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex ... 2023-10-21T04:48:27.225+05:30 [Info] switched currmeta from 341 -> 341 force true 2023-10-21T04:48:27.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.227+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index destroyed 2023-10-21T04:48:27.227+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.227+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 6 Destroy Done 2023-10-21T04:48:27.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex ... 2023-10-21T04:48:27.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex ... 2023-10-21T04:48:27.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex ... 2023-10-21T04:48:27.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex ... 2023-10-21T04:48:27.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex sucessfully destroyed 2023-10-21T04:48:27.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index destroyed 2023-10-21T04:48:27.251+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.251+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 3 Destroy Done 2023-10-21T04:48:27.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex ... 2023-10-21T04:48:27.252+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex ... 2023-10-21T04:48:27.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:48:27.256+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index destroyed 2023-10-21T04:48:27.256+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4397940545762329850, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:48:27.256+05:30 [Info] Indexer::cleanupIndexData IndexInst 4397940545762329850 Partition 8 Destroy Done 2023-10-21T04:48:27.468+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:48:27.468+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:48:27.468+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:48:27.468+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:48:27.468+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 {Index inst : [partitions]}: map[4397940545762329850:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 1697843682957894080 2023-10-21T04:48:27.469+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0] 15281531523624343311:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 {Index inst : [partitions]}: map[4397940545762329850:[1 2 3 4 5 6 7 8]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 1697843682957894080 2023-10-21T04:48:27.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 18374611443167983589 2023-10-21T04:48:27.693+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 6455624821495495854 2023-10-21T04:48:27.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 6455624821495495854 2023-10-21T04:48:27.693+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 18374611443167983589 2023-10-21T04:48:27.693+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [18374611443167983589 6455624821495495854], elapsed: 492.243762ms 2023-10-21T04:48:27.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:27.694+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:48:27.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper stopped 2023-10-21T04:48:27.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': stopped 2023-10-21T04:48:27.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:27.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:27.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:27.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:27.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:48:27.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:27.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:48:27.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:48:27.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:27.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:27.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:27.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:27.696+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:48:27.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:48:27.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:27.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:27.698+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:27.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:48:27.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shutdown completed 2023-10-21T04:48:27.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:27.698+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:48:27.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:48:27.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:48:27.698+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance destroyed 2023-10-21T04:48:27.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:48:27.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:27.700+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:48:27.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:27.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:27.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:48:27.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:48:27.700+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:48:27.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:27.700+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:48:27.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:48:27.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:27.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:27.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:27.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:48:27.702+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:48:27.704+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:48:27.704+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:48:27.704+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:48:27.704+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:48:27.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:48:27.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shutdown completed 2023-10-21T04:48:27.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:48:27.704+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:48:27.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:48:27.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:48:27.704+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance destroyed 2023-10-21T04:48:27.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:48:27.706+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Cleaned up Tenant with name:bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:48:27.706+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [18374611443167983589 6455624821495495854], elapsed(sec): 0.50475329 2023-10-21T04:48:27.709+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenf5:68:e7:32:31:80:46:d9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 85664c0432a8df82b3d3efd913473cb1 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken3a:41:36:5d:f0:c3:a0:ee Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16515427720591431191 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 2034303749683961252 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 52515533635989429 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:48:27.727+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokenf5:68:e7:32:31:80:46:d9 2023-10-21T04:48:27.727+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: 85664c0432a8df82b3d3efd913473cb1 TransferTokenId: ShardTokenf5:68:e7:32:31:80:46:d9 Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:48:27.727+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.00012729 2023-10-21T04:48:27.727+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:48:27.727+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:48:27.727+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.804µs 2023-10-21T04:48:27.727+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenf5_68_e7_32_31_80_46_d9, shardId :0 2023-10-21T04:48:27.727+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/85664c0432a8df82b3d3efd913473cb1_ShardTokenf5_68_e7_32_31_80_46_d9, shardId :0 2023-10-21T04:48:27.727+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:48:27.727+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:48:27.727+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:48:27.730+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:48:27.730+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:48:27.730+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:48:27.731+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:48:27.731+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 28.658µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:27.732+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:48:27.732+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:48:27.732+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:48:27.732+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:48:27.732+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:48:27.732+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:48:27.733+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:48:27.733+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 34.258µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:27.735+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:48:27.735+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:48:27.735+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 7.624418343s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:48:27.735+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/85664c0432a8df82b3d3efd913473cb1, rev: [] call. err: 2023-10-21T04:48:27.735+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 18] call. taskList: &{Rev:[0 0 0 0 0 0 0 19] Tasks:[]} 2023-10-21T04:48:27.736+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 19] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:48:27.739+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:48:27.739+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 19] Tasks:[]} 2023-10-21T04:48:27.740+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:48:27.740+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 19] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:48:27.742+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 19] 2023-10-21T04:48:27.742+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 19] 2023-10-21T04:48:28.149+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.149+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.154+05:30 [Info] switched currmeta from 341 -> 344 force true 2023-10-21T04:48:28.162+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.162+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.166+05:30 [Info] switched currmeta from 344 -> 344 force true 2023-10-21T04:48:28.208+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.208+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.211+05:30 [Info] switched currmeta from 344 -> 344 force true 2023-10-21T04:48:28.220+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:28.261+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.261+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.264+05:30 [Info] switched currmeta from 344 -> 345 force true 2023-10-21T04:48:28.320+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.320+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.324+05:30 [Info] switched currmeta from 345 -> 346 force true 2023-10-21T04:48:28.365+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14845378835615010356:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.365+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14845378835615010356:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.365+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14845378835615010356:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.365+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14845378835615010356:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.365+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12606056968936758026:[8 1 2 3 4 5 6 7] 14845378835615010356:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.488+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.488+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.494+05:30 [Info] switched currmeta from 346 -> 346 force true 2023-10-21T04:48:28.517+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.517+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.521+05:30 [Info] switched currmeta from 346 -> 347 force true 2023-10-21T04:48:28.539+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.539+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.542+05:30 [Info] switched currmeta from 347 -> 348 force true 2023-10-21T04:48:28.596+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.599+05:30 [Info] switched currmeta from 348 -> 348 force true 2023-10-21T04:48:28.635+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:28.635+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:28.639+05:30 [Info] switched currmeta from 348 -> 349 force true 2023-10-21T04:48:28.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:28.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:48:29.231+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:29.720+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:48:29.725+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:29.725+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:29.728+05:30 [Info] switched currmeta from 349 -> 349 force true 2023-10-21T04:48:30.237+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:30.408+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:60520 2023-10-21T04:48:30.427+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 16.430977ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:48:30.450+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:60520. Reason = EOF. Cleaning Pipe. 2023-10-21T04:48:30.450+05:30 [Info] messageListener.start(): message channel closed. Remove peer 8b:56:57:8f:c8:73:43:1f as follower. 2023-10-21T04:48:30.928+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:48:31.243+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:48:31.732+05:30 [Info] memstats {"Alloc":132401800, "TotalAlloc":601643064, "Sys":227236152, "Lookups":0, "Mallocs":4701574,"Frees":4385761, "HeapAlloc":132401800, "HeapSys":206012416, "HeapIdle":62513152, "HeapInuse":143499264,"HeapReleased":3145728, "HeapObjects":315813,"MSpanInuse":793120, "MSpanSys": 1354560, "StackInuse": 3702784,"GCSys":13499128, "LastGC":1697843820424888565,"PauseTotalNs":3900263, "PauseNs":[82518], "NumGC":11} 2023-10-21T04:48:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:48:32.787+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:48:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:48:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:48:33.184+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:48:33.204+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:48:33.479+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:48:33.514+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:48:33.752+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:48:34.310+05:30 [Info] janitor: running cleanup. 2023-10-21T04:48:34.310+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[18046371353728341033 5465835793930010953 14292583894719865678 7392796230534034390 17797267771846799851 12582561312368048736 12027054384338714804 1301336828089581719 10606939961365971246 4286621850367430051 16706481854123898870 7147407900589480727 12833670934079048728 9665410041638025825 5932387718257253322 2048006213723315482 4987105685507282285 14889401051926000176 11276338709617092948 3381524789655566900 4860549750398470299 1252359852432998131 14549775068266281786 249887442666643345 128675491261474353 11619165464560280276 4535197604682045916 8669649359784409226 7045704526128519604 3674078063353481907 7525085113233084253 18026660730857541767 4532813151040661705 8910459001587546151 8504686614261377811 16483001637761719171 17656144409930763141 8378417662949994532 13856704970496601958 18323460706027642575 13268712595133993155 16337157693634799600 6985049432225870291 15431181238515168303 16961575949607441659 10761885902580101111 15968368593314043432 15973521466438086342 12195715103909237684 4138047918192025493 12824340686309573428 8857269605176042330 12140458060295645238 14741949230702299196 15945117906390072636 16162012140118277065 11365633301282701856 12338036531800130039 17476654447546601131 4790214984019693894 573521267164877540 12599498416578472779 17917116660857103172 16632040625897953619]} 2023-10-21T04:48:35.390+05:30 [Info] Indexer::ReadMemstats Time Taken 164.58µs 2023-10-21T04:48:36.667+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9993824791482085, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:48:37.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9991564174511359, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:48:38.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8043528857069066, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:48:39.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.44781654638479196, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:48:44.776+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:44.776+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:44.822+05:30 [Info] switched currmeta from 349 -> 353 force true 2023-10-21T04:48:44.860+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:44.860+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:44.898+05:30 [Info] switched currmeta from 353 -> 353 force true 2023-10-21T04:48:45.088+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:45.088+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:45.100+05:30 [Info] switched currmeta from 353 -> 357 force true 2023-10-21T04:48:45.159+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:45.159+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:45.162+05:30 [Info] switched currmeta from 357 -> 357 force true 2023-10-21T04:48:49.846+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:48:49.862+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:48:49.862+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:48:49.867+05:30 [Info] switched currmeta from 357 -> 357 force true 2023-10-21T04:48:49.888+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.888+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.888+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.888+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.889+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.889+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.889+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.890+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.890+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:49.890+05:30 [Warn] pools::bucketFinalizer: Finalizing a bucket with active connections. 2023-10-21T04:48:57.742+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 19] call. err: operation_canceled 2023-10-21T04:48:57.742+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:48:57.742+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 19] Tasks:[]} 2023-10-21T04:48:57.746+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:48:57.746+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 19] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:48:57.748+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 19] 2023-10-21T04:48:57.748+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 19] 2023-10-21T04:49:03.756+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:49:05.389+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 16367703496921140088 2023-10-21T04:49:05.389+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 186.904µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:05.516+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:46238 2023-10-21T04:49:05.522+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:46238 2023-10-21T04:49:05.566+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:46238 2023-10-21T04:49:05.620+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 16367703496921140088 2023-10-21T04:49:05.620+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 234.166µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:05.658+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:05.658+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:05.674+05:30 [Info] switched currmeta from 357 -> 360 force true 2023-10-21T04:49:05.681+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:05.681+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:05.691+05:30 [Info] switched currmeta from 360 -> 361 force true 2023-10-21T04:49:05.711+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:05.711+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:05.724+05:30 [Info] switched currmeta from 361 -> 362 force true 2023-10-21T04:49:05.772+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:05.772+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:05.777+05:30 [Info] switched currmeta from 362 -> 364 force true 2023-10-21T04:49:05.782+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:05.782+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:05.784+05:30 [Info] switched currmeta from 364 -> 364 force true 2023-10-21T04:49:05.794+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:05.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:05.823+05:30 [Info] switched currmeta from 364 -> 366 force true 2023-10-21T04:49:05.917+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:05.917+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:05.926+05:30 [Info] switched currmeta from 366 -> 366 force true 2023-10-21T04:49:06.306+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33182 2023-10-21T04:49:06.329+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 18.55394ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:06.432+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 16367703496921140088. No update is performed. 2023-10-21T04:49:06.432+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 126.383µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:06.469+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33182. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:06.469+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5a:7a:c2:d1:86:81:2c:f1 as follower. 2023-10-21T04:49:06.521+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33202 2023-10-21T04:49:06.604+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 78.694037ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:06.653+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:49:06.653+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:49:06.672+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:49:06.704+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:49:06.704+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:49:06.704+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:49:06.745+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33220 2023-10-21T04:49:06.745+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:49:06.767+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33226 2023-10-21T04:49:06.785+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.468123ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:06.814+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 66.678204ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:06.815+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:49:06.815+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:49:06.815+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:49:06.831+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33202. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:06.831+05:30 [Info] messageListener.start(): message channel closed. Remove peer a8:42:b8:c7:f3:ea:6d:b8 as follower. 2023-10-21T04:49:06.839+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:06.850+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:49:06.931+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:49:06.931+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:49:06.931+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:49:06.945+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:49:06.959+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33226. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:06.960+05:30 [Info] messageListener.start(): message channel closed. Remove peer b6:9a:f:bd:88:6f:b:3b as follower. 2023-10-21T04:49:06.981+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:49:06.981+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:49:06.981+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:49:06.981+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:49:06.981+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:49:06.981+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:49:06.981+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:49:06.981+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:49:06.981+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:49:06.981+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:49:06.981+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:57844->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:06.982+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:52726->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:06.982+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:53156->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:06.982+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:33220->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:06.982+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33220. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:06.982+05:30 [Info] messageListener.start(): message channel closed. Remove peer 79:f9:97:2e:5b:b0:a2:23 as follower. 2023-10-21T04:49:07.105+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:07.286+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:07.315+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:07.570+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:07.570+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:07.574+05:30 [Info] switched currmeta from 366 -> 367 force true 2023-10-21T04:49:07.653+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33268 2023-10-21T04:49:07.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9815209451868991, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:49:07.677+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 18.761879ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:07.734+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33276 2023-10-21T04:49:07.761+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:07.761+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:07.763+05:30 [Info] switched currmeta from 367 -> 368 force true 2023-10-21T04:49:07.765+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 26.080265ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:07.850+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:07.851+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:07.854+05:30 [Info] switched currmeta from 368 -> 369 force true 2023-10-21T04:49:08.044+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:08.044+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:08.048+05:30 [Info] switched currmeta from 369 -> 370 force true 2023-10-21T04:49:08.178+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:08.266+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:08.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9937469616310265, throttleDelayMs (new, old, change): (7, 3, 4) 2023-10-21T04:49:08.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10558612751966480746:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:49:08.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10558612751966480746:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:49:08.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10558612751966480746:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:49:08.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10558612751966480746:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:49:08.862+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10558612751966480746:[0]]. New rollbacktime: 1697843354158819025, Old rollbacktime: 0 2023-10-21T04:49:08.970+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17085981505417849194:[0]]. New rollbacktime: 1697843354166979310, Old rollbacktime: 0 2023-10-21T04:49:09.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.99840203942828, throttleDelayMs (new, old, change): (10, 7, 3) 2023-10-21T04:49:10.076+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:10.152+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:10.152+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:10.171+05:30 [Info] switched currmeta from 370 -> 370 force true 2023-10-21T04:49:10.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9998050735847218 2023-10-21T04:49:11.383+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 6145407649125550089 2023-10-21T04:49:11.383+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 134.552µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:11.667+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9998174083069978 2023-10-21T04:49:11.888+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:46238 2023-10-21T04:49:11.940+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:46238 2023-10-21T04:49:12.161+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:46238 2023-10-21T04:49:12.574+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 6145407649125550089 2023-10-21T04:49:12.574+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 356.836µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:12.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:49:12.907+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:12.907+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:12.918+05:30 [Info] switched currmeta from 370 -> 379 force true 2023-10-21T04:49:13.112+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:13.112+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:13.163+05:30 [Info] switched currmeta from 379 -> 379 force true 2023-10-21T04:49:13.667+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:49:13.668+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:49:13.668+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:49:13.701+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:49:14.145+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:49:14.145+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:49:14.145+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:49:14.223+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33494 2023-10-21T04:49:14.224+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:49:14.341+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 114.79915ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:14.341+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:49:14.341+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:49:14.341+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:49:14.421+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:49:14.560+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:49:14.560+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:49:14.560+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:49:14.655+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:49:14.667+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9993955304738601 2023-10-21T04:49:14.696+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33512 2023-10-21T04:49:14.841+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:14.950+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 250.135123ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:15.055+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:15.078+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:15.078+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:15.089+05:30 [Info] switched currmeta from 379 -> 379 force true 2023-10-21T04:49:15.098+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 6145407649125550089. No update is performed. 2023-10-21T04:49:15.098+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 156.773µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:15.139+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33512. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:15.139+05:30 [Info] messageListener.start(): message channel closed. Remove peer dd:2c:63:f3:63:e4:2a:9 as follower. 2023-10-21T04:49:15.216+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33268. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:15.216+05:30 [Info] messageListener.start(): message channel closed. Remove peer 15:a6:5b:ed:89:30:79:7 as follower. 2023-10-21T04:49:15.271+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:49:15.271+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:49:15.271+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:49:15.271+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:49:15.271+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:49:15.271+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:49:15.271+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:49:15.271+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:49:15.271+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:49:15.271+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:49:15.271+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:52992->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:15.272+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:53418->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:15.272+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:33494->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:15.272+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:58102->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:49:15.272+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33494. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:15.272+05:30 [Info] messageListener.start(): message channel closed. Remove peer 8:b7:49:de:83:df:43:89 as follower. 2023-10-21T04:49:15.451+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:15.642+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33534 2023-10-21T04:49:15.660+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:33536 2023-10-21T04:49:15.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9993406587360655 2023-10-21T04:49:15.790+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 142.943917ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:15.824+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 155.796469ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:16.074+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33276. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:16.074+05:30 [Info] messageListener.start(): message channel closed. Remove peer f:48:89:43:9e:7d:ae:7f as follower. 2023-10-21T04:49:16.103+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33534. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:16.103+05:30 [Info] messageListener.start(): message channel closed. Remove peer 1b:78:21:da:7d:5c:20:1e as follower. 2023-10-21T04:49:16.194+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:33536. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:16.194+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4e:77:96:f5:2b:60:8b:5f as follower. 2023-10-21T04:49:16.291+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:16.326+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:16.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9564414481570348 2023-10-21T04:49:17.439+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:17.439+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:17.442+05:30 [Info] switched currmeta from 379 -> 380 force true 2023-10-21T04:49:17.563+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:17.563+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:17.567+05:30 [Info] switched currmeta from 380 -> 381 force true 2023-10-21T04:49:17.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8607884315372615, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:49:17.992+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:17.993+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:17.996+05:30 [Info] switched currmeta from 381 -> 382 force true 2023-10-21T04:49:18.057+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:18.057+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:18.061+05:30 [Info] switched currmeta from 382 -> 383 force true 2023-10-21T04:49:18.470+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16653501928284257012:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:49:18.470+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16653501928284257012:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:49:18.470+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16653501928284257012:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:49:18.470+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16653501928284257012:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:49:18.470+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16653501928284257012:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:49:18.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6861401752615773, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:49:18.864+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8497280533643391851:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:49:18.864+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8497280533643391851:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:49:18.864+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8497280533643391851:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:49:18.864+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8497280533643391851:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:49:18.864+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[8497280533643391851:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:49:19.948+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:19.953+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:19.953+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:19.955+05:30 [Info] switched currmeta from 383 -> 383 force true 2023-10-21T04:49:27.749+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:49:27.749+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 19] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:49:27.749+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 19] call. err: operation_canceled 2023-10-21T04:49:27.749+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:49:27.749+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 19] Tasks:[]} 2023-10-21T04:49:27.750+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 19] 2023-10-21T04:49:27.750+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 19] 2023-10-21T04:49:30.277+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:49:30.424+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:30.424+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:30.429+05:30 [Info] switched currmeta from 383 -> 383 force true 2023-10-21T04:49:30.633+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:49:30.669+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:49:30.669+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:49:30.712+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:30.800+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:30.800+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:30.850+05:30 [Info] switched currmeta from 383 -> 383 force true 2023-10-21T04:49:30.876+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:30.876+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:30.882+05:30 [Info] switched currmeta from 383 -> 383 force true 2023-10-21T04:49:31.733+05:30 [Info] memstats {"Alloc":197987112, "TotalAlloc":773238792, "Sys":240081208, "Lookups":0, "Mallocs":6079266,"Frees":5403289, "HeapAlloc":197987112, "HeapSys":218103808, "HeapIdle":15745024, "HeapInuse":202358784,"HeapReleased":917504, "HeapObjects":675977,"MSpanInuse":1347040, "MSpanSys": 1566720, "StackInuse": 4194304,"GCSys":13505272, "LastGC":1697843894611997667,"PauseTotalNs":4125123, "PauseNs":[224860], "NumGC":12} 2023-10-21T04:49:32.526+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {a98493b6c7598b784e28f4b16eb254f8 [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:49:32.529+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 13.246µs 2023-10-21T04:49:32.529+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:49:32.529+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:49:32.529+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:49:32.531+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:49:32.531+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:32.531+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 12.336µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:32.531+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:49:32.531+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:49:32.531+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 20] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:49:32.532+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 19] call. taskList: &{Rev:[0 0 0 0 0 0 0 20] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a98493b6c7598b784e28f4b16eb254f8 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a98493b6c7598b784e28f4b16eb254f8]}]} 2023-10-21T04:49:32.531+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 2.132858ms 2023-10-21T04:49:32.540+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 20] 2023-10-21T04:49:32.540+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 20] 2023-10-21T04:49:32.619+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {ba6304988a6335d65281381d6cfc5e32 a98493b6c7598b784e28f4b16eb254f8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:49:32.630+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {ba6304988a6335d65281381d6cfc5e32 a98493b6c7598b784e28f4b16eb254f8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:49:32.630+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"a98493b6c7598b784e28f4b16eb254f8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:49:32.634+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{ba6304988a6335d65281381d6cfc5e32 a98493b6c7598b784e28f4b16eb254f8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:49:32.634+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{ba6304988a6335d65281381d6cfc5e32 a98493b6c7598b784e28f4b16eb254f8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:49:32.635+05:30 [Info] ShardRebalancer::observeRebalance &{ba6304988a6335d65281381d6cfc5e32 a98493b6c7598b784e28f4b16eb254f8 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:49:32.691+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"a98493b6c7598b784e28f4b16eb254f8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:49:32.691+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:49:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:49:32.787+05:30 [Info] client stats current counts: current: 16, not current: 0 2023-10-21T04:49:32.805+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:49:32.812+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:49:32.857+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32826 2023-10-21T04:49:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:49:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:49:32.943+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbc:ff:3d:f:5b:9b:8b:6b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8497280533643391851 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1856681538226480532 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15262821106108281777 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.007+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.045+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbc:ff:3d:f:5b:9b:8b:6b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8497280533643391851 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1856681538226480532 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15262821106108281777 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.080+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb1:70:f2:cb:a5:ff:fd:1d MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16653501928284257012 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3266194727357921673 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4523737201364197125 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.096+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.120+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.154+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb1:70:f2:cb:a5:ff:fd:1d MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16653501928284257012 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3266194727357921673 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4523737201364197125 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.155+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:49:33.155+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:49:33.155+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:49:33.155+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 21.801µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:33.173+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb1:70:f2:cb:a5:ff:fd:1d MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16653501928284257012 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3266194727357921673 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4523737201364197125 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.206+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbc:ff:3d:f:5b:9b:8b:6b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8497280533643391851 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1856681538226480532 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15262821106108281777 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.237+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:49:33.250+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.251+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:49:33.251+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_1:REBALANCE_INITIATED]} 2023-10-21T04:49:33.251+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_1:REBALANCE_INITIATED] 2023-10-21T04:49:33.251+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.36µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:33.255+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:49:33.266+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.289+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.419+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"ba6304988a6335d65281381d6cfc5e32","RebalId":"a98493b6c7598b784e28f4b16eb254f8","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:49:33.419+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:49:33.419+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:49:33.419+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:49:33.419+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:49:33.419+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:49:33.419+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 13.601µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:33.422+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.452+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.527+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:49:33.533+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{ba6304988a6335d65281381d6cfc5e32 a98493b6c7598b784e28f4b16eb254f8 Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD}. 2023-10-21T04:49:33.561+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:49:33.609+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.609+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:49:33.609+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_1:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:49:33.610+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_1:REBALANCE_TRANSFER_DONE] 2023-10-21T04:49:33.610+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 25.301µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:33.610+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[5629943257627822009:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/5629943257627822009/shards/shard5629943257627822009 17327901024118572215:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/17327901024118572215/shards/shard17327901024118572215] a98493b6c7598b784e28f4b16eb254f8 ShardToken47:57:c0:39:1d:ee:e8:c9 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00487e600 0xc00487e660 map[] 0xc009ec22a0 0xc004d17800 false} 2023-10-21T04:49:33.610+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/5629943257627822009/shards/shard5629943257627822009 2023-10-21T04:49:33.611+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/17327901024118572215/shards/shard17327901024118572215 2023-10-21T04:49:33.626+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/17327901024118572215/shards/shard17327901024118572215/data: super block snapshot head offset :0 tail offset:1716224 sbgen :242 2023-10-21T04:49:33.626+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/17327901024118572215/shards/shard17327901024118572215/data/recovery: super block snapshot head offset :0 tail offset:1556480 sbgen :242 2023-10-21T04:49:33.628+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/5629943257627822009/shards/shard5629943257627822009/data: super block snapshot head offset :0 tail offset:1409024 sbgen :221 2023-10-21T04:49:33.628+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/5629943257627822009/shards/shard5629943257627822009/data/recovery: super block snapshot head offset :0 tail offset:1204224 sbgen :221 2023-10-21T04:49:33.637+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/17327901024118572215/shards/shard17327901024118572215: metadata saved successfully 2023-10-21T04:49:33.638+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/5629943257627822009/shards/shard5629943257627822009: metadata saved successfully 2023-10-21T04:49:33.638+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 (17327901024118572215) added 2023-10-21T04:49:33.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:49:33.638+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 117569536 totalBytes: 3291947 numShards: 2 duration: 28.39542ms 2023-10-21T04:49:33.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/17327901024118572215/shards/shard17327901024118572215, newshardId 17327901024118572215, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, shardsize 3291947, duration 27.140685ms 2023-10-21T04:49:33.638+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 17327901024118572215, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215, err: , elapsed(sec): 0.0286137 2023-10-21T04:49:33.638+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 17327901024118572215 bytesWritten: 3291947, totalBytes: 3291947, transferRate: 1.2192396296296297e+08 2023-10-21T04:49:33.639+05:30 [Info] tenant 'bucket_1_c25fee48adbe357b2d711be58a1deb67': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 (5629943257627822009) added 2023-10-21T04:49:33.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 from metadata completed, tenant :bucket_1_c25fee48adbe357b2d711be58a1deb67 2023-10-21T04:49:33.639+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardToken47_57_c0_39_1d_ee_e8_c9/5629943257627822009/shards/shard5629943257627822009, newshardId 5629943257627822009, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, shardsize 2632334, duration 29.002153ms 2023-10-21T04:49:33.639+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 5629943257627822009, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009, err: , elapsed(sec): 0.029315461 2023-10-21T04:49:33.639+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: a98493b6c7598b784e28f4b16eb254f8, transferId: ShardToken47:57:c0:39:1d:ee:e8:c9 2023-10-21T04:49:33.639+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: a98493b6c7598b784e28f4b16eb254f8, transferId: ShardToken47:57:c0:39:1d:ee:e8:c9, elapsed(sec): 0.000134887 2023-10-21T04:49:33.639+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[5629943257627822009: 17327901024118572215:], shardPaths: map[5629943257627822009:/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 17327901024118572215:/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.029519735 2023-10-21T04:49:33.639+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 5629943257627822009 bytesWritten: 2632334, totalBytes: 2632334, transferRate: 9.401192857142857e+07 2023-10-21T04:49:33.639+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [17327901024118572215 5629943257627822009], ttid: ShardToken47:57:c0:39:1d:ee:e8:c9, elapsed(sec): 0.029998354 2023-10-21T04:49:33.643+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:33.658+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.658+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken47:57:c0:39:1d:ee:e8:c9 2023-10-21T04:49:33.658+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[5629943257627822009:true 17327901024118572215:true] 2023-10-21T04:49:33.658+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.604µs 2023-10-21T04:49:33.658+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [17327901024118572215 5629943257627822009], isLockedForRecovery: true 2023-10-21T04:49:33.658+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [17327901024118572215 5629943257627822009], errMap: map[5629943257627822009: 17327901024118572215:], elapsed: 11.645µs 2023-10-21T04:49:33.659+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 16367703496921140088, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:49:33.659+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 16367703496921140088, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:49:33.694+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.732+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17085981505417849194,realInstId: 0, indexDefn DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:49:33.733+05:30 [Info] Indexer::handleRecoverIndex InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:49:33.733+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 373.863µs 2023-10-21T04:49:33.733+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 424.474µs 2023-10-21T04:49:33.734+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17085981505417849194 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:33.734+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17085981505417849194 PartitionId: 0 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:33.734+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': added 2023-10-21T04:49:33.734+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17085981505417849194, indexDefn DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:49:33.735+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 75.971662ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:33.740+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCleaner initialized 2023-10-21T04:49:33.741+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:33.741+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:33.741+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:33.747+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCleaner initialized for recovery 2023-10-21T04:49:33.749+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:33.749+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCleaner initialized 2023-10-21T04:49:33.750+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:33.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:33.750+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery 2023-10-21T04:49:33.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:33.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data], Shared [true] 2023-10-21T04:49:33.751+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1556480] Data log Head [0] Tail [1716224] 2023-10-21T04:49:33.752+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1556480] 2023-10-21T04:49:33.754+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.755+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex 2023-10-21T04:49:33.755+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swap is disabled 2023-10-21T04:49:33.756+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.756+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex 2023-10-21T04:49:33.756+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.755+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCleaner initialized for recovery 2023-10-21T04:49:33.756+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:33.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery 2023-10-21T04:49:33.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:33.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data], Shared [true] 2023-10-21T04:49:33.757+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1204224] Data log Head [0] Tail [1409024] 2023-10-21T04:49:33.756+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex 2023-10-21T04:49:33.758+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.758+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex 2023-10-21T04:49:33.757+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1204224] 2023-10-21T04:49:33.760+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:49:33.759+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.761+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex 2023-10-21T04:49:33.761+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.761+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex 2023-10-21T04:49:33.762+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.762+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex 2023-10-21T04:49:33.763+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.763+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex 2023-10-21T04:49:33.764+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.764+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex 2023-10-21T04:49:33.767+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.773+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:33.774+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.774+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:33.775+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.776+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:33.778+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.778+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:33.779+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.780+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:33.780+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.780+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:33.781+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.781+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:33.783+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.784+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:33.784+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.795+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.797+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:33.835+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:33.798+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb1:70:f2:cb:a5:ff:fd:1d MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16653501928284257012 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3266194727357921673 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4523737201364197125 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.836+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbc:ff:3d:f:5b:9b:8b:6b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8497280533643391851 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1856681538226480532 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15262821106108281777 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.831+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex 2023-10-21T04:49:33.845+05:30 [Info] switched currmeta from 383 -> 384 force true 2023-10-21T04:49:33.845+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex 2023-10-21T04:49:33.846+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : replayOffset: 1708032 (RP:1699840) (CP:1708032) 2023-10-21T04:49:33.846+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1708032] took [94.323703ms] 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916281 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916281 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916237 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916193 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916149 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916105 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1055776, resident 1, memSz 65733, memSzIndex 459, numRecords 1000 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916105 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916061 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28916017 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915973 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915973 controller mem 1662 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915973 controller mem 1662 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915973 controller mem 1662 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1055626, resident 1, memSz 1441, memSzIndex 0, numRecords 16 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915973 controller mem 1662 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915929 controller mem 1662 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915929 controller mem 1662 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915929 controller mem 1662 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 28915929 controller mem 1662 2023-10-21T04:49:33.847+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1716224] replayOffset [1708032] 2023-10-21T04:49:33.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [95.781951ms] 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.847+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:33.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data], Shared [true]. Built [18] plasmas, took [96.500084ms] 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1055776 memSz 1086 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 20 count: 1000 metaSz: 2507 rp [1] sn: 21 count: 1000 metaSz: 2517 2023-10-21T04:49:33.848+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 218525 recovery UsedSpace 22422 2023-10-21T04:49:33.884+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.884+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex 2023-10-21T04:49:33.888+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.888+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex 2023-10-21T04:49:33.889+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.889+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex 2023-10-21T04:49:33.891+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.892+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex 2023-10-21T04:49:33.892+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.892+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex 2023-10-21T04:49:33.894+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.895+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex 2023-10-21T04:49:33.896+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.896+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex 2023-10-21T04:49:33.902+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.902+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:33.909+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex 2023-10-21T04:49:33.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:33.911+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:33.911+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swapper started 2023-10-21T04:49:33.912+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.912+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex 2023-10-21T04:49:33.912+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex started 2023-10-21T04:49:33.912+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:33.912+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:33.913+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:33.913+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.913+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:33.914+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.915+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:33.921+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.921+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:33.921+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.922+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:33.922+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.925+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:33.925+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.925+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:33.927+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:33.947+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex 2023-10-21T04:49:33.948+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:33.949+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : replayOffset: 1400832 (RP:1384448) (CP:1400832) 2023-10-21T04:49:33.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1400832] took [192.275011ms] 2023-10-21T04:49:33.950+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.950+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651241 controller mem 4040 2023-10-21T04:49:33.951+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.951+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651241 controller mem 4040 2023-10-21T04:49:33.952+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.966+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651241 controller mem 4040 2023-10-21T04:49:33.974+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.975+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651241 controller mem 4040 2023-10-21T04:49:33.975+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.978+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651241 controller mem 4040 2023-10-21T04:49:33.979+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.980+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651197 controller mem 4040 2023-10-21T04:49:33.980+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:33.980+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651153 controller mem 4040 2023-10-21T04:49:33.981+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.027+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651153 controller mem 4040 2023-10-21T04:49:34.027+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.027+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651153 controller mem 4040 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651109 controller mem 4040 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651065 controller mem 4040 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651065 controller mem 4040 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1056176, resident 1, memSz 68673, memSzIndex 813, numRecords 1000 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651065 controller mem 4040 2023-10-21T04:49:34.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44651021 controller mem 4040 2023-10-21T04:49:34.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44650977 controller mem 4040 2023-10-21T04:49:34.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44650933 controller mem 4040 2023-10-21T04:49:34.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:34.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: controller ('bucket_1_c25fee48adbe357b2d711be58a1deb67') quota after header replay 44650933 controller mem 4040 2023-10-21T04:49:34.029+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1409024] replayOffset [1400832] 2023-10-21T04:49:34.087+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [329.19163ms] 2023-10-21T04:49:34.087+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.088+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.088+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.089+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.089+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.089+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.090+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.090+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.090+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.091+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.091+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.091+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.091+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.092+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.092+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.092+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.092+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:34.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data], Shared [true]. Built [17] plasmas, took [335.251873ms] 2023-10-21T04:49:34.093+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1056176 memSz 1866 2023-10-21T04:49:34.093+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 20 count: 1000 metaSz: 2507 rp [1] sn: 21 count: 1000 metaSz: 2517 2023-10-21T04:49:34.094+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 192635 recovery UsedSpace 24586 2023-10-21T04:49:34.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex 2023-10-21T04:49:34.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:34.139+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:34.139+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex 2023-10-21T04:49:34.140+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex started 2023-10-21T04:49:34.140+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.140+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.140+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17085981505417849194 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:49:34.215+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17085981505417849194 PartitionId 0 Warmup took 481.233642ms 2023-10-21T04:49:34.216+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17085981505417849194 partitionId 0 WriterThreads 4 2023-10-21T04:49:34.216+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17085981505417849194 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index Index: 17085981505417849194 Partition: 0 2023-10-21T04:49:34.216+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17085981505417849194, shardIdMap: map[0:[17327901024118572215 5629943257627822009]] err: 2023-10-21T04:49:34.217+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17085981505417849194, partnShardIdMap: map[0:[17327901024118572215 5629943257627822009]], err: 2023-10-21T04:49:34.217+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:34.218+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17085981505417849194 2023-10-21T04:49:34.219+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:49:34.225+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.905544ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:34.240+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:34.240+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:34.241+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbc:ff:3d:f:5b:9b:8b:6b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8497280533643391851 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1856681538226480532 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15262821106108281777 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:34.241+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:49:34.241+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:34.241+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:49:34.241+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:34.242+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17085981505417849194 Partitions [0] 2023-10-21T04:49:34.242+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 106.689µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:34.257+05:30 [Info] StorageMgr::openSnapshot IndexInst:17085981505417849194 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:49:34.257+05:30 [Info] switched currmeta from 384 -> 387 force true 2023-10-21T04:49:34.310+05:30 [Info] janitor: running cleanup. 2023-10-21T04:49:34.310+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[15945117906390072636 12599498416578472779 1252359852432998131 15431181238515168303 17797267771846799851 16483001637761719171 6985049432225870291 16337157693634799600 4860549750398470299 16162012140118277065 11619165464560280276 10606939961365971246 11276338709617092948 8669649359784409226 16632040625897953619 12824340686309573428 18046371353728341033 12027054384338714804 7045704526128519604 12140458060295645238 14741949230702299196 14889401051926000176 7147407900589480727 4790214984019693894 4138047918192025493 12195715103909237684 14549775068266281786 4987105685507282285 14292583894719865678 9665410041638025825 11365633301282701856 4532813151040661705 12833670934079048728 13268712595133993155 12582561312368048736 18026660730857541767 16961575949607441659 12338036531800130039 4841852412665776737 4286621850367430051 573521267164877540 1301336828089581719 3381524789655566900 2608585389864151823 17917116660857103172 2048006213723315482 7525085113233084253 8857269605176042330 8504686614261377811 18323460706027642575 16706481854123898870 3674078063353481907 15973521466438086342 128675491261474353 13856704970496601958 249887442666643345 17656144409930763141 8910459001587546151 4535197604682045916 15968368593314043432 8378417662949994532 5465835793930010953 7392796230534034390 10761885902580101111 5932387718257253322 17476654447546601131]} 2023-10-21T04:49:34.314+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenbc:ff:3d:f:5b:9b:8b:6b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8497280533643391851 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1856681538226480532 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15262821106108281777 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:34.376+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb1:70:f2:cb:a5:ff:fd:1d MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16653501928284257012 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3266194727357921673 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4523737201364197125 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:34.376+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:49:34.376+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:49:34.376+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:49:34.376+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 28.077µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:34.376+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[6455624821495495854:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/6455624821495495854/shards/shard6455624821495495854 18374611443167983589:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/18374611443167983589/shards/shard18374611443167983589] a98493b6c7598b784e28f4b16eb254f8 ShardTokenb1:70:f2:cb:a5:ff:fd:1d /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00487e600 0xc00487e660 map[] 0xc00a15e900 0xc006ab87e0 false} 2023-10-21T04:49:34.377+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/6455624821495495854/shards/shard6455624821495495854 2023-10-21T04:49:34.377+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/18374611443167983589/shards/shard18374611443167983589 2023-10-21T04:49:34.382+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/18374611443167983589/shards/shard18374611443167983589/data: super block snapshot head offset :0 tail offset:1724416 sbgen :240 2023-10-21T04:49:34.382+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/18374611443167983589/shards/shard18374611443167983589/data/recovery: super block snapshot head offset :0 tail offset:1548288 sbgen :240 2023-10-21T04:49:34.383+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/6455624821495495854/shards/shard6455624821495495854/data: super block snapshot head offset :0 tail offset:1396736 sbgen :219 2023-10-21T04:49:34.384+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/6455624821495495854/shards/shard6455624821495495854/data/recovery: super block snapshot head offset :0 tail offset:1179648 sbgen :219 2023-10-21T04:49:34.386+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/18374611443167983589/shards/shard18374611443167983589: metadata saved successfully 2023-10-21T04:49:34.386+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) added 2023-10-21T04:49:34.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:49:34.386+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/18374611443167983589/shards/shard18374611443167983589, newshardId 18374611443167983589, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, shardsize 3291988, duration 9.010196ms 2023-10-21T04:49:34.386+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 18374611443167983589, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589, err: , elapsed(sec): 0.010022928 2023-10-21T04:49:34.386+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 18374611443167983589 bytesWritten: 3291988, totalBytes: 3291988, transferRate: 3.657764444444444e+08 2023-10-21T04:49:34.388+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/6455624821495495854/shards/shard6455624821495495854: metadata saved successfully 2023-10-21T04:49:34.389+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) added 2023-10-21T04:49:34.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:49:34.390+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d/6455624821495495854/shards/shard6455624821495495854, newshardId 6455624821495495854, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, shardsize 2595505, duration 13.257072ms 2023-10-21T04:49:34.390+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 6455624821495495854, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854, err: , elapsed(sec): 0.01381145 2023-10-21T04:49:34.390+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 6455624821495495854 bytesWritten: 2595505, totalBytes: 2595505, transferRate: 1.9965423076923078e+08 2023-10-21T04:49:34.392+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: a98493b6c7598b784e28f4b16eb254f8, transferId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d 2023-10-21T04:49:34.392+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: a98493b6c7598b784e28f4b16eb254f8, transferId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d, elapsed(sec): 0.000295251 2023-10-21T04:49:34.392+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[6455624821495495854: 18374611443167983589:], shardPaths: map[6455624821495495854:/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 18374611443167983589:/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.015888769 2023-10-21T04:49:34.392+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [18374611443167983589 6455624821495495854], ttid: ShardTokenb1:70:f2:cb:a5:ff:fd:1d, elapsed(sec): 0.016508667 2023-10-21T04:49:34.406+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:34.424+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:34.424+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:34.431+05:30 [Info] switched currmeta from 387 -> 387 force true 2023-10-21T04:49:34.440+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenb1:70:f2:cb:a5:ff:fd:1d MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16653501928284257012 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3266194727357921673 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4523737201364197125 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:34.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9755069822889987, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:49:34.744+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[17085981505417849194:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:49:34.744+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 7790129686881525591, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:49:34.744+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 7790129686881525591, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:49:34.801+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 16759912838651900043,realInstId: 0, indexDefn DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:49:34.802+05:30 [Info] Indexer::handleRecoverIndex InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:49:34.802+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 337.946µs 2023-10-21T04:49:34.802+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 384.275µs 2023-10-21T04:49:34.802+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:34.802+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 16759912838651900043, indexDefn DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:49:34.802+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 7 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:34.802+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 58.089279ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:34.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:34.803+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.803+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 26 count: 0 metaSz: 2482 rp [1] sn: 27 count: 0 metaSz: 2483 rp [2] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:34.803+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 145256 2023-10-21T04:49:34.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:34.803+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.803+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 27 count: 0 metaSz: 2483 rp [1] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:34.803+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 153470 2023-10-21T04:49:34.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex 2023-10-21T04:49:34.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:34.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:34.809+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex 2023-10-21T04:49:34.809+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex started 2023-10-21T04:49:34.809+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.809+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex 2023-10-21T04:49:34.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:34.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:34.810+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex 2023-10-21T04:49:34.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex started 2023-10-21T04:49:34.810+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:49:34.811+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.811+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.840+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 7 Warmup took 37.547363ms 2023-10-21T04:49:34.840+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 7 WriterThreads 1 2023-10-21T04:49:34.840+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 7 Stopped Billing WUs 2023-10-21T04:49:34.840+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index Index: 16759912838651900043 Partition: 7 2023-10-21T04:49:34.841+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:34.841+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 8 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:34.841+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:34.841+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.841+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 27 count: 0 metaSz: 2483 rp [1] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:34.841+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 153470 2023-10-21T04:49:34.842+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:34.842+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.842+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 27 count: 0 metaSz: 2483 rp [1] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:34.842+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 153470 2023-10-21T04:49:34.851+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:34.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex 2023-10-21T04:49:34.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:34.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:34.851+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex 2023-10-21T04:49:34.852+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex started 2023-10-21T04:49:34.852+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.852+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex 2023-10-21T04:49:34.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:34.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:34.854+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex 2023-10-21T04:49:34.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex started 2023-10-21T04:49:34.854+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:49:34.855+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.855+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.878+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 8 Warmup took 37.375165ms 2023-10-21T04:49:34.879+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 8 WriterThreads 1 2023-10-21T04:49:34.879+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 8 Stopped Billing WUs 2023-10-21T04:49:34.879+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index Index: 16759912838651900043 Partition: 8 2023-10-21T04:49:34.879+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:34.879+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 1 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:34.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:34.880+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.880+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 27 count: 0 metaSz: 2483 rp [1] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:34.880+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 153470 2023-10-21T04:49:34.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:34.880+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.881+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 26 count: 0 metaSz: 2482 rp [1] sn: 27 count: 0 metaSz: 2483 rp [2] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:34.881+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 87900 recovery UsedSpace 145256 2023-10-21T04:49:34.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex 2023-10-21T04:49:34.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:34.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:34.887+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex 2023-10-21T04:49:34.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex started 2023-10-21T04:49:34.887+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.887+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex 2023-10-21T04:49:34.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:34.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:34.892+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex 2023-10-21T04:49:34.892+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex started 2023-10-21T04:49:34.892+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:49:34.892+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.892+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.923+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 1 Warmup took 43.68547ms 2023-10-21T04:49:34.923+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 1 WriterThreads 1 2023-10-21T04:49:34.923+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 1 Stopped Billing WUs 2023-10-21T04:49:34.923+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index Index: 16759912838651900043 Partition: 1 2023-10-21T04:49:34.923+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:34.924+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 2 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:34.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:34.924+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.924+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 25 count: 0 metaSz: 2482 rp [1] sn: 26 count: 0 metaSz: 2483 rp [2] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:49:34.924+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 145256 2023-10-21T04:49:34.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:34.927+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:34.931+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 26 count: 0 metaSz: 2483 rp [1] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:49:34.930+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex 2023-10-21T04:49:34.933+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:34.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:34.933+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 153470 2023-10-21T04:49:34.937+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex 2023-10-21T04:49:34.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex started 2023-10-21T04:49:34.943+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.943+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex 2023-10-21T04:49:34.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:34.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:34.957+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex 2023-10-21T04:49:34.957+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex started 2023-10-21T04:49:34.957+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:49:34.958+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:34.958+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:34.979+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:34.979+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:34.993+05:30 [Info] switched currmeta from 387 -> 388 force true 2023-10-21T04:49:35.014+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 2 Warmup took 90.67056ms 2023-10-21T04:49:35.016+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 2 WriterThreads 1 2023-10-21T04:49:35.017+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 2 Stopped Billing WUs 2023-10-21T04:49:35.019+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index Index: 16759912838651900043 Partition: 2 2023-10-21T04:49:35.020+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:35.021+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 3 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:35.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:35.023+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.023+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 27 count: 0 metaSz: 2483 rp [1] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:35.024+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 153470 2023-10-21T04:49:35.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:35.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 26 count: 0 metaSz: 2482 rp [1] sn: 27 count: 0 metaSz: 2483 rp [2] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:35.029+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 145256 2023-10-21T04:49:35.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex 2023-10-21T04:49:35.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:35.034+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:35.035+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex 2023-10-21T04:49:35.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex started 2023-10-21T04:49:35.035+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.035+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.044+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex 2023-10-21T04:49:35.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:35.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:35.045+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex 2023-10-21T04:49:35.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex started 2023-10-21T04:49:35.046+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.046+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.046+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:49:35.086+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 3 Warmup took 63.306183ms 2023-10-21T04:49:35.088+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 3 WriterThreads 1 2023-10-21T04:49:35.090+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 3 Stopped Billing WUs 2023-10-21T04:49:35.091+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index Index: 16759912838651900043 Partition: 3 2023-10-21T04:49:35.093+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:35.096+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 4 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:35.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:35.097+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.098+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 27 count: 0 metaSz: 2483 rp [1] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:35.098+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 153470 2023-10-21T04:49:35.103+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:35.103+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.103+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 26 count: 0 metaSz: 2482 rp [1] sn: 27 count: 0 metaSz: 2483 rp [2] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:35.103+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 145256 2023-10-21T04:49:35.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex 2023-10-21T04:49:35.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:35.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:35.107+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex 2023-10-21T04:49:35.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex started 2023-10-21T04:49:35.107+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.107+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex 2023-10-21T04:49:35.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:35.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:35.112+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex 2023-10-21T04:49:35.112+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex started 2023-10-21T04:49:35.112+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:49:35.115+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.115+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.153+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 4 Warmup took 56.425516ms 2023-10-21T04:49:35.154+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 4 WriterThreads 1 2023-10-21T04:49:35.154+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 4 Stopped Billing WUs 2023-10-21T04:49:35.154+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index Index: 16759912838651900043 Partition: 4 2023-10-21T04:49:35.154+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:35.155+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 5 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:35.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:35.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 26 count: 0 metaSz: 2482 rp [1] sn: 27 count: 0 metaSz: 2483 rp [2] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:35.156+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 145256 2023-10-21T04:49:35.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:35.157+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.157+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 26 count: 0 metaSz: 2482 rp [1] sn: 27 count: 0 metaSz: 2483 rp [2] sn: 28 count: 0 metaSz: 2483 2023-10-21T04:49:35.157+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 145256 2023-10-21T04:49:35.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex 2023-10-21T04:49:35.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:35.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:35.171+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex 2023-10-21T04:49:35.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex started 2023-10-21T04:49:35.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.177+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex 2023-10-21T04:49:35.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:35.178+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:35.179+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex 2023-10-21T04:49:35.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex started 2023-10-21T04:49:35.179+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:49:35.180+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.180+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.212+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 5 Warmup took 57.282331ms 2023-10-21T04:49:35.213+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 5 WriterThreads 1 2023-10-21T04:49:35.213+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 5 Stopped Billing WUs 2023-10-21T04:49:35.213+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index Index: 16759912838651900043 Partition: 5 2023-10-21T04:49:35.213+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16759912838651900043 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:35.213+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16759912838651900043 PartitionId: 6 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:35.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:35.214+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:35.214+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:49:35.214+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: RP version: 18 numRPs: 3 rp [0] sn: 25 count: 0 metaSz: 2482 rp [1] sn: 26 count: 0 metaSz: 2483 rp [2] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:49:35.214+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 26 count: 0 metaSz: 2483 rp [1] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:49:35.214+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 145256 2023-10-21T04:49:35.214+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 153470 2023-10-21T04:49:35.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex 2023-10-21T04:49:35.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:35.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:35.236+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex 2023-10-21T04:49:35.236+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex started 2023-10-21T04:49:35.237+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex 2023-10-21T04:49:35.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:35.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:35.237+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex 2023-10-21T04:49:35.237+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex started 2023-10-21T04:49:35.237+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:49:35.237+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.237+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.237+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.262+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16759912838651900043 PartitionId 6 Warmup took 48.382203ms 2023-10-21T04:49:35.262+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16759912838651900043 partitionId 6 WriterThreads 1 2023-10-21T04:49:35.262+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 16759912838651900043 PartitionId 6 Stopped Billing WUs 2023-10-21T04:49:35.262+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16759912838651900043 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index Index: 16759912838651900043 Partition: 6 2023-10-21T04:49:35.263+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 16759912838651900043, shardIdMap: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] err: 2023-10-21T04:49:35.263+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 16759912838651900043, partnShardIdMap: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]], err: 2023-10-21T04:49:35.263+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:35.263+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 16759912838651900043 2023-10-21T04:49:35.265+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:35.265+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:49:35.266+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:35.266+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:35.268+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:35.269+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 16759912838651900043 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:49:35.271+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:8 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.272+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.381671ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:35.279+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 135.355µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:35.281+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:1 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.305+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:2 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.317+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:3 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.333+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:4 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.340+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:5 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.347+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:6 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.358+05:30 [Info] StorageMgr::openSnapshot IndexInst:16759912838651900043 Partition:7 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:49:35.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996835113805856, throttleDelayMs (new, old, change): (8, 3, 5) 2023-10-21T04:49:35.733+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:35.854+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[16759912838651900043:INDEX_STATE_RECOVERED 17085981505417849194:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:49:35.855+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 16823504650100619022, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:49:35.855+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 16823504650100619022, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:49:35.865+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:35.865+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:35.893+05:30 [Info] switched currmeta from 388 -> 391 force true 2023-10-21T04:49:35.922+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 4186965058349029465,realInstId: 0, indexDefn DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:49:35.922+05:30 [Info] Indexer::handleRecoverIndex InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:49:35.923+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 4186965058349029465, indexDefn DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:49:35.923+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 68.354588ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:35.923+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 956.213µs 2023-10-21T04:49:35.923+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 673.025µs 2023-10-21T04:49:35.923+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:35.923+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 7 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:35.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:35.924+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:35.924+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 63015 recovery UsedSpace 8236 2023-10-21T04:49:35.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:35.926+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:35.926+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:35.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:35.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:35.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:35.935+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:35.935+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex started 2023-10-21T04:49:35.935+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.935+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:35.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:35.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:35.936+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:35.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex started 2023-10-21T04:49:35.936+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:49:35.936+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:35.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:49:35.936+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:35.936+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:49:35.936+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:49:35.936+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:35.936+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex stopped 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:35.936+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:35.942+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:35.942+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:49:35.943+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex closed 2023-10-21T04:49:35.943+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:35.943+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:35.949+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:35.949+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:35.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:49:35.950+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:35.950+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:49:35.950+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:49:35.950+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:35.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex stopped 2023-10-21T04:49:35.950+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:35.951+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:35.951+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:35.951+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:35.966+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:35.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:49:35.966+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex closed 2023-10-21T04:49:35.966+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:35.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex ... 2023-10-21T04:49:35.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex ... 2023-10-21T04:49:35.982+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:35.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex sucessfully destroyed 2023-10-21T04:49:35.983+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex ... 2023-10-21T04:49:35.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex ... 2023-10-21T04:49:35.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:35.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index destroyed 2023-10-21T04:49:35.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:35.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:35.997+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:35.997+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.002+05:30 [Info] Indexer::ReadMemstats Time Taken 934.513µs 2023-10-21T04:49:36.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:36.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.004+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:36.004+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:49:36.005+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex started 2023-10-21T04:49:36.005+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.005+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.017+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:36.019+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.019+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.020+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:36.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.028+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:36.028+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:49:36.028+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex started 2023-10-21T04:49:36.028+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.028+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 7 Warmup took 104.751601ms 2023-10-21T04:49:36.029+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 7 WriterThreads 1 2023-10-21T04:49:36.029+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 7 Stopped Billing WUs 2023-10-21T04:49:36.029+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index Index: 4186965058349029465 Partition: 7 2023-10-21T04:49:36.029+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:36.029+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 8 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:36.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.030+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.030+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:49:36.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.030+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.031+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:36.031+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:36.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.036+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:36.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex started 2023-10-21T04:49:36.036+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.036+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:36.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.039+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:36.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex started 2023-10-21T04:49:36.039+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:49:36.039+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:36.040+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.040+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.040+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:49:36.040+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:36.039+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.039+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.042+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:49:36.042+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:49:36.043+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:36.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex stopped 2023-10-21T04:49:36.043+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.043+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.043+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.043+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.055+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:49:36.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex closed 2023-10-21T04:49:36.055+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:36.055+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.056+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.056+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:49:36.056+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:36.056+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:49:36.056+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:49:36.057+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:36.057+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex stopped 2023-10-21T04:49:36.057+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.057+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.057+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.057+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.069+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:49:36.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex closed 2023-10-21T04:49:36.069+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:36.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex ... 2023-10-21T04:49:36.070+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex ... 2023-10-21T04:49:36.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex sucessfully destroyed 2023-10-21T04:49:36.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex ... 2023-10-21T04:49:36.073+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex ... 2023-10-21T04:49:36.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.079+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index destroyed 2023-10-21T04:49:36.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.088+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.089+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.097+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.097+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:36.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.105+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:36.105+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:36.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.106+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:36.106+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:49:36.106+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex started 2023-10-21T04:49:36.106+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.106+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.107+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:49:36.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex started 2023-10-21T04:49:36.107+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 8 Warmup took 78.036914ms 2023-10-21T04:49:36.108+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.108+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.108+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 8 WriterThreads 1 2023-10-21T04:49:36.109+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 8 Stopped Billing WUs 2023-10-21T04:49:36.109+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index Index: 4186965058349029465 Partition: 8 2023-10-21T04:49:36.109+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:36.109+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 1 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:36.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.110+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.110+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:49:36.110+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.111+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.111+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:36.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:36.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.120+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:36.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex started 2023-10-21T04:49:36.120+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.120+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:36.128+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.129+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:36.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex started 2023-10-21T04:49:36.129+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:49:36.129+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:36.129+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.129+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.130+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.130+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.130+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.130+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:49:36.131+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:36.131+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:49:36.131+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:49:36.131+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:36.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex stopped 2023-10-21T04:49:36.131+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.131+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.131+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.131+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.136+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:49:36.136+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex closed 2023-10-21T04:49:36.136+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:36.136+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.136+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.137+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:49:36.137+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:36.137+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:49:36.137+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:49:36.137+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:36.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex stopped 2023-10-21T04:49:36.137+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.137+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.137+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.137+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.141+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:49:36.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex closed 2023-10-21T04:49:36.141+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:36.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex ... 2023-10-21T04:49:36.141+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex ... 2023-10-21T04:49:36.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex sucessfully destroyed 2023-10-21T04:49:36.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex ... 2023-10-21T04:49:36.146+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex ... 2023-10-21T04:49:36.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.150+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index destroyed 2023-10-21T04:49:36.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.153+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.153+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.153+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.156+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.156+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.162+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:36.162+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:36.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:36.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.166+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:36.166+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:49:36.166+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex started 2023-10-21T04:49:36.166+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.167+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:36.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.168+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:36.168+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:49:36.168+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex started 2023-10-21T04:49:36.168+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 1 Warmup took 58.48773ms 2023-10-21T04:49:36.168+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 1 WriterThreads 1 2023-10-21T04:49:36.168+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 1 Stopped Billing WUs 2023-10-21T04:49:36.168+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index Index: 4186965058349029465 Partition: 1 2023-10-21T04:49:36.168+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:36.169+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 2 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:36.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.169+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.169+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:49:36.170+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.170+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.171+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:36.180+05:30 [Info] switched currmeta from 391 -> 392 force true 2023-10-21T04:49:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.191+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:36.191+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex started 2023-10-21T04:49:36.191+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.191+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:36.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.198+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:36.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex started 2023-10-21T04:49:36.198+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:49:36.198+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:49:36.198+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:36.198+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:49:36.198+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:49:36.198+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:36.198+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex stopped 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.198+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.199+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:49:36.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex closed 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:49:36.206+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:36.206+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:49:36.206+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:49:36.206+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:36.206+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex stopped 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.206+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.213+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:49:36.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex closed 2023-10-21T04:49:36.214+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:36.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex ... 2023-10-21T04:49:36.214+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex ... 2023-10-21T04:49:36.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex sucessfully destroyed 2023-10-21T04:49:36.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex ... 2023-10-21T04:49:36.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex ... 2023-10-21T04:49:36.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index destroyed 2023-10-21T04:49:36.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.244+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.244+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.247+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.247+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.247+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:36.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.253+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:36.253+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:49:36.253+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex started 2023-10-21T04:49:36.253+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.253+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:36.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.255+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:36.255+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:49:36.255+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex started 2023-10-21T04:49:36.256+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 2 Warmup took 86.850986ms 2023-10-21T04:49:36.256+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 2 WriterThreads 1 2023-10-21T04:49:36.256+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 2 Stopped Billing WUs 2023-10-21T04:49:36.256+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index Index: 4186965058349029465 Partition: 2 2023-10-21T04:49:36.256+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:36.257+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 3 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:36.257+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.257+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.258+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:49:36.256+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.256+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.258+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.258+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:36.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:36.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.265+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:36.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex started 2023-10-21T04:49:36.265+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.265+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:36.274+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.275+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:36.275+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex started 2023-10-21T04:49:36.275+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:49:36.275+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:36.276+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.276+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.276+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.275+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.275+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.276+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:49:36.277+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:36.277+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:49:36.277+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:49:36.277+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:36.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex stopped 2023-10-21T04:49:36.278+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.278+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.278+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.278+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.320+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.320+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:49:36.321+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex closed 2023-10-21T04:49:36.321+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:36.321+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.322+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.323+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.323+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:49:36.323+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:36.324+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:49:36.324+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:49:36.324+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:36.324+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex stopped 2023-10-21T04:49:36.325+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.325+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.325+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.325+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.345+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:49:36.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex closed 2023-10-21T04:49:36.346+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:36.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex ... 2023-10-21T04:49:36.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex ... 2023-10-21T04:49:36.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex sucessfully destroyed 2023-10-21T04:49:36.357+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex ... 2023-10-21T04:49:36.359+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex ... 2023-10-21T04:49:36.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.381+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index destroyed 2023-10-21T04:49:36.392+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.393+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.393+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.393+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.396+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.396+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.396+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:36.397+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:36.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:36.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.400+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:36.400+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:49:36.400+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex started 2023-10-21T04:49:36.400+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.400+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:36.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.404+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:36.404+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:49:36.404+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex started 2023-10-21T04:49:36.404+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 3 Warmup took 147.563438ms 2023-10-21T04:49:36.405+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 3 WriterThreads 1 2023-10-21T04:49:36.405+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.405+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 3 Stopped Billing WUs 2023-10-21T04:49:36.405+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index Index: 4186965058349029465 Partition: 3 2023-10-21T04:49:36.405+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:36.405+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.405+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 4 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:36.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.405+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.405+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.406+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:36.405+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:49:36.414+05:30 [Info] switched currmeta from 392 -> 393 force true 2023-10-21T04:49:36.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:36.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.416+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:36.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex started 2023-10-21T04:49:36.416+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.416+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.418+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:36.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.419+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:36.419+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex started 2023-10-21T04:49:36.419+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:49:36.420+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:36.420+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.420+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.420+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.421+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.421+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:49:36.422+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:36.422+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:49:36.422+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:49:36.422+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:36.422+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex stopped 2023-10-21T04:49:36.422+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.422+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.422+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.423+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.433+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:49:36.434+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex closed 2023-10-21T04:49:36.434+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:36.434+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.434+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.435+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.435+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:49:36.435+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:36.435+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:49:36.436+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:49:36.436+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:36.436+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex stopped 2023-10-21T04:49:36.436+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.437+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.437+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.437+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.443+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:49:36.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex closed 2023-10-21T04:49:36.443+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:36.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex ... 2023-10-21T04:49:36.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex ... 2023-10-21T04:49:36.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex sucessfully destroyed 2023-10-21T04:49:36.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex ... 2023-10-21T04:49:36.446+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex ... 2023-10-21T04:49:36.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index destroyed 2023-10-21T04:49:36.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.453+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.453+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.453+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.462+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.462+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:36.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.471+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:36.471+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:49:36.471+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex started 2023-10-21T04:49:36.471+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.471+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:36.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.495+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:36.495+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:49:36.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex started 2023-10-21T04:49:36.496+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.496+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 4 Warmup took 90.659331ms 2023-10-21T04:49:36.496+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.497+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 4 WriterThreads 1 2023-10-21T04:49:36.497+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 4 Stopped Billing WUs 2023-10-21T04:49:36.497+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index Index: 4186965058349029465 Partition: 4 2023-10-21T04:49:36.497+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:36.497+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 5 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:36.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.498+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.498+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:36.498+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.498+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.499+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:49:36.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:36.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.505+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:36.505+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex started 2023-10-21T04:49:36.505+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.505+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:36.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.514+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:36.514+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex started 2023-10-21T04:49:36.514+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:49:36.515+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:49:36.515+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:36.515+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.515+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:49:36.515+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:36.515+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex stopped 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.515+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.516+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:49:36.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex closed 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:49:36.531+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:36.531+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:49:36.531+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:49:36.531+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:36.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex stopped 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.531+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.541+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:49:36.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex closed 2023-10-21T04:49:36.542+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:36.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex ... 2023-10-21T04:49:36.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex ... 2023-10-21T04:49:36.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex sucessfully destroyed 2023-10-21T04:49:36.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex ... 2023-10-21T04:49:36.544+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex ... 2023-10-21T04:49:36.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.547+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index destroyed 2023-10-21T04:49:36.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.550+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.550+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.550+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.556+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.556+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.556+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:36.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.563+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:36.563+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:49:36.563+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex started 2023-10-21T04:49:36.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.563+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:36.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.567+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:36.567+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:49:36.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex started 2023-10-21T04:49:36.567+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 5 Warmup took 69.766133ms 2023-10-21T04:49:36.567+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 5 WriterThreads 1 2023-10-21T04:49:36.567+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 5 Stopped Billing WUs 2023-10-21T04:49:36.567+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index Index: 4186965058349029465 Partition: 5 2023-10-21T04:49:36.567+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 4186965058349029465 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:36.568+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 4186965058349029465 PartitionId: 6 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:36.568+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.568+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.568+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:49:36.569+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.569+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.569+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:49:36.569+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:49:36.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:36.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.587+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:36.587+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex started 2023-10-21T04:49:36.588+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.588+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:36.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.591+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:36.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex started 2023-10-21T04:49:36.591+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:49:36.592+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:49:36.592+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.593+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.593+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.593+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:49:36.594+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:36.592+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.592+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.593+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:36.595+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:49:36.596+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:49:36.596+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:36.597+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex stopped 2023-10-21T04:49:36.597+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.596+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:36.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.598+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:49:36.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex closed 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:36.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:49:36.606+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:36.606+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:49:36.606+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:49:36.606+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:36.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex stopped 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:36.606+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:36.613+05:30 [Info] switched currmeta from 393 -> 393 force true 2023-10-21T04:49:36.614+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:36.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:49:36.614+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex closed 2023-10-21T04:49:36.615+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:36.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex ... 2023-10-21T04:49:36.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex ... 2023-10-21T04:49:36.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex sucessfully destroyed 2023-10-21T04:49:36.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex ... 2023-10-21T04:49:36.619+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex ... 2023-10-21T04:49:36.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.624+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index destroyed 2023-10-21T04:49:36.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.630+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:49:36.630+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.630+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:36.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:36.636+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:36.636+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:49:36.643+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:36.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:49:36.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:49:36.644+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:36.644+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:49:36.644+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex started 2023-10-21T04:49:36.644+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.644+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:36.646+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:36.646+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:49:36.646+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex started 2023-10-21T04:49:36.646+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 4186965058349029465 PartitionId 6 Warmup took 78.678437ms 2023-10-21T04:49:36.647+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 4186965058349029465 partitionId 6 WriterThreads 1 2023-10-21T04:49:36.647+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 4186965058349029465 PartitionId 6 Stopped Billing WUs 2023-10-21T04:49:36.647+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 4186965058349029465 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index Index: 4186965058349029465 Partition: 6 2023-10-21T04:49:36.647+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 4186965058349029465, shardIdMap: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]] err: 2023-10-21T04:49:36.647+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 4186965058349029465, partnShardIdMap: map[1:[17327901024118572215 5629943257627822009] 2:[17327901024118572215 5629943257627822009] 3:[17327901024118572215 5629943257627822009] 4:[17327901024118572215 5629943257627822009] 5:[17327901024118572215 5629943257627822009] 6:[17327901024118572215 5629943257627822009] 7:[17327901024118572215 5629943257627822009] 8:[17327901024118572215 5629943257627822009]], err: 2023-10-21T04:49:36.648+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:36.648+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 4186965058349029465 2023-10-21T04:49:36.648+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 4186965058349029465 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:36.649+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:49:36.649+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:36.651+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 4186965058349029465 Name: idx_partitioned_defer Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:36.652+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:36.652+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4186965058349029465 Partitions [7 8 1 2 3 4 5 6] 2023-10-21T04:49:36.649+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:36.649+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:36.658+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.823843ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:36.659+05:30 [Info] StorageMgr::openSnapshot IndexInst:4186965058349029465 Partition:6 No Snapshot Found. 2023-10-21T04:49:36.660+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 232.648µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:36.662+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 4186965058349029465 Adding Nil Snapshot. 2023-10-21T04:49:36.667+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9998741131722582, throttleDelayMs (new, old, change): (10, 8, 2) 2023-10-21T04:49:36.760+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:36.760+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:36.793+05:30 [Info] switched currmeta from 393 -> 394 force true 2023-10-21T04:49:37.020+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[4186965058349029465:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:49:37.020+05:30 [Info] Indexer::handleMergePartition Source 5272948714421587967 Target 4186965058349029465 2023-10-21T04:49:37.020+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006456540 2023-10-21T04:49:37.021+05:30 [Info] MergePartition: sent async request to update index instance 4186965058349029465 rstate moved to ACTIVE 2023-10-21T04:49:37.033+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.019248ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:37.033+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 5272948714421587967, realInst: 4186965058349029465 2023-10-21T04:49:37.033+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [16367703496921140088 7790129686881525591] belonging to collectionId: c, ttid: ShardToken47:57:c0:39:1d:ee:e8:c9. Initiating build 2023-10-21T04:49:37.127+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [17085981505417849194 16759912838651900043] &{2} 2023-10-21T04:49:37.131+05:30 [Info] Indexer::handleBuildRecoveredIndexes [17085981505417849194 16759912838651900043] 2023-10-21T04:49:37.132+05:30 [Info] bulkUpdateRState: Index instance 17085981505417849194 rstate moved to PENDING 2023-10-21T04:49:37.132+05:30 [Info] bulkUpdateRState: Index instance 16759912838651900043 rstate moved to PENDING 2023-10-21T04:49:37.132+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [17085981505417849194 16759912838651900043] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:49:37.132+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:37.132+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:37.133+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:37.153+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:49:37.154+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:37.154+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:49:37.154+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:49:37.154+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:49:37.155+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:37.158+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:49:37.158+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:49:37.159+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:49:37.161+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId 91310 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} 2023-10-21T04:49:37.164+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:49:37.166+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:49:37.166+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17085981505417849194 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:49:37.166+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[] INIT_STREAM:map[bucket_1:_default:c1:2]] 2023-10-21T04:49:37.167+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 16759912838651900043 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:49:37.168+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:49:37.169+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=18,1=31,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=25,12=23,13=24,14=21,15=25,16=35,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=26,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=27,46=23,47=28,48=20,49=23,50=22,51=27,52=18,53=30,54=22,55=26,56=26,57=27,58=25,59=21,60=24,61=24,62=17,63=24] IndexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 91310 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} NumVBuckets: 64 2023-10-21T04:49:37.170+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:49:37.170+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:49:37.171+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:37.173+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 42.041335ms 2023-10-21T04:49:37.173+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 42.08454ms 2023-10-21T04:49:37.173+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [17085981505417849194 16759912838651900043] 2023-10-21T04:49:37.173+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 139.979623ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:37.174+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[16759912838651900043:true 17085981505417849194:true] 2023-10-21T04:49:37.179+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:49:37.186+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.70697ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:37.196+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.088844ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:37.223+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:49:37.235+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:37.236+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:37.236+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:37.265+05:30 [Info] switched currmeta from 394 -> 400 force true 2023-10-21T04:49:37.390+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:37.390+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:37.396+05:30 [Info] switched currmeta from 400 -> 400 force true 2023-10-21T04:49:37.462+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:37.463+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:37.493+05:30 [Info] switched currmeta from 400 -> 401 force true 2023-10-21T04:49:37.589+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:37.589+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:37.607+05:30 [Info] switched currmeta from 401 -> 403 force true 2023-10-21T04:49:37.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:49:37.680+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35048" doAuth() ... 2023-10-21T04:49:37.680+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35048" auth successful 2023-10-21T04:49:37.680+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:35048" +1 2023-10-21T04:49:37.713+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:37.713+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:37.750+05:30 [Info] switched currmeta from 403 -> 405 force true 2023-10-21T04:49:37.879+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:37.879+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:37.945+05:30 [Info] switched currmeta from 405 -> 405 force true 2023-10-21T04:49:38.260+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:38.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9999360875792777 2023-10-21T04:49:39.267+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:39.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9995471046179587 2023-10-21T04:49:40.228+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:40.257+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:40.257+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:40.261+05:30 [Info] switched currmeta from 405 -> 405 force true 2023-10-21T04:49:40.273+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:40.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9973795023516614 2023-10-21T04:49:41.271+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 67576817345502 18 91310 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:41.272+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 44231811555836 31 91310 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:49:41.275+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 95570275214302 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.276+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 197901620499212 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.276+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 39196762516158 19 91310 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:49:41.278+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:41.279+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 143637217187246 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.279+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 95215784105904 25 91310 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:41.287+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 4065155603859 28 91310 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:41.287+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 91960699361891 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.288+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 147449900650979 28 91310 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:41.289+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 77783296532664 25 91310 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:49:41.289+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 27353924173936 22 91310 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:49:41.291+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 233597919143467 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.292+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 198938078557080 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.296+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 122353375037127 21 91310 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:49:41.297+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 129413110409107 25 91310 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:41.297+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 33320884322178 35 91310 172.31.5.112:9000. HWT [34-35,35]. 2023-10-21T04:49:41.299+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 132000914750455 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.299+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 25078414014136 18 91310 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:41.301+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 79253753007122 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.303+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 7646518130509 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.303+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 101186302195296 32 91310 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:49:41.306+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 215732063158788 18 91310 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:41.308+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 230675090583633 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.310+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 232497361012987 18 91310 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:41.310+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 113904165092305 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.315+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 61904892722472 19 91310 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:49:41.315+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 131315346703752 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.315+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 201780742117921 20 91310 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:41.321+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 241043521322196 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.321+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 248212138718363 17 91310 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:49:41.321+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 201745596055144 26 91310 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:41.328+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 190878005091193 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.328+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 258788849496338 20 91310 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:41.329+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 19399479457331 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.335+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 280972182784873 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.335+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 173020619068939 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.339+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 63194637348236 27 91310 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:41.342+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 13155294694475 25 91310 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:41.342+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 119373194724283 26 91310 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:41.349+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 277109307527403 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.349+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 186283152043223 18 91310 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:41.351+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 209574230498887 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.354+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 255609313786736 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.356+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 49352389572227 27 91310 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:49:41.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 83754344146634 23 91310 172.31.5.112:9000. HWT [23-23,23]. 2023-10-21T04:49:41.361+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 270106388647522 25 91310 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:41.362+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 62156602796507 28 91310 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:41.364+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 193080054939721 22 91310 172.31.5.112:9000. HWT [22-22,22]. 2023-10-21T04:49:41.364+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 80989250905985 20 91310 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:41.366+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 204579891639872 27 91310 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:41.366+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 150369066093368 18 91310 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:41.369+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 102294135990395 22 91310 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:49:41.369+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 212804514607186 30 91310 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:49:41.371+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 279208201382660 23 91310 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:41.375+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 65071891036109 26 91310 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:41.376+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 152904045160406 26 91310 172.31.5.112:9000. HWT [26-26,26]. 2023-10-21T04:49:41.379+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 218103594015078 27 91310 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:41.383+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 90104317736129 21 91310 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:41.383+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 35015089382353 25 91310 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:41.387+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 223555064174984 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.388+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 184273998690001 17 91310 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:49:41.388+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 257224089448052 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.393+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 202830985870043 24 91310 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:41.397+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_default:c1 InstanceIds [16759912838651900043 17085981505417849194] 2023-10-21T04:49:41.397+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 91310 2023-10-21T04:49:41.398+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91310 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 17085981505417849194 INDEX_STATE_CATCHUP 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 16759912838651900043 INDEX_STATE_CATCHUP 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 17085981505417849194 true 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 16759912838651900043 true 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 17085981505417849194 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 91310 BuildTS: [0=18,1=31,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=25,12=23,13=24,14=21,15=25,16=35,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=26,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=27,46=23,47=28,48=20,49=23,50=22,51=27,52=18,53=30,54=22,55=26,56=26,57=27,58=25,59=21,60=24,61=24,62=17,63=24] 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_default:c1, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} , fetchKVSeq false 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:49:41.398+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:49:41.398+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91310 STREAM_ACTIVE 2023-10-21T04:49:41.398+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 91310 2023-10-21T04:49:41.398+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:49:41.398+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:49:41.398+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_1:_default:c1 91310 2023-10-21T04:49:41.398+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:41.398+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:41.398+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:41.399+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:41.399+05:30 [Info] send progress stats to clients 2023-10-21T04:49:41.399+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91310 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:41.399+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17085981505417849194 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:49:41.399+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 16759912838651900043 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:49:41.399+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:49:41.399+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:49:41.399+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 1 CurrMaint 0 New 0 2023-10-21T04:49:41.399+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91310 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:41.400+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:49:41.400+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:49:41.400+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:49:41.400+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:49:41.400+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:35048: use of closed network connection 2023-10-21T04:49:41.400+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:35048" exit: read tcp4 127.0.0.1:9121->127.0.0.1:35048: use of closed network connection 2023-10-21T04:49:41.400+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35048" closed ! 2023-10-21T04:49:41.400+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:49:41.402+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_1:_default:c1 2023-10-21T04:49:41.402+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:49:41.418+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 17.662317ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:41.451+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 27.88462ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:41.501+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:41.501+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:41.510+05:30 [Info] switched currmeta from 405 -> 409 force true 2023-10-21T04:49:41.527+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:41.528+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:41.533+05:30 [Info] switched currmeta from 409 -> 409 force true 2023-10-21T04:49:41.601+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:49:41.601+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:49:41.601+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:49:41.601+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91310 2023-10-21T04:49:41.601+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_1:_default:c1 2023-10-21T04:49:41.601+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_1 2023-10-21T04:49:41.601+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:41.601+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:49:41.601+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:49:41.601+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:49:41.601+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:41.601+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed MAINT_STREAM 2023-10-21T04:49:41.601+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_1 reset bucket state to NIL 2023-10-21T04:49:41.601+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:49:41.601+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_1 SessionId 51546 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} 2023-10-21T04:49:41.601+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1 Added for Stream MAINT_STREAM 2023-10-21T04:49:41.601+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:1] INIT_STREAM:map[]] 2023-10-21T04:49:41.602+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:2] INIT_STREAM:map[]] 2023-10-21T04:49:41.602+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_1 2023-10-21T04:49:41.602+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: true SessionId: 51546 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} NumVBuckets: 64 2023-10-21T04:49:41.602+05:30 [Info] DATP[->dataport ":9123"] started ... 2023-10-21T04:49:41.602+05:30 [Info] MutationStreamReader: Setting Stream Workers MAINT_STREAM 32 2023-10-21T04:49:41.608+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> ] 2023-10-21T04:49:41.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9384753212778915, throttleDelayMs (new, old, change): (9, 10, -1) 2023-10-21T04:49:41.671+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:59130" doAuth() ... 2023-10-21T04:49:41.671+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:59130" auth successful 2023-10-21T04:49:41.671+05:30 [Info] DATP[->dataport ":9123"] new connection "127.0.0.1:59130" +1 2023-10-21T04:49:41.894+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:41.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:41.896+05:30 [Info] switched currmeta from 409 -> 410 force true 2023-10-21T04:49:41.926+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:41.926+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:41.928+05:30 [Info] switched currmeta from 410 -> 411 force true 2023-10-21T04:49:42.017+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:42.017+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:42.026+05:30 [Info] switched currmeta from 411 -> 412 force true 2023-10-21T04:49:42.078+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:42.078+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:42.084+05:30 [Info] switched currmeta from 412 -> 413 force true 2023-10-21T04:49:42.282+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:42.283+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00c2ceb40 2023-10-21T04:49:42.283+05:30 [Info] handleUpdateIndexRState: Index instance 17085981505417849194 rstate moved to ACTIVE 2023-10-21T04:49:42.289+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.805881ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:42.289+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 544693394183450628. Retrying... 2023-10-21T04:49:42.289+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[16759912838651900043:true 17085981505417849194:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:49:42.289+05:30 [Info] Indexer::handleMergePartition Source 13917065492675748491 Target 16759912838651900043 2023-10-21T04:49:42.289+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00807cd20 2023-10-21T04:49:42.289+05:30 [Info] MergePartition: sent async request to update index instance 16759912838651900043 rstate moved to ACTIVE 2023-10-21T04:49:42.303+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.404933ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:42.303+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 13917065492675748491, realInst: 16759912838651900043 2023-10-21T04:49:42.308+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 668551233566132751, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:49:42.309+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 668551233566132751, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:49:42.329+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:42.329+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:42.333+05:30 [Info] switched currmeta from 413 -> 415 force true 2023-10-21T04:49:42.347+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:42.347+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:42.359+05:30 [Info] switched currmeta from 415 -> 415 force true 2023-10-21T04:49:42.373+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 544693394183450628,realInstId: 0, indexDefn DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:49:42.373+05:30 [Info] Indexer::handleRecoverIndex InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:49:42.373+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 278.86µs 2023-10-21T04:49:42.373+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 334.053µs 2023-10-21T04:49:42.374+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 544693394183450628 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[17327901024118572215 5629943257627822009]] 2023-10-21T04:49:42.374+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 544693394183450628 PartitionId: 0 using shardIds: [17327901024118572215 5629943257627822009], rebalance running: true 2023-10-21T04:49:42.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:49:42.375+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: after recovery: resident ratio 0 requestQuota 1055626 memSz 64 2023-10-21T04:49:42.375+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: RP version: 19 numRPs: 2 rp [0] sn: 19 count: 6 metaSz: 2483 rp [1] sn: 21 count: 6 metaSz: 2483 2023-10-21T04:49:42.375+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: doInit: data UsedSpace 116300 recovery UsedSpace 153934 2023-10-21T04:49:42.375+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 544693394183450628, indexDefn DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:49:42.376+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 67.211833ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:42.404+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:49:42.420+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 1 44231811555836 31 51546 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:49:42.420+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 0 67576817345502 18 51546 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:42.426+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 5 95215784105904 25 51546 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:42.426+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 3 39196762516158 19 51546 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:49:42.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex 2023-10-21T04:49:42.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:49:42.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:49:42.431+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex 2023-10-21T04:49:42.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex started 2023-10-21T04:49:42.431+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 544693394183450628 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:49:42.431+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:42.431+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:42.440+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 7 4065155603859 28 51546 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:42.442+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 9 147449900650979 28 51546 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:42.448+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:42.448+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 2 197901620499212 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.449+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 544693394183450628 PartitionId 0 Warmup took 74.565374ms 2023-10-21T04:49:42.449+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 544693394183450628 partitionId 0 WriterThreads 4 2023-10-21T04:49:42.449+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 544693394183450628 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index Index: 544693394183450628 Partition: 0 2023-10-21T04:49:42.449+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 544693394183450628, shardIdMap: map[0:[17327901024118572215]] err: 2023-10-21T04:49:42.450+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 544693394183450628, partnShardIdMap: map[0:[17327901024118572215]], err: 2023-10-21T04:49:42.450+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:42.450+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 544693394183450628 2023-10-21T04:49:42.450+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:42.451+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:49:42.451+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:42.451+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:42.452+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:42.453+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 544693394183450628 Partitions [0] 2023-10-21T04:49:42.454+05:30 [Info] StorageMgr::openSnapshot IndexInst:544693394183450628 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:49:42.455+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 13 198938078557080 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.455+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 6 143637217187246 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.455+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 11 77783296532664 25 51546 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:49:42.458+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 15 129413110409107 25 51546 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:42.462+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.293851ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:42.462+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 64.321µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:49:42.464+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 4 95570275214302 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.476+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:42.477+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:42.477+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 21 7646518130509 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.478+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 8 91960699361891 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.478+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 17 25078414014136 18 51546 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:42.478+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 19 79253753007122 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.478+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 12 233597919143467 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.481+05:30 [Info] switched currmeta from 415 -> 417 force true 2023-10-21T04:49:42.499+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:42.500+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:42.505+05:30 [Info] switched currmeta from 417 -> 417 force true 2023-10-21T04:49:42.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16759912838651900043:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:49:42.506+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[16759912838651900043:[8 1 2 3 4 5 6 7] 17085981505417849194:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:49:42.520+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 14 122353375037127 21 51546 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:49:42.527+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 26 201780742117921 20 51546 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:42.527+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 27 131315346703752 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.527+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 25 232497361012987 18 51546 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:42.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 23 230675090583633 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 16 33320884322178 35 51546 172.31.5.112:9000. HWT [34-35,35]. 2023-10-21T04:49:42.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 22 215732063158788 18 51546 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:42.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 18 132000914750455 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 10 27353924173936 22 51546 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:49:42.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 24 113904165092305 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.528+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 20 101186302195296 32 51546 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:49:42.529+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 29 241043521322196 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.534+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 31 201745596055144 26 51546 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:42.536+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 28 61904892722472 19 51546 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:49:42.537+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 32 19399479457331 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.538+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 33 258788849496338 20 51546 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:42.538+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 30 248212138718363 17 51546 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:49:42.538+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 35 173020619068939 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.538+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 34 190878005091193 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.541+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 36 280972182784873 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.543+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9316375249999768625:[8 1 2 3 4 5 6 7]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:49:42.543+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9316375249999768625:[8 1 2 3 4 5 6 7] 10558612751966480746:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:49:42.545+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 37 63194637348236 27 51546 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:42.551+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 40 186283152043223 18 51546 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:42.552+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 38 13155294694475 25 51546 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:42.552+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 39 119373194724283 26 51546 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:42.557+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 43 270106388647522 25 51546 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:42.557+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 42 209574230498887 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.557+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 41 277109307527403 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.558+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 45 49352389572227 27 51546 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:49:42.559+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 44 255609313786736 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.563+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 46 83754344146634 23 51546 172.31.5.112:9000. HWT [23-23,23]. 2023-10-21T04:49:42.563+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 47 62156602796507 28 51546 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:42.567+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 48 80989250905985 20 51546 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:42.567+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 49 279208201382660 23 51546 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:42.569+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 51 204579891639872 27 51546 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:42.570+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 50 193080054939721 22 51546 172.31.5.112:9000. HWT [22-22,22]. 2023-10-21T04:49:42.572+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 53 212804514607186 30 51546 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:49:42.572+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 52 150369066093368 18 51546 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:42.577+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 55 65071891036109 26 51546 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:42.577+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 54 102294135990395 22 51546 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:49:42.581+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 57 218103594015078 27 51546 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:42.581+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 56 152904045160406 26 51546 172.31.5.112:9000. HWT [26-26,26]. 2023-10-21T04:49:42.582+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 59 90104317736129 21 51546 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:42.582+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 58 35015089382353 25 51546 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:42.585+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 60 223555064174984 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.587+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 61 257224089448052 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.593+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1 InstanceIds [17085981505417849194 16759912838651900043] 2023-10-21T04:49:42.594+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_1 Cid SessionId 51546 2023-10-21T04:49:42.594+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 63 202830985870043 24 51546 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:42.595+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 51546 2023-10-21T04:49:42.595+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:49:42.595+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_1 2023-10-21T04:49:42.595+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:49:42.595+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_1 2023-10-21T04:49:42.596+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 51546 2023-10-21T04:49:42.596+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_1 SessionId 51546 STREAM_ACTIVE 2023-10-21T04:49:42.596+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:49:42.600+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_1 62 184273998690001 17 51546 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:49:42.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.928100034604014, throttleDelayMs (new, old, change): (7, 9, -2) 2023-10-21T04:49:43.027+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:43.027+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:43.032+05:30 [Info] switched currmeta from 417 -> 418 force true 2023-10-21T04:49:43.074+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:43.074+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:43.089+05:30 [Info] switched currmeta from 418 -> 419 force true 2023-10-21T04:49:43.452+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[544693394183450628:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:49:43.452+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [668551233566132751] belonging to collectionId: 9, ttid: ShardToken47:57:c0:39:1d:ee:e8:c9. Initiating build 2023-10-21T04:49:43.461+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [544693394183450628] &{2} 2023-10-21T04:49:43.461+05:30 [Info] Indexer::handleBuildRecoveredIndexes [544693394183450628] 2023-10-21T04:49:43.461+05:30 [Info] bulkUpdateRState: Index instance 544693394183450628 rstate moved to PENDING 2023-10-21T04:49:43.461+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [544693394183450628] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:49:43.462+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:43.462+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:43.462+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:43.463+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_system:_query 2023-10-21T04:49:43.463+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:49:43.463+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:49:43.463+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:43.463+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:49:43.463+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:49:43.463+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:49:43.463+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:43.463+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:49:43.463+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:49:43.463+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:49:43.463+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_system:_query reset bucket state to NIL 2023-10-21T04:49:43.463+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:49:43.463+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId 25669 RestartTS bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} 2023-10-21T04:49:43.464+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:49:43.464+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:49:43.464+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:49:43.464+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:2] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:49:43.464+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 544693394183450628 stream INIT_STREAM keyspaceId bucket_1:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:49:43.464+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:49:43.464+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=18,1=31,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=25,12=23,13=24,14=21,15=25,16=35,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=26,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=27,46=23,47=28,48=20,49=23,50=22,51=27,52=18,53=30,54=22,55=26,56=26,57=27,58=25,59=21,60=24,61=24,62=17,63=24] IndexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 25669 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} NumVBuckets: 64 2023-10-21T04:49:43.471+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:49:43.471+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:49:43.475+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_system:_query Check pending stream merge. 2023-10-21T04:49:43.475+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:43.476+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 15.172599ms 2023-10-21T04:49:43.478+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 16.38691ms 2023-10-21T04:49:43.478+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [544693394183450628] 2023-10-21T04:49:43.480+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 26.843337ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:43.481+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[544693394183450628:true] 2023-10-21T04:49:43.482+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:49:43.493+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:43.493+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:43.497+05:30 [Info] switched currmeta from 419 -> 420 force true 2023-10-21T04:49:43.503+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.538143ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:43.510+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:43.521+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:43.522+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:43.526+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35240" doAuth() ... 2023-10-21T04:49:43.526+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35240" auth successful 2023-10-21T04:49:43.526+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:35240" +1 2023-10-21T04:49:43.546+05:30 [Info] switched currmeta from 420 -> 421 force true 2023-10-21T04:49:43.596+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:49:43.596+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_1 2023-10-21T04:49:43.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7194150159373862, throttleDelayMs (new, old, change): (2, 7, -5) 2023-10-21T04:49:43.779+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 1 44231811555836 31 25669 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:49:43.779+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 0 67576817345502 18 25669 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:43.781+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 2 197901620499212 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.783+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 3 39196762516158 19 25669 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:49:43.787+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 5 95215784105904 25 25669 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:43.787+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 6 143637217187246 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.787+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 4 95570275214302 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.789+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 7 4065155603859 28 25669 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:43.791+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 8 91960699361891 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.795+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 10 27353924173936 22 25669 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:49:43.795+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 9 147449900650979 28 25669 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:43.798+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 11 77783296532664 25 25669 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:49:43.800+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 12 233597919143467 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.804+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 14 122353375037127 21 25669 172.31.5.112:9000. HWT [21-21,21]. 2023-10-21T04:49:43.804+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 13 198938078557080 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.805+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 15 129413110409107 25 25669 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:43.808+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 16 33320884322178 35 25669 172.31.5.112:9000. HWT [34-35,35]. 2023-10-21T04:49:43.809+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 17 25078414014136 18 25669 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:43.812+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 18 132000914750455 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.813+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 19 79253753007122 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.815+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 20 101186302195296 32 25669 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:49:43.817+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 21 7646518130509 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.819+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 22 215732063158788 18 25669 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:43.821+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 23 230675090583633 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.824+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 25 232497361012987 18 25669 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:43.825+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 24 113904165092305 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.826+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 26 201780742117921 20 25669 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:43.829+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 27 131315346703752 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.831+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 28 61904892722472 19 25669 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:49:43.833+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 29 241043521322196 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.833+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 30 248212138718363 17 25669 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:49:43.836+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 31 201745596055144 26 25669 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:43.841+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 33 258788849496338 20 25669 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:43.841+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 32 19399479457331 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.843+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 34 190878005091193 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 35 173020619068939 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 37 63194637348236 27 25669 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:43.850+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 36 280972182784873 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.851+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 38 13155294694475 25 25669 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:43.852+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 39 119373194724283 26 25669 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:43.855+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 40 186283152043223 18 25669 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:43.857+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 41 277109307527403 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.867+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 44 255609313786736 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.867+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 43 270106388647522 25 25669 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:43.867+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 42 209574230498887 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.867+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 45 49352389572227 27 25669 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:49:43.871+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 47 62156602796507 28 25669 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:49:43.871+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 46 83754344146634 23 25669 172.31.5.112:9000. HWT [23-23,23]. 2023-10-21T04:49:43.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 49 279208201382660 23 25669 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:49:43.879+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 48 80989250905985 20 25669 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:49:43.882+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 51 204579891639872 27 25669 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:43.882+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 50 193080054939721 22 25669 172.31.5.112:9000. HWT [22-22,22]. 2023-10-21T04:49:43.885+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 52 150369066093368 18 25669 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:49:43.888+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 53 212804514607186 30 25669 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:49:43.892+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 54 102294135990395 22 25669 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:49:43.897+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 56 152904045160406 26 25669 172.31.5.112:9000. HWT [26-26,26]. 2023-10-21T04:49:43.897+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 55 65071891036109 26 25669 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:49:43.899+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 57 218103594015078 27 25669 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:49:43.901+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 58 35015089382353 25 25669 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:49:43.902+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 59 90104317736129 21 25669 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:49:43.905+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 60 223555064174984 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.906+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 61 257224089448052 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.910+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 63 202830985870043 24 25669 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:49:43.910+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_system:_query 62 184273998690001 17 25669 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:49:43.914+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_system:_query InstanceIds [544693394183450628] 2023-10-21T04:49:43.914+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query Cid 9 SessionId 25669 2023-10-21T04:49:43.914+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 25669 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 544693394183450628 INDEX_STATE_CATCHUP 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 544693394183450628 true 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 544693394183450628 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query Session: 25669 BuildTS: [0=18,1=31,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=25,12=23,13=24,14=21,15=25,16=35,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=26,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=27,46=23,47=28,48=20,49=23,50=22,51=27,52=18,53=30,54=22,55=26,56=26,57=27,58=25,59=21,60=24,61=24,62=17,63=24] 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_1:_system:_query, initFlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} , fetchKVSeq false 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:49:43.914+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:49:43.914+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 25669 STREAM_ACTIVE 2023-10-21T04:49:43.915+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 25669 2023-10-21T04:49:43.915+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_system:_query 25669 2023-10-21T04:49:43.915+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:43.915+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:43.915+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:43.915+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:43.915+05:30 [Info] send progress stats to clients 2023-10-21T04:49:43.915+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:49:43.915+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:49:43.915+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[bucket_1:_system:_query:1]] 2023-10-21T04:49:43.915+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 25669 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:49:43.916+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:49:43.916+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:49:43.916+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [0=4,1=31,2=4,3=4,4=4,5=4,6=4,7=4,8=4,9=4,10=4,11=25,12=4,13=4,14=21,15=4,16=35,17=4,18=4,19=4,20=4,21=4,22=4,23=4,24=4,25=4,26=4,27=4,28=4,29=4,30=4,31=26,32=4,33=4,34=4,35=4,36=4,37=4,38=4,39=4,40=4,41=4,42=4,43=4,44=4,45=27,46=23,47=4,48=4,49=4,50=22,51=4,52=4,53=4,54=4,55=26,56=26,57=4,58=4,59=4,60=4,61=4,62=4,63=4] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:43.920+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_1 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:49:43.930+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.587071ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:43.947+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:43.947+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:43.949+05:30 [Info] switched currmeta from 421 -> 422 force true 2023-10-21T04:49:43.951+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_1 2023-10-21T04:49:43.951+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 25669 2023-10-21T04:49:43.951+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 25669 2023-10-21T04:49:43.951+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_system:_query set minMergeTs 2023-10-21T04:49:43.952+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:49:43.952+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_system:_query 544693394183450628 false 2023-10-21T04:49:43.952+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_system:_query 544693394183450628 2023-10-21T04:49:43.952+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_system:_query. 2023-10-21T04:49:43.952+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 25669 2023-10-21T04:49:44.465+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:44.465+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:44.468+05:30 [Info] switched currmeta from 422 -> 423 force true 2023-10-21T04:49:44.514+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:44.597+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:49:44.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.5762543335203234, throttleDelayMs (new, old, change): (0, 2, -2) 2023-10-21T04:49:44.917+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:49:44.917+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_system:_query 2023-10-21T04:49:44.917+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 544693394183450628 Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query SessionId: 25669, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType FORCE_COMMIT OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 31 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 25 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 21 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 34 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 27 27} { 46 4c2c93ee66ca 2 23 23 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 22 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 26 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} 2023-10-21T04:49:44.917+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_system:_query 544693394183450628 INDEX_STATE_ACTIVE 2023-10-21T04:49:44.917+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_system:_query State Changed to INACTIVE 2023-10-21T04:49:44.917+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:49:44.917+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:49:44.917+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:49:44.917+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId: 25669 2023-10-21T04:49:44.917+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:44.918+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:44.918+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:49:44.918+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 25669 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:44.918+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 544693394183450628 from stream INIT_STREAM keyspaceId bucket_1:_system:_query 2023-10-21T04:49:44.918+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_system:_query 2023-10-21T04:49:44.918+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:49:44.918+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:49:44.918+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 25669 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:44.918+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:49:44.918+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:49:44.918+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_system:_query latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:49:44.919+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:49:44.919+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:35240: use of closed network connection 2023-10-21T04:49:44.919+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:35240" exit: read tcp4 127.0.0.1:9121->127.0.0.1:35240: use of closed network connection 2023-10-21T04:49:44.919+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:35240" closed ! 2023-10-21T04:49:44.919+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:49:44.919+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:49:44.919+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:49:44.919+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:49:44.919+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:49:44.919+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:49:44.919+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:44.919+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_system:_query Stream: INIT_STREAM SessionId 25669 2023-10-21T04:49:44.920+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_system:_query] 2023-10-21T04:49:44.932+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.174541ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:44.963+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_system:_query]. Err genServer.closed 2023-10-21T04:49:44.963+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:49:44.963+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_system:_query] Treating genServer.closed As Success 2023-10-21T04:49:44.963+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 25669 2023-10-21T04:49:44.963+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query SessionId 25669 2023-10-21T04:49:44.963+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_system:_query 2023-10-21T04:49:44.983+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:44.983+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:44.986+05:30 [Info] switched currmeta from 423 -> 424 force true 2023-10-21T04:49:45.104+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:45.112+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:45.112+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:45.115+05:30 [Info] switched currmeta from 424 -> 424 force true 2023-10-21T04:49:45.491+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:45.492+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:45.498+05:30 [Info] switched currmeta from 424 -> 425 force true 2023-10-21T04:49:45.518+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:45.518+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0059f82a0 2023-10-21T04:49:45.518+05:30 [Info] handleUpdateIndexRState: Index instance 544693394183450628 rstate moved to ACTIVE 2023-10-21T04:49:45.527+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.179245ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:45.527+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[544693394183450628:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:49:45.527+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken47:57:c0:39:1d:ee:e8:c9, elapsed(sec): 11.868557900999999 2023-10-21T04:49:45.537+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken47:57:c0:39:1d:ee:e8:c9 2023-10-21T04:49:45.537+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardTokenb1:70:f2:cb:a5:ff:fd:1d 2023-10-21T04:49:45.537+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[6455624821495495854:true 18374611443167983589:true] 2023-10-21T04:49:45.537+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.486µs 2023-10-21T04:49:45.537+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [18374611443167983589 6455624821495495854], isLockedForRecovery: true 2023-10-21T04:49:45.537+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [18374611443167983589 6455624821495495854], errMap: map[6455624821495495854: 18374611443167983589:], elapsed: 11.347µs 2023-10-21T04:49:45.537+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 6145407649125550089, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:49:45.537+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:45.537+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 6145407649125550089, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:49:45.578+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 16653501928284257012,realInstId: 0, indexDefn DefnId: 6145407649125550089 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:49:45.579+05:30 [Info] Indexer::handleRecoverIndex InstId: 16653501928284257012 Defn: DefnId: 6145407649125550089 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:49:45.579+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 675.578µs 2023-10-21T04:49:45.580+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 814.977µs 2023-10-21T04:49:45.580+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 16653501928284257012 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[18374611443167983589 6455624821495495854]] 2023-10-21T04:49:45.580+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 16653501928284257012 PartitionId: 0 using shardIds: [18374611443167983589 6455624821495495854], rebalance running: true 2023-10-21T04:49:45.581+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': added 2023-10-21T04:49:45.582+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCleaner initialized 2023-10-21T04:49:45.582+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:45.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:49:45.583+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCleaner initialized for recovery 2023-10-21T04:49:45.583+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:45.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery 2023-10-21T04:49:45.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery) 2023-10-21T04:49:45.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true] 2023-10-21T04:49:45.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1179648] Data log Head [0] Tail [1396736] 2023-10-21T04:49:45.583+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1179648] 2023-10-21T04:49:45.586+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.586+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex 2023-10-21T04:49:45.586+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:49:45.587+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.587+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex 2023-10-21T04:49:45.589+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCleaner initialized 2023-10-21T04:49:45.589+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:45.589+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:49:45.590+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCleaner initialized for recovery 2023-10-21T04:49:45.590+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:49:45.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery 2023-10-21T04:49:45.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery) 2023-10-21T04:49:45.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true] 2023-10-21T04:49:45.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1548288] Data log Head [0] Tail [1724416] 2023-10-21T04:49:45.590+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1548288] 2023-10-21T04:49:45.592+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.582+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 16653501928284257012, indexDefn DefnId: 6145407649125550089 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:49:45.598+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 60.967421ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:45.602+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 8, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:49:45.592+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex 2023-10-21T04:49:45.621+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:45.621+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:45.621+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.621+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex 2023-10-21T04:49:45.621+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.621+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex 2023-10-21T04:49:45.622+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.622+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex 2023-10-21T04:49:45.622+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.622+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex 2023-10-21T04:49:45.622+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.622+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex 2023-10-21T04:49:45.622+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.622+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.622+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex 2023-10-21T04:49:45.622+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.622+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex 2023-10-21T04:49:45.622+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex 2023-10-21T04:49:45.623+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.623+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex 2023-10-21T04:49:45.635+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.636+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex 2023-10-21T04:49:45.635+05:30 [Info] switched currmeta from 425 -> 427 force true 2023-10-21T04:49:45.636+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.638+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex 2023-10-21T04:49:45.638+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.641+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex 2023-10-21T04:49:45.642+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.642+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.644+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex 2023-10-21T04:49:45.644+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.644+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex 2023-10-21T04:49:45.644+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.645+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex 2023-10-21T04:49:45.646+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.646+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex 2023-10-21T04:49:45.646+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.646+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex 2023-10-21T04:49:45.647+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.647+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex 2023-10-21T04:49:45.648+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.648+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex 2023-10-21T04:49:45.648+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.648+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex 2023-10-21T04:49:45.649+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.649+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex 2023-10-21T04:49:45.649+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.650+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex 2023-10-21T04:49:45.650+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.650+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex 2023-10-21T04:49:45.652+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.652+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex 2023-10-21T04:49:45.653+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : replayOffset: 1388544 (RP:1282048) (CP:1388544) 2023-10-21T04:49:45.653+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1388544] took [69.6626ms] 2023-10-21T04:49:45.653+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.653+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299300 controller mem 2948 2023-10-21T04:49:45.654+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.654+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299256 controller mem 2948 2023-10-21T04:49:45.654+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.654+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299256 controller mem 2948 2023-10-21T04:49:45.654+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.654+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299212 controller mem 2948 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299168 controller mem 2948 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299168 controller mem 2948 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1056226, resident 1, memSz 68768, memSzIndex 858, numRecords 1000 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299168 controller mem 2948 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.655+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299124 controller mem 2948 2023-10-21T04:49:45.656+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.656+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299080 controller mem 2948 2023-10-21T04:49:45.656+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.656+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299036 controller mem 2948 2023-10-21T04:49:45.656+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.656+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299036 controller mem 2948 2023-10-21T04:49:45.657+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.657+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37299036 controller mem 2948 2023-10-21T04:49:45.657+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.657+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37298992 controller mem 2948 2023-10-21T04:49:45.657+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.657+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37298992 controller mem 2948 2023-10-21T04:49:45.658+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.658+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37298992 controller mem 2948 2023-10-21T04:49:45.658+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.658+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37298992 controller mem 2948 2023-10-21T04:49:45.658+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.658+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 37298992 controller mem 2948 2023-10-21T04:49:45.658+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1396736] replayOffset [1388544] 2023-10-21T04:49:45.659+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [75.17278ms] 2023-10-21T04:49:45.659+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.659+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.659+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.659+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.659+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.659+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.660+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.660+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.660+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.660+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.660+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.660+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.660+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.661+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.661+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.661+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.661+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data], Shared [true]. Built [17] plasmas, took [77.582923ms] 2023-10-21T04:49:45.661+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: after recovery: resident ratio 0 requestQuota 1056226 memSz 1860 2023-10-21T04:49:45.661+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 11 count: 1000 metaSz: 2505 rp [1] sn: 12 count: 1000 metaSz: 2516 2023-10-21T04:49:45.661+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: doInit: data UsedSpace 190435 recovery UsedSpace 24570 2023-10-21T04:49:45.664+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex 2023-10-21T04:49:45.668+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.669+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex 2023-10-21T04:49:45.669+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:45.669+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:45.670+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.671+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex 2023-10-21T04:49:45.672+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.672+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex 2023-10-21T04:49:45.675+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.676+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex 2023-10-21T04:49:45.678+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.678+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex 2023-10-21T04:49:45.685+05:30 [Info] switched currmeta from 427 -> 427 force true 2023-10-21T04:49:45.689+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.691+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex 2023-10-21T04:49:45.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Assign plasmaId 24 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex 2023-10-21T04:49:45.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 2023-10-21T04:49:45.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data 2023-10-21T04:49:45.709+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper started 2023-10-21T04:49:45.709+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex 2023-10-21T04:49:45.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex started 2023-10-21T04:49:45.711+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:45.712+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:45.731+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:49:45.732+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex 2023-10-21T04:49:45.737+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : replayOffset: 1716224 (RP:1699840) (CP:1716224) 2023-10-21T04:49:45.737+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1716224] took [146.935044ms] 2023-10-21T04:49:45.737+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1055626, resident 1, memSz 1439, memSzIndex 0, numRecords 16 2023-10-21T04:49:45.737+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651515 controller mem 3828 2023-10-21T04:49:45.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651471 controller mem 3828 2023-10-21T04:49:45.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651427 controller mem 3828 2023-10-21T04:49:45.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.738+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651383 controller mem 3828 2023-10-21T04:49:45.739+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.739+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651339 controller mem 3828 2023-10-21T04:49:45.739+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.739+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651295 controller mem 3828 2023-10-21T04:49:45.739+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.740+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651251 controller mem 3828 2023-10-21T04:49:45.740+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.740+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651251 controller mem 3828 2023-10-21T04:49:45.740+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.740+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651251 controller mem 3828 2023-10-21T04:49:45.740+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.741+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651251 controller mem 3828 2023-10-21T04:49:45.741+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1055956, resident 1, memSz 70492, memSzIndex 368, numRecords 1000 2023-10-21T04:49:45.742+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651251 controller mem 3828 2023-10-21T04:49:45.743+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.743+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651207 controller mem 3828 2023-10-21T04:49:45.743+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.743+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651207 controller mem 3828 2023-10-21T04:49:45.744+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.744+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651207 controller mem 3828 2023-10-21T04:49:45.744+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.744+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651207 controller mem 3828 2023-10-21T04:49:45.744+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.745+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651207 controller mem 3828 2023-10-21T04:49:45.745+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.745+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651163 controller mem 3828 2023-10-21T04:49:45.746+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:49:45.746+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44651163 controller mem 3828 2023-10-21T04:49:45.746+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1724416] replayOffset [1716224] 2023-10-21T04:49:45.749+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [158.445771ms] 2023-10-21T04:49:45.749+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.749+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.750+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_10158339249566909515_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.750+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.750+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.751+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.751+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.752+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.752+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.752+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.753+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.753+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.754+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.754+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.754+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.754+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.755+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_4397940545762329850_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.755+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_12396348969790516872_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:49:45.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data], Shared [true]. Built [18] plasmas, took [165.398393ms] 2023-10-21T04:49:45.756+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: after recovery: resident ratio 0 requestQuota 1055956 memSz 880 2023-10-21T04:49:45.756+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 11 count: 1000 metaSz: 2505 rp [1] sn: 12 count: 1000 metaSz: 2516 2023-10-21T04:49:45.756+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: doInit: data UsedSpace 217350 recovery UsedSpace 22411 2023-10-21T04:49:45.762+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex 2023-10-21T04:49:45.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 2023-10-21T04:49:45.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data 2023-10-21T04:49:45.763+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex 2023-10-21T04:49:45.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex started 2023-10-21T04:49:45.763+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16653501928284257012 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:49:45.763+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:49:45.763+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:49:45.828+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 16653501928284257012 PartitionId 0 Warmup took 247.794349ms 2023-10-21T04:49:45.829+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 16653501928284257012 partitionId 0 WriterThreads 4 2023-10-21T04:49:45.829+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 16653501928284257012 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index Index: 16653501928284257012 Partition: 0 2023-10-21T04:49:45.830+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 16653501928284257012, shardIdMap: map[0:[18374611443167983589 6455624821495495854]] err: 2023-10-21T04:49:45.831+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 16653501928284257012, partnShardIdMap: map[0:[18374611443167983589 6455624821495495854]], err: 2023-10-21T04:49:45.831+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 16653501928284257012 Defn: DefnId: 6145407649125550089 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:45.832+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 16653501928284257012 2023-10-21T04:49:45.832+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16653501928284257012 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:49:45.832+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:49:45.832+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:45.832+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16653501928284257012 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:49:45.832+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 16653501928284257012 Defn: DefnId: 6145407649125550089 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:49:45.832+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 16653501928284257012 Partitions [0] 2023-10-21T04:49:45.833+05:30 [Info] StorageMgr::openSnapshot IndexInst:16653501928284257012 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:49:45.846+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 13.786072ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:45.846+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 101.343µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:49:45.858+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:45.858+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:45.863+05:30 [Info] switched currmeta from 427 -> 428 force true 2023-10-21T04:49:46.095+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:46.095+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:46.098+05:30 [Info] switched currmeta from 428 -> 429 force true 2023-10-21T04:49:46.120+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:46.205+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:6c:7c:ea:2e:d8:4:8b MasterId: SourceId: DestId: RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:46.251+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:49:46.251+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 20] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/a98493b6c7598b784e28f4b16eb254f8 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:a98493b6c7598b784e28f4b16eb254f8]}]} 2023-10-21T04:49:46.283+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:49:46.283+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 20] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:49:46.285+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/a98493b6c7598b784e28f4b16eb254f8, rev: [] 2023-10-21T04:49:46.285+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/a98493b6c7598b784e28f4b16eb254f8, rev: [] 2023-10-21T04:49:46.285+05:30 [Info] RebalanceServiceManager::cancelPrepareTaskLOCKED Initiating cleanup on rebalance follower 2023-10-21T04:49:46.285+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:49:46.286+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:49:46.286+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:49:46.330+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:46.330+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:46.345+05:30 [Info] switched currmeta from 429 -> 430 force true 2023-10-21T04:49:46.423+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:49:46.544+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:46.544+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:46.550+05:30 [Info] switched currmeta from 430 -> 434 force true 2023-10-21T04:49:46.603+05:30 [Info] ShardRebalancer::waitForIndexState Cancel Received 2023-10-21T04:49:46.635+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:49:46.745+05:30 [Info] indexer::CancelMergePartition. IndexStateMap map[0:Invalid] 2023-10-21T04:49:46.745+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken11:6c:7c:ea:2e:d8:4:8b MasterId: SourceId: DestId: RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:46.745+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken47:57:c0:39:1d:ee:e8:c9 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:5d:73:6f:a:7a:8f:89 Shards: [17327901024118572215 5629943257627822009] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 17085981505417849194 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 13917065492675748491 RealInstId: 16759912838651900043 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 5272948714421587967 RealInstId: 4186965058349029465 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 544693394183450628 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:46.746+05:30 [Info] RebalanceServiceManager::cleanupShardTokenForDest Skipping cleaning up token: ShardToken47:57:c0:39:1d:ee:e8:c9 on dest as ShardTokenDropOnSource is posted for this token 2023-10-21T04:49:46.746+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardToken47:5d:73:6f:a:7a:8f:89 MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken47:57:c0:39:1d:ee:e8:c9 Shards: [11114961608238406563 5748926106587204814] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 10558612751966480746 RealInstId: 0 Name: idx_secondary Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10113643360820949771 RealInstId: 9316375249999768625 Name: idx_partitioned Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 13828149546090165956 RealInstId: 3751526798537645569 Name: idx_partitioned_defer Bucket: bucket_1 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 18137927630988528516 RealInstId: 0 Name: #primary Bucket: bucket_1 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:46.746+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenbc:ff:3d:f:5b:9b:8b:6b MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 8497280533643391851 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 1856681538226480532 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 15262821106108281777 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:46.746+05:30 [Info] RebalanceServiceManager::cleanupShardTransferTokens Cleaning Up ShardTokenb1:70:f2:cb:a5:ff:fd:1d MasterId: ba6304988a6335d65281381d6cfc5e32 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: a98493b6c7598b784e28f4b16eb254f8 ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move Error: Shard rebalance cancel received SiblingTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 16653501928284257012 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 3266194727357921673 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 4523737201364197125 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:49:46.746+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: a98493b6c7598b784e28f4b16eb254f8 TransferTokenId: ShardTokenb1:70:f2:cb:a5:ff:fd:1d Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:49:46.746+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000117342 2023-10-21T04:49:46.746+05:30 [Info] RebalanceServiceManager::cleanupLocalIndexInstsAndShardToken Cleaning up for ttid: ShardTokenb1:70:f2:cb:a5:ff:fd:1d, dropIndexes: true 2023-10-21T04:49:46.746+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d, shardId :0 2023-10-21T04:49:46.747+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenb1_70_f2_cb_a5_ff_fd_1d, shardId :0 2023-10-21T04:49:46.768+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 16653501928284257012 &{1} 2023-10-21T04:49:46.768+05:30 [Info] Indexer::handleDropIndex - IndexInstId 16653501928284257012 2023-10-21T04:49:46.768+05:30 [Info] Indexer::handleDropIndex Allowing drop index during rebalance for index defnId: 6145407649125550089, instId: 16653501928284257012 2023-10-21T04:49:46.768+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [16653501928284257012] 2023-10-21T04:49:46.768+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [16653501928284257012] 2023-10-21T04:49:46.768+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:46.768+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [16653501928284257012] 2023-10-21T04:49:46.768+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 16653501928284257012 Defn: DefnId: 6145407649125550089 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:49:46.769+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 915.005µs 2023-10-21T04:49:46.769+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 955.298µs 2023-10-21T04:49:46.769+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16653501928284257012, PartitionId 0, IndexDefnId 6145407649125550089. 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:46.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex] 2023-10-21T04:49:46.769+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex] 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex] 2023-10-21T04:49:46.769+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex 2023-10-21T04:49:46.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex stopped 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:46.769+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:46.769+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 16653501928284257012 2023-10-21T04:49:46.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:46.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex] 2023-10-21T04:49:46.775+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex closed 2023-10-21T04:49:46.775+05:30 [Info] bucket_%2/idx_secondary/Mainstore#16653501928284257012:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:46.776+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:46.776+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:46.776+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:46.776+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex] 2023-10-21T04:49:46.776+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex 2023-10-21T04:49:46.776+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex] 2023-10-21T04:49:46.800+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 53.262825ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:46.805+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 4390029557575378783 instance id 3266194727357921673 real instance id 4397940545762329850 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:49:46.805+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 169.696µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:46.806+05:30 [Info] LifecycleMgr.DeleteOrPruneIndexInstance() : index defnId 17703248484023971553 instance id 4523737201364197125 real instance id 12396348969790516872 partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:49:46.806+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_OR_PRUNE_INSTANCE elapsed 111.039µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:46.807+05:30 [Info] LifecycleMgr.DeleteIndex() : index 5277316367626246544 does not exist. 2023-10-21T04:49:46.807+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX_REBAL elapsed 72.603µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:46.807+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage processing command: Message: MsgDestroyLocalShardData ShardIds: [18374611443167983589 6455624821495495854] 2023-10-21T04:49:46.834+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex] 2023-10-21T04:49:46.834+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex 2023-10-21T04:49:46.834+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex stopped 2023-10-21T04:49:46.834+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:46.834+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:46.834+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:46.834+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:46.840+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:46.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex] 2023-10-21T04:49:46.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex closed 2023-10-21T04:49:46.840+05:30 [Info] bucket_%2/idx_secondary/Backstore#16653501928284257012:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:46.840+05:30 [Info] Indexer::cleanupIndexData IndexInst 16653501928284257012 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:46.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex ... 2023-10-21T04:49:46.840+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex ... 2023-10-21T04:49:46.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : metadata saved successfully 2023-10-21T04:49:46.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/mainIndex sucessfully destroyed 2023-10-21T04:49:46.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex ... 2023-10-21T04:49:46.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex ... 2023-10-21T04:49:46.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : metadata saved successfully 2023-10-21T04:49:46.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_16653501928284257012_0.index destroyed 2023-10-21T04:49:46.855+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16653501928284257012, PartitionId 0, IndexDefnId 6145407649125550089. 2023-10-21T04:49:46.855+05:30 [Info] Indexer::cleanupIndexData IndexInst 16653501928284257012 Partition 0 Destroy Done 2023-10-21T04:49:47.101+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:47.101+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:47.106+05:30 [Info] switched currmeta from 434 -> 441 force true 2023-10-21T04:49:47.116+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:47.117+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:47.122+05:30 [Info] switched currmeta from 441 -> 441 force true 2023-10-21T04:49:47.187+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:49:47.305+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:47.315+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:47.315+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:47.319+05:30 [Info] switched currmeta from 441 -> 441 force true 2023-10-21T04:49:47.392+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[544693394183450628:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:49:47.392+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[544693394183450628:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:49:47.392+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[544693394183450628:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:49:47.541+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18137927630988528516:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:49:47.542+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18137927630988528516:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:49:47.542+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[18137927630988528516:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:49:47.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9890041212502027, throttleDelayMs (new, old, change): (4, 0, 4) 2023-10-21T04:49:47.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 18374611443167983589 2023-10-21T04:49:47.694+05:30 [Info] ShardTransferManager::updateSliceStatus Closing the notifyCh for shardId: 6455624821495495854 2023-10-21T04:49:47.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 6455624821495495854 2023-10-21T04:49:47.694+05:30 [Info] ShardTranferManager::waitForSliceClose - Exiting wait as all slices are closed for shard: 18374611443167983589 2023-10-21T04:49:47.694+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage All slices closed. Initiating shard destroy for shards: [18374611443167983589 6455624821495495854], elapsed: 886.708099ms 2023-10-21T04:49:47.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:49:47.694+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:49:47.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper stopped 2023-10-21T04:49:47.694+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': stopped 2023-10-21T04:49:47.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:49:47.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:49:47.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:49:47.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:49:47.694+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:49:47.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:49:47.694+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:49:47.697+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:49:47.697+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:49:47.697+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:49:47.697+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:49:47.697+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:49:47.697+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Begin close 2023-10-21T04:49:47.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.Close: Done close 2023-10-21T04:49:47.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:49:47.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:49:47.699+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589/data/recovery(shard18374611443167983589) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:49:47.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:49:47.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shutdown completed 2023-10-21T04:49:47.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:49:47.699+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589 (18374611443167983589) removed 2023-10-21T04:49:47.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All daemons stopped 2023-10-21T04:49:47.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance closed 2023-10-21T04:49:47.699+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : All instance destroyed 2023-10-21T04:49:47.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard18374611443167983589(18374611443167983589)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:49:47.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:49:47.701+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:49:47.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:49:47.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:49:47.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: Begin stopping daemons 2023-10-21T04:49:47.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.stopDaemons: all deamons stopped 2023-10-21T04:49:47.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:49:47.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:49:47.701+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:49:47.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:49:47.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:49:47.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:49:47.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:49:47.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Begin cleanup 2023-10-21T04:49:47.705+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Begin close 2023-10-21T04:49:47.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.Close: Done close 2023-10-21T04:49:47.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: Begin cleaner cleanup 2023-10-21T04:49:47.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanupLSSCleaner: LSSCleaner stopped 2023-10-21T04:49:47.707+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854/data/recovery(shard6455624821495495854) : LSSCtx.cleanup: Done cleanup 2023-10-21T04:49:47.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:49:47.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shutdown completed 2023-10-21T04:49:47.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : shard removed from ShardMgr 2023-10-21T04:49:47.707+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854 (6455624821495495854) removed 2023-10-21T04:49:47.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All daemons stopped 2023-10-21T04:49:47.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance closed 2023-10-21T04:49:47.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : All instance destroyed 2023-10-21T04:49:47.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard6455624821495495854(6455624821495495854)(0-0-0) : Shard Destroyed Successfully 2023-10-21T04:49:47.709+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Cleaned up Tenant with name:bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:49:47.709+05:30 [Info] ShardTransferManager::processDestroyLocalShardMessage Done clean-up for shards: [18374611443167983589 6455624821495495854], elapsed(sec): 0.901419783 2023-10-21T04:49:47.721+05:30 [Info] RebalanceServiceManager::cleanupAllDropOnSourceTokens Cleaning up transferred data for ttid: ShardTokenbc:ff:3d:f:5b:9b:8b:6b 2023-10-21T04:49:47.721+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Initiating command: RebalanceId: a98493b6c7598b784e28f4b16eb254f8 TransferTokenId: ShardTokenbc:ff:3d:f:5b:9b:8b:6b Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir Region: 2023-10-21T04:49:47.722+05:30 [Info] ShardTransferManager::processTransferCleanupMessage Clean-up initiated for all shards, elapsed(sec): 0.000142513 2023-10-21T04:49:47.722+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:49:47.722+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 3 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 4 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 5 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 6 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 7 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 8 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 1 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 16759912838651900043, partnId: 2 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 4 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 5 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 6 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 7 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 8 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 1 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 2 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 4186965058349029465, partnId: 3 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 544693394183450628, partnId: 0 2023-10-21T04:49:47.722+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17085981505417849194, partnId: 0 2023-10-21T04:49:47.722+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.841µs 2023-10-21T04:49:47.722+05:30 [Info] plasma CleanupContext : Cleaning up storage path :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenbc_ff_3d_f_5b_9b_8b_6b, shardId :0 2023-10-21T04:49:47.722+05:30 [Info] plasma CleanupContext : Cleaned up shard data at location :/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/a98493b6c7598b784e28f4b16eb254f8_ShardTokenbc_ff_3d_f_5b_9b_8b_6b, shardId :0 2023-10-21T04:49:47.722+05:30 [Info] ShardTransferManager::handleRestoreAndUnlockShards Initiating RestoreShardDone for shardId: 5629943257627822009 2023-10-21T04:49:47.722+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : all deamons started 2023-10-21T04:49:47.722+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : all deamons started 2023-10-21T04:49:47.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:49:47.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:49:47.722+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : RestoreShardDone completed 2023-10-21T04:49:47.722+05:30 [Info] ShardTransferManager::handleRestoreAndUnlockShards Initiating unlock for shardId: 5629943257627822009 2023-10-21T04:49:47.722+05:30 [Info] ShardTransferManager::handleRestoreAndUnlockShards Initiating RestoreShardDone for shardId: 17327901024118572215 2023-10-21T04:49:47.722+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : all deamons started 2023-10-21T04:49:47.722+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : all deamons started 2023-10-21T04:49:47.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:49:47.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:49:47.723+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : RestoreShardDone completed 2023-10-21T04:49:47.723+05:30 [Info] ShardTransferManager::handleRestoreAndUnlockShards Initiating unlock for shardId: 17327901024118572215 2023-10-21T04:49:47.723+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:49:47.723+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:49:47.723+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data(shard5629943257627822009) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery(shard5629943257627822009) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data(shard17327901024118572215) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:49:47.724+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery(shard17327901024118572215) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:49:47.737+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:49:47.737+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:49:47.737+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:49:47.737+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:49:47.737+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 35.367µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:47.739+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:49:47.739+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:49:47.739+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:49:47.739+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:49:47.739+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:49:47.739+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:49:47.739+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:49:47.739+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 31.72µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:47.749+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:49:47.749+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:49:47.749+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.463231094s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:49:47.749+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/a98493b6c7598b784e28f4b16eb254f8, rev: [] call. err: 2023-10-21T04:49:47.749+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 20] call. taskList: &{Rev:[0 0 0 0 0 0 0 21] Tasks:[]} 2023-10-21T04:49:47.749+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 21] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:49:47.753+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:49:47.753+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 21] Tasks:[]} 2023-10-21T04:49:47.754+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:49:47.754+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 21] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:49:47.755+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 21] 2023-10-21T04:49:47.756+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 21] 2023-10-21T04:49:48.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7292339473902664, throttleDelayMs (new, old, change): (0, 4, -4) 2023-10-21T04:49:49.309+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:49.309+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:49.325+05:30 [Info] switched currmeta from 441 -> 443 force true 2023-10-21T04:49:49.353+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:49.353+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:49.356+05:30 [Info] switched currmeta from 443 -> 443 force true 2023-10-21T04:49:50.129+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:50.135+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:50.135+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:50.137+05:30 [Info] switched currmeta from 443 -> 443 force true 2023-10-21T04:49:50.449+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:34726 2023-10-21T04:49:50.452+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:49:50.477+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 23.906395ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:50.501+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:49:50.508+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:34726. Reason = EOF. Cleaning Pipe. 2023-10-21T04:49:50.510+05:30 [Info] messageListener.start(): message channel closed. Remove peer 34:c1:80:d8:35:f4:36:5e as follower. 2023-10-21T04:49:50.978+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:49:51.335+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.335+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.342+05:30 [Info] switched currmeta from 443 -> 444 force true 2023-10-21T04:49:51.358+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.358+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.366+05:30 [Info] switched currmeta from 444 -> 446 force true 2023-10-21T04:49:51.416+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.416+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.425+05:30 [Info] switched currmeta from 446 -> 446 force true 2023-10-21T04:49:51.435+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.436+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.439+05:30 [Info] switched currmeta from 446 -> 446 force true 2023-10-21T04:49:51.458+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.458+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.462+05:30 [Info] switched currmeta from 446 -> 447 force true 2023-10-21T04:49:51.509+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:49:51.525+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.525+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.531+05:30 [Info] switched currmeta from 447 -> 448 force true 2023-10-21T04:49:51.572+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.572+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.575+05:30 [Info] switched currmeta from 448 -> 448 force true 2023-10-21T04:49:51.695+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.695+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.697+05:30 [Info] switched currmeta from 448 -> 449 force true 2023-10-21T04:49:51.794+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.795+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.799+05:30 [Info] switched currmeta from 449 -> 450 force true 2023-10-21T04:49:51.885+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.885+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.888+05:30 [Info] switched currmeta from 450 -> 450 force true 2023-10-21T04:49:51.923+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:51.923+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:51.936+05:30 [Info] switched currmeta from 450 -> 451 force true 2023-10-21T04:49:52.584+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:49:53.591+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:49:55.144+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:49:55.150+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:55.150+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:55.152+05:30 [Info] switched currmeta from 451 -> 451 force true 2023-10-21T04:49:56.713+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swap is enabled 2023-10-21T04:49:59.859+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 17085981505417849194 &{0} 2023-10-21T04:49:59.869+05:30 [Info] Indexer::handleDropIndex - IndexInstId 17085981505417849194 2023-10-21T04:49:59.870+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:49:59.870+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:59.870+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17085981505417849194 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:49:59.871+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [17085981505417849194] 2023-10-21T04:49:59.871+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [17085981505417849194] 2023-10-21T04:49:59.871+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:49:59.871+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [17085981505417849194] 2023-10-21T04:49:59.871+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:59.872+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17085981505417849194, PartitionId 0, IndexDefnId 16367703496921140088. 2023-10-21T04:49:59.872+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:59.872+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:59.872+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:59.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex] 2023-10-21T04:49:59.872+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex 2023-10-21T04:49:59.872+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex] 2023-10-21T04:49:59.872+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex] 2023-10-21T04:49:59.872+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex 2023-10-21T04:49:59.872+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex stopped 2023-10-21T04:49:59.872+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:59.872+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:59.872+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:59.872+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:59.871+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 17085981505417849194 Defn: DefnId: 16367703496921140088 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:49:59.872+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 2.879495ms 2023-10-21T04:49:59.872+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 17085981505417849194 2023-10-21T04:49:59.873+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [17085981505417849194] 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:59.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex] 2023-10-21T04:49:59.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex closed 2023-10-21T04:49:59.876+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:49:59.876+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51546 2023-10-21T04:49:59.876+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swap is disabled 2023-10-21T04:49:59.876+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 3.889368ms 2023-10-21T04:49:59.876+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 6.835054ms 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:49:59.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex] 2023-10-21T04:49:59.876+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex 2023-10-21T04:49:59.876+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex] 2023-10-21T04:49:59.876+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex] 2023-10-21T04:49:59.876+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex 2023-10-21T04:49:59.876+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex stopped 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:49:59.876+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:49:59.890+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:49:59.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex] 2023-10-21T04:49:59.890+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex closed 2023-10-21T04:49:59.890+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:49:59.891+05:30 [Info] Indexer::cleanupIndexData IndexInst 17085981505417849194 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:49:59.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex ... 2023-10-21T04:49:59.891+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex ... 2023-10-21T04:49:59.894+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 55.30854ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:49:59.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:49:59.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/mainIndex sucessfully destroyed 2023-10-21T04:49:59.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex ... 2023-10-21T04:49:59.896+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 24 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex ... 2023-10-21T04:49:59.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:49:59.898+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_17085981505417849194_0.index destroyed 2023-10-21T04:49:59.899+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17085981505417849194, PartitionId 0, IndexDefnId 16367703496921140088. 2023-10-21T04:49:59.899+05:30 [Info] Indexer::cleanupIndexData IndexInst 17085981505417849194 Partition 0 Destroy Done 2023-10-21T04:49:59.982+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:49:59.982+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:49:59.985+05:30 [Info] switched currmeta from 451 -> 455 force true 2023-10-21T04:50:00.005+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:00.005+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:00.009+05:30 [Info] switched currmeta from 455 -> 455 force true 2023-10-21T04:50:00.168+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:50:00.173+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:00.173+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:00.175+05:30 [Info] switched currmeta from 455 -> 455 force true 2023-10-21T04:50:03.111+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 244] aggregate storage info: map[bucket_%2:map[kv:2472128 index:17703407] bucket_1:map[kv:2541760 index:14762138]] 2023-10-21T04:50:03.761+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:50:11.129+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 16364390102182676976 2023-10-21T04:50:11.129+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 171.788µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:11.624+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48392 2023-10-21T04:50:11.659+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48392 2023-10-21T04:50:11.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9995087696310858, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:50:11.839+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48392 2023-10-21T04:50:12.118+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 16364390102182676976, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:50:12.118+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 16364390102182676976, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:50:12.152+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 14033078509407174050, indexDefn DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:50:12.156+05:30 [Info] Indexer::handleCreateIndex InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:50:12.161+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 14033078509407174050 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:50:12.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:50:12.171+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:50:12.173+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:12.173+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:50:12.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:50:12.181+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:50:12.181+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:12.182+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:50:12.184+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex 2023-10-21T04:50:12.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:50:12.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:50:12.185+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex 2023-10-21T04:50:12.185+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex 2023-10-21T04:50:12.185+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex started 2023-10-21T04:50:12.185+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:12.186+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:12.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex 2023-10-21T04:50:12.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:50:12.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:50:12.189+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex 2023-10-21T04:50:12.189+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex 2023-10-21T04:50:12.189+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex started 2023-10-21T04:50:12.190+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 14033078509407174050 partitionId 0 WriterThreads 4 2023-10-21T04:50:12.190+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 14033078509407174050 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index Index: 14033078509407174050 Partition: 0 2023-10-21T04:50:12.190+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 14033078509407174050 2023-10-21T04:50:12.190+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:12.190+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:50:12.190+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:50:12.190+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:12.190+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 34.085075ms 2023-10-21T04:50:12.190+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 34.123982ms 2023-10-21T04:50:12.190+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:12.190+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:12.190+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 14033078509407174050, partnShardIdMap: map[0:[17327901024118572215 5629943257627822009]], indexDefn DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:50:12.212+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [14033078509407174050] &{0} 2023-10-21T04:50:12.212+05:30 [Info] Indexer::handleBuildIndex [14033078509407174050] 2023-10-21T04:50:12.215+05:30 [Info] bulkUpdateRState: Index instance 14033078509407174050 rstate moved to ACTIVE 2023-10-21T04:50:12.215+05:30 [Info] Indexer::handleBuildIndex Added Index: [14033078509407174050] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:50:12.215+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:12.215+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:50:12.215+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:12.215+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:50:12.215+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:50:12.215+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:50:12.215+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91310 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:50:12.215+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:50:12.215+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:50:12.215+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:50:12.215+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91310 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:50:12.215+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:50:12.215+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:50:12.216+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:50:12.216+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:50:12.216+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:50:12.216+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:50:12.216+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:50:12.216+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:50:12.216+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:3] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:50:12.216+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 14033078509407174050 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:50:12.216+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:50:12.216+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=18,1=22,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=23,12=23,13=24,14=19,15=25,16=24,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=25,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=18,46=21,47=28,48=20,49=23,50=20,51=27,52=18,53=30,54=22,55=25,56=22,57=27,58=25,59=21,60=24,61=24,62=17,63=24] IndexList: [ InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 91311 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:50:12.217+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:50:12.217+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:50:12.218+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:50:12.218+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 6.518891ms 2023-10-21T04:50:12.218+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 6.561533ms 2023-10-21T04:50:12.218+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [14033078509407174050] 2023-10-21T04:50:12.226+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:50:12.264+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:50:12.270+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:12.270+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:12.272+05:30 [Info] switched currmeta from 455 -> 462 force true 2023-10-21T04:50:12.272+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 16364390102182676976 2023-10-21T04:50:12.272+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 154.746861ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:12.281+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.354143ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:12.433+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:12.433+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:12.464+05:30 [Info] switched currmeta from 462 -> 464 force true 2023-10-21T04:50:12.506+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:36092" doAuth() ... 2023-10-21T04:50:12.506+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:36092" auth successful 2023-10-21T04:50:12.506+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:36092" +1 2023-10-21T04:50:12.592+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:12.592+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:12.607+05:30 [Info] switched currmeta from 464 -> 464 force true 2023-10-21T04:50:12.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996503021049864, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:50:12.686+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:50:12.686+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:50:12.699+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:50:12.790+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:50:12.790+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:50:12.790+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:50:12.897+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:50:13.057+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:50:13.057+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:50:13.057+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:50:13.101+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:50:13.166+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:50:13.166+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:50:13.167+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:50:13.173+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35376 2023-10-21T04:50:13.173+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:50:13.195+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35378 2023-10-21T04:50:13.214+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 13.161362ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:13.219+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 39.224759ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:13.219+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:50:13.219+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:50:13.219+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:50:13.219+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:50:13.219+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:50:13.219+05:30 [Info] DDLServiceMgr: processing create index token /indexing/ddl/commandToken/create/16364390102182676976/0 2023-10-21T04:50:13.237+05:30 [Info] DDLServiceMgr: Update Replica Count. Index Defn 16364390102182676976 replica Count {true 1 0 0} 2023-10-21T04:50:13.238+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 35.523µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:13.255+05:30 [Info] DDLServiceMgr: Remove create index token 16364390102182676976. 2023-10-21T04:50:13.255+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:50:13.255+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:50:13.255+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:50:13.255+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:50:13.255+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:50:13.256+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:54848->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:13.256+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:55314->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:13.256+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:35376->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:13.256+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:59960->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:13.257+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35376. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:13.257+05:30 [Info] messageListener.start(): message channel closed. Remove peer a0:8:1b:5a:d:54:c2:1c as follower. 2023-10-21T04:50:13.294+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35378. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:13.294+05:30 [Info] messageListener.start(): message channel closed. Remove peer 64:66:3f:47:15:2f:13:d4 as follower. 2023-10-21T04:50:13.432+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35390 2023-10-21T04:50:13.494+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 57.198681ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:13.526+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35390. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:13.526+05:30 [Info] messageListener.start(): message channel closed. Remove peer aa:2e:ba:48:7f:34:b6:c6 as follower. 2023-10-21T04:50:13.553+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35404 2023-10-21T04:50:13.614+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 54.586878ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:13.644+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35404. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:13.644+05:30 [Info] messageListener.start(): message channel closed. Remove peer 1:eb:2b:b1:c4:81:f4:35 as follower. 2023-10-21T04:50:13.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9978215941881183 2023-10-21T04:50:13.716+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:13.720+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:13.995+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:14.043+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 67576817345502 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.047+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 39196762516158 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.054+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 95570275214302 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.054+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 44231811555836 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.056+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 4065155603859 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.059+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 91960699361891 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.059+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 77783296532664 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.062+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 233597919143467 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 143637217187246 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 197901620499212 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 33320884322178 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 129413110409107 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 95215784105904 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.074+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 147449900650979 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.074+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 198938078557080 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.074+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 132000914750455 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.074+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 27353924173936 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.074+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 79253753007122 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.074+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 122353375037127 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.083+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 25078414014136 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.093+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 215732063158788 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.093+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 101186302195296 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.095+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 113904165092305 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.095+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 230675090583633 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.097+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 7646518130509 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.106+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 201780742117921 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 232497361012987 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 131315346703752 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.108+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 61904892722472 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.116+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:14.117+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 248212138718363 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.117+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 241043521322196 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.126+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 173020619068939 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.127+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 258788849496338 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.127+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 201745596055144 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 19399479457331 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 190878005091193 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 63194637348236 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.130+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 280972182784873 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.135+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 13155294694475 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.135+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 119373194724283 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.135+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 186283152043223 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.140+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 209574230498887 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.141+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 277109307527403 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.141+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 270106388647522 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.152+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 193080054939721 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.153+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 49352389572227 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.153+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 62156602796507 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.153+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 83754344146634 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.158+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 212804514607186 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.159+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 255609313786736 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.159+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 80989250905985 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.159+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 279208201382660 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.163+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 102294135990395 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.165+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 204579891639872 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.165+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 218103594015078 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.166+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 150369066093368 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.166+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 152904045160406 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.168+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 65071891036109 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.172+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 35015089382353 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.174+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 90104317736129 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.177+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 223555064174984 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.179+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 184273998690001 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.179+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 257224089448052 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.181+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 202830985870043 0 91311 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:50:14.195+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_default:c1 InstanceIds [14033078509407174050] 2023-10-21T04:50:14.195+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 91311 2023-10-21T04:50:14.195+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:50:14.195+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:50:14.195+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:50:14.195+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:50:14.195+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:50:14.195+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:50:14.201+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=18,1=22,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=23,12=23,13=24,14=19,15=25,16=24,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=25,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=18,46=21,47=28,48=20,49=23,50=20,51=27,52=18,53=30,54=22,55=25,56=22,57=27,58=25,59=21,60=24,61=24,62=17,63=24] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:50:14.326+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swap is enabled 2023-10-21T04:50:14.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.952976952976953 2023-10-21T04:50:14.879+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:14.879+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:14.885+05:30 [Info] switched currmeta from 464 -> 465 force true 2023-10-21T04:50:14.917+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:50:14.929+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14033078509407174050 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:50:14.929+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14033078509407174050 PartitionId: 0 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.852µs SnapOpenDur 125.512µs 2023-10-21T04:50:14.929+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14033078509407174050, PartitionId 0 SnapshotId 20 Creating recovery point ... 2023-10-21T04:50:14.929+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 14033078509407174050 INDEX_STATE_CATCHUP 2023-10-21T04:50:14.930+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 14033078509407174050 true 2023-10-21T04:50:14.930+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 14033078509407174050 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 91311 BuildTS: [0=18,1=22,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=23,12=23,13=24,14=19,15=25,16=24,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=25,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=18,46=21,47=28,48=20,49=23,50=20,51=27,52=18,53=30,54=22,55=25,56=22,57=27,58=25,59=21,60=24,61=24,62=17,63=24] 2023-10-21T04:50:14.930+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 91311 2023-10-21T04:50:14.930+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 91311 2023-10-21T04:50:14.930+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:14.930+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:50:14.930+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:14.930+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:50:14.931+05:30 [Info] send progress stats to clients 2023-10-21T04:50:14.931+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:50:14.931+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:50:14.931+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:4] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:50:14.931+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 91311 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:50:14.935+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:50:14.942+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_1 2023-10-21T04:50:14.943+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91311 2023-10-21T04:50:14.943+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91311 2023-10-21T04:50:14.943+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:50:14.943+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:50:14.943+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 14033078509407174050 false 2023-10-21T04:50:14.943+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 14033078509407174050 2023-10-21T04:50:14.943+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:50:14.943+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 91311 2023-10-21T04:50:14.941+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.031637ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:14.950+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:14.950+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:14.952+05:30 [Info] switched currmeta from 465 -> 466 force true 2023-10-21T04:50:14.970+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14033078509407174050, PartitionId 0 SnapshotId 20 Created recovery point (took 40.88623ms) 2023-10-21T04:50:15.196+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:50:15.196+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:50:15.196+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 14033078509407174050 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 91311, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 31 0 31} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 0 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 0 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 35 0 35} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 20 0 20} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 27 0 27} { 46 4c2c93ee66ca 2 23 0 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 0 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 26 0 26} { 56 8b10bf4be7d6 2 26 0 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} 2023-10-21T04:50:15.196+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 14033078509407174050 INDEX_STATE_ACTIVE 2023-10-21T04:50:15.196+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:50:15.196+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:50:15.196+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:50:15.196+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 47 CurrMaint 0 New 0 2023-10-21T04:50:15.196+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 91311 2023-10-21T04:50:15.196+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:15.196+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:50:15.197+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:50:15.197+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91311 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:50:15.197+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 14033078509407174050 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:50:15.197+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:50:15.197+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:50:15.197+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:50:15.197+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91311 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:50:15.197+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:50:15.197+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:11 1:15 2:16 3:12 4:16 5:18 6:16 7:21 8:14 9:21 10:15 11:16 12:16 13:17 14:12 15:18 16:17 17:11 18:14 19:14 20:25 21:17 22:11 23:16 24:14 25:11 26:13 27:14 28:12 29:17 30:10 31:18 32:16 33:13 34:14 35:14 36:17 37:20 38:18 39:19 40:11 41:17 42:16 43:18 44:16 45:11 46:14 47:21 48:13 49:16 50:13 51:20 52:11 53:23 54:15 55:18 56:15 57:20 58:18 59:14 60:17 61:17 62:10 63:17] 2023-10-21T04:50:15.197+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:18 1:22 2:23 3:19 4:23 5:25 6:23 7:28 8:21 9:28 10:22 11:23 12:23 13:24 14:19 15:25 16:24 17:18 18:21 19:21 20:32 21:24 22:18 23:23 24:21 25:18 26:20 27:21 28:19 29:24 30:17 31:25 32:23 33:20 34:21 35:21 36:24 37:27 38:25 39:26 40:18 41:24 42:23 43:25 44:23 45:18 46:21 47:28 48:20 49:23 50:20 51:27 52:18 53:30 54:22 55:25 56:22 57:27 58:25 59:21 60:24 61:24 62:17 63:24] 2023-10-21T04:50:15.197+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:50:15.197+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:36092: use of closed network connection 2023-10-21T04:50:15.197+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:36092" exit: read tcp4 127.0.0.1:9121->127.0.0.1:36092: use of closed network connection 2023-10-21T04:50:15.197+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:36092" closed ! 2023-10-21T04:50:15.197+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:50:15.198+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:36092" already gone 2023-10-21T04:50:15.198+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:50:15.198+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:50:15.198+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:50:15.198+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:50:15.198+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:50:15.198+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:50:15.198+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 91311 2023-10-21T04:50:15.198+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:50:15.209+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.057168ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:15.210+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:50:15.211+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:50:15.211+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:50:15.211+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91311 2023-10-21T04:50:15.211+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91311 2023-10-21T04:50:15.211+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:50:15.232+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:15.232+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:15.236+05:30 [Info] switched currmeta from 466 -> 467 force true 2023-10-21T04:50:15.278+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:15.278+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:15.286+05:30 [Info] switched currmeta from 467 -> 468 force true 2023-10-21T04:50:15.378+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:50:15.383+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:15.383+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:15.386+05:30 [Info] switched currmeta from 468 -> 468 force true 2023-10-21T04:50:15.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.7327906827858812, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:50:16.041+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10417104488879353350:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:50:16.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14033078509407174050:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:50:16.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14033078509407174050:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:50:16.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14033078509407174050:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:50:16.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.4957759623745514, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:50:17.755+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 21] call. err: operation_canceled 2023-10-21T04:50:17.756+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:50:17.756+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 21] Tasks:[]} 2023-10-21T04:50:17.756+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:50:17.756+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 21] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:50:17.757+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 21] 2023-10-21T04:50:17.758+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 21] 2023-10-21T04:50:24.535+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:24.535+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:24.544+05:30 [Info] switched currmeta from 468 -> 472 force true 2023-10-21T04:50:24.557+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:24.557+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:24.559+05:30 [Info] switched currmeta from 472 -> 472 force true 2023-10-21T04:50:25.383+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:50:25.389+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:25.389+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:25.391+05:30 [Info] switched currmeta from 472 -> 472 force true 2023-10-21T04:50:30.712+05:30 [Info] Periodic Aggregated StorageStats: { "num_tenant": 1, "num_controller": 1, "num_shard": 2, "num_shard_altid": 0, "num_instances": 35, "num_lssctx": 4, "num_sctxs": 73, "num_active_sctxs": 16, "num_free_sctxs": 57, "num_trim_sctxs": 2, "num_wctxs": 155, "num_free_wctxs": 120, "num_active_wctxs": 35, "num_trim_wctxs": 0, "num_readers": 9, "num_writers": 8, "cpu": 0.06999, "total_core": 4, "threads": 19, "avail_thread_barrier": 8000, "assigned_quota": 1415577600, "current_quota": 1415577600, "tenant_quota_assigned": 9002571, "tenant_quota_mandatory": 9002571, "tenant_quota_discretionary": 0, "tenant_quota_mutation": 8976375, "tenant_working_set": 0, "heap_limit": max, "memory_resident": 229163008, "memory_jemalloc": 14532608, "memory_jemalloc_active": 8146944, "memory_jemalloc_allocated": 7299200, "memory_jemalloc_dirty": 1548288, "estimated_mem_frag": 101.98488, "memory_stats_size_total": 712351, "memory_stats_size_global": 141114, "memory_stats_size_page": 136539, "memory_stats_size_index": 1183, "memory_stats_size_bloom": 0, "memory_stats_size_delta": 22304, "reclaim_pending": 416438, "reclaim_pending_global": 416438, "buf_memused": 158191, "lss_fbuf_size": 8388608, "under_swap": false, "num_freeOS": 0, "num_smr_ctx ": 1, "num_smr_worker ": 8, "num_active_smr_worker ": 0, "num_smr_queued ": 0, "num_total_smr": 35, "num_active_smr": 0, "num_pending_smr": 0, "num_pages": 49, "items_count": 2006, "total_records": 2018, "avg_item_size": 49, "total_ops": 2002, "compacts": 47, "purges": 0, "merges_emptypages": 0, "lss_data_size": 749590, "lss_used_space": 7184384, "lss_disk_size": 7184384, "lss_open_filedesc": 8, "max_open_filedesc": 200000, "lss_fragmentation": 89, "lss_num_reads": 33, "lss_read_bs": 1879177, "lss_blk_read_bs": 815104, "rlss_num_reads": 8, "lss_rdr_reads_bs": 288, "lss_blk_rdr_reads_bs": 32768, "lookup_num_reads": 17, "lookup_reads_bs": 2433, "lookup_blk_reads_bs": 69632, "bytes_written": 1298432, "bytes_incoming": 100724, "write_amp_avg": 6.76, "num_swapper_ctx": 1, "avg_sweep_interval": 5m0s, "avg_run_interval": 100ms, "avg_clock_interval": 0s, "num_swapperWorker": 2, "cached_records": 2002, "resident_ratio": 0.99207, "num_rec_compressed": 0, "num_rec_compressible": 1686, "compressed_percent": 0, "compressed_compressible_perc": 0, "compression_ratio": 3.50370, "num_burst_visits": 0, "num_periodic_visits": 0, "num_evicted": 0, "num_evictable": 0, "cleaner_num_reads": 0, "cleaner_read_bs": 0, "cleaner_blk_read_bs": 0, "recovery_num_lssctx": 2, "recovery_num_sctxs": 10, "recovery_num_active_sctxs": 8, "recovery_num_free_sctxs": 2, "recovery_num_trim_sctxs": 2, "recovery_buf_memused": 0, "recovery_fbuf_size": 4194304, "recovery_data_size": 362755, "recovery_used_space": 3379200, "recovery_disk_size": 3379200, "recovery_lss_open_filedesc": 4, "recovery_lss_blk_read_bs": 663552, "recovery_lss_read_bs": 1847285, "recovery_lss__num_reads": 0, "recovery_bytes_written": 618496, "recovery_lss_frag": 89, "recovery_cleaner_num_reads": 0, "recovery_cleaner_read_bs": 0, "recovery_cleaner_blk_read_bs": 0, "mvcc_purge_ratio": 1.00598, "reader_purge_count": 0, "reader_emptypage_count": 344, "fbuf_tuner_status": , "active_copy_ctx": 0, "num_shard_upload": 0 (0), "num_shard_restored": 22 (22), "upload_bytes": 0, "avg_upload_bw": 0, "download_bytes": 44851295, "avg_download_bw": 452884077, "num_upload_cleanups": 19 (19), "num_staging_cleanups": 0 (0) } 2023-10-21T04:50:30.713+05:30 [Info] Periodic TenantStats: { "bucket_1_c25fee48adbe357b2d711be58a1deb67": { "num_shards": 2, "num_instances": 35, "num_controllers": 1, "quota": 9002571, "mandatory_quota": 9002571, "max_mandatory_quota": 9002571, "discretionary_quota": 0, "resident_quota": 26196, "mutation_quota": 8976375, "max_mutation_quota": 8976375, "idle_quota": 22854, "derived_quota": 4448, "working_set": 0, "mem_in_use": 137722, "mem_index": 1183, "total_records": 2018, "avg_key_size": 49, "avg_mem_per_key": 68, "resident_ratio": 0.99158, "idle": false, "total_ops": 2616, "read_ops": 609, "new_ops": 0, "new_read_ops": 0, "compacts": 47, "compresses": 0, "cache_hits": 2603, "cache_misses": 13, "cache_hit_ratio": 1.00000, "lookup_dur": "0s", "writer_dur": "0s", "writer_throttle_dur": "0s", "controller": { "build": [false], "swap": [true], "instances": [35], "pages": [49], "quota": [9002571], "mem_in_use": [137722], "mem_used_global": [137722], "mandatory_quota": [9002571], "discretionary_quota": [0], "resident_quota": [26196], "derived_quota": [4448], "mutation_quota": [8976375], "idle_quota": [22854], "new_evict_cycle": [0], "new_evict_visit": [0], "new_evicted": [0], "swapper_instances": [[35]], "sweep_duration": [["0s"]], "swapper_pages": [[35]], "swapper_cycles": [[0]], "evict_visit": [[0]], "evict_burst": [[0]], "evict_periodic": [[0]], "evictable": [[0]], "evicted": [[0]], "compressed": [[0]], "resident": [[0.99207]] } } } 2023-10-21T04:50:30.721+05:30 DEBU REGU.impl.dumpStats() at aggrecorder.go:364 [id 16] aggRecorder: [bucket_1] recorded 14715 aborted 0 refunded 0 2023-10-21T04:50:31.627+05:30 [Info] DDLServiceMgr checking create token progress 2023-10-21T04:50:31.639+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9118 2023-10-21T04:50:31.639+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:50:31.643+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35844 2023-10-21T04:50:31.643+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:50:31.656+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 9.845154ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:31.657+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:50:31.657+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9124 2023-10-21T04:50:31.657+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:50:31.662+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:50:31.681+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:50:31.681+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9106 2023-10-21T04:50:31.681+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:50:31.689+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:50:31.709+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:50:31.710+05:30 [Info] DDLServiceMgr::cleanupTokens:: connecting to node 127.0.0.1:9112 2023-10-21T04:50:31.711+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:50:31.716+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:50:31.733+05:30 [Info] memstats {"Alloc":182138888, "TotalAlloc":846342808, "Sys":244275512, "Lookups":0, "Mallocs":6722320,"Frees":6234171, "HeapAlloc":182138888, "HeapSys":222134272, "HeapIdle":36749312, "HeapInuse":185384960,"HeapReleased":917504, "HeapObjects":488149,"MSpanInuse":982240, "MSpanSys": 1566720, "StackInuse": 4358144,"GCSys":13507320, "LastGC":1697843929883254746,"PauseTotalNs":4380064, "PauseNs":[254941], "NumGC":13} 2023-10-21T04:50:31.739+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:50:31.739+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:50:31.739+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:50:31.739+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:50:31.739+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:50:31.764+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/10606939961365971246 2023-10-21T04:50:31.778+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/10761885902580101111 2023-10-21T04:50:31.811+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/11276338709617092948 2023-10-21T04:50:31.837+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/11365633301282701856 2023-10-21T04:50:31.870+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/11619165464560280276 2023-10-21T04:50:31.905+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12027054384338714804 2023-10-21T04:50:31.964+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12140458060295645238 2023-10-21T04:50:32.019+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12195715103909237684 2023-10-21T04:50:32.052+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12338036531800130039 2023-10-21T04:50:32.098+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/1252359852432998131 2023-10-21T04:50:32.111+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12582561312368048736 2023-10-21T04:50:32.156+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:32.177+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12599498416578472779 2023-10-21T04:50:32.243+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12824340686309573428 2023-10-21T04:50:32.304+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/12833670934079048728 2023-10-21T04:50:32.334+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/128675491261474353 2023-10-21T04:50:32.383+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/1301336828089581719 2023-10-21T04:50:32.449+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/13268712595133993155 2023-10-21T04:50:32.488+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/13856704970496601958 2023-10-21T04:50:32.506+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14292583894719865678 2023-10-21T04:50:32.565+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14549775068266281786 2023-10-21T04:50:32.604+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14741949230702299196 2023-10-21T04:50:32.623+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/14889401051926000176 2023-10-21T04:50:32.700+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15431181238515168303 2023-10-21T04:50:32.757+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15945117906390072636 2023-10-21T04:50:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:50:32.787+05:30 [Info] client stats current counts: current: 14, not current: 0 2023-10-21T04:50:32.790+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15968368593314043432 2023-10-21T04:50:32.824+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/15973521466438086342 2023-10-21T04:50:32.862+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16162012140118277065 2023-10-21T04:50:32.866+05:30 [Info] num concurrent scans {0} 2023-10-21T04:50:32.866+05:30 [Info] average scan response {0 ms} 2023-10-21T04:50:32.887+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16337157693634799600 2023-10-21T04:50:32.920+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16367703496921140088 2023-10-21T04:50:32.985+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16483001637761719171 2023-10-21T04:50:32.998+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:35890 2023-10-21T04:50:33.046+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16632040625897953619 2023-10-21T04:50:33.077+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16706481854123898870 2023-10-21T04:50:33.087+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 80.496417ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:33.119+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/16961575949607441659 2023-10-21T04:50:33.179+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/17476654447546601131 2023-10-21T04:50:33.203+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/17656144409930763141 2023-10-21T04:50:33.256+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/17797267771846799851 2023-10-21T04:50:33.304+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/17917116660857103172 2023-10-21T04:50:33.311+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:50:33.311+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:50:33.373+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18026660730857541767 2023-10-21T04:50:33.423+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18046371353728341033 2023-10-21T04:50:33.472+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/18323460706027642575 2023-10-21T04:50:33.523+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/2048006213723315482 2023-10-21T04:50:33.579+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:50:33.588+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:33.595+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/249887442666643345 2023-10-21T04:50:33.601+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:50:33.651+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/2608585389864151823 2023-10-21T04:50:33.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9796942255715619, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:50:33.757+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/3381524789655566900 2023-10-21T04:50:33.764+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:50:33.870+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/3674078063353481907 2023-10-21T04:50:33.976+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4138047918192025493 2023-10-21T04:50:34.030+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4286621850367430051 2023-10-21T04:50:34.124+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4532813151040661705 2023-10-21T04:50:34.183+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4535197604682045916 2023-10-21T04:50:34.272+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4790214984019693894 2023-10-21T04:50:34.310+05:30 [Info] janitor: running cleanup. 2023-10-21T04:50:34.310+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[12833670934079048728 8857269605176042330 12824340686309573428 8669649359784409226 15968368593314043432 7392796230534034390 3674078063353481907 4535197604682045916 11619165464560280276 12195715103909237684 8378417662949994532 11276338709617092948 16483001637761719171 7147407900589480727 12027054384338714804 573521267164877540 12582561312368048736 6985049432225870291 14292583894719865678 4790214984019693894 15945117906390072636 12338036531800130039 8504686614261377811 1301336828089581719 17917116660857103172 8910459001587546151 7045704526128519604 18026660730857541767 4860549750398470299 16961575949607441659 16337157693634799600 16367703496921140088 4987105685507282285 17476654447546601131 5465835793930010953 5932387718257253322 2608585389864151823 10761885902580101111 14889401051926000176 15973521466438086342 13268712595133993155 16162012140118277065 13856704970496601958 10606939961365971246 18323460706027642575 14549775068266281786 1252359852432998131 16632040625897953619 249887442666643345 16706481854123898870 14741949230702299196 15431181238515168303 6145407649125550089 17656144409930763141 18046371353728341033 3381524789655566900 7525085113233084253 11365633301282701856 4532813151040661705 2048006213723315482 4138047918192025493 12140458060295645238 9665410041638025825 128675491261474353 17797267771846799851 12599498416578472779 4841852412665776737 4286621850367430051]} 2023-10-21T04:50:34.358+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4841852412665776737 2023-10-21T04:50:34.538+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4860549750398470299 2023-10-21T04:50:34.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9989181248697326, throttleDelayMs (new, old, change): (8, 3, 5) 2023-10-21T04:50:34.716+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/4987105685507282285 2023-10-21T04:50:35.087+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/5465835793930010953 2023-10-21T04:50:35.208+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/573521267164877540 2023-10-21T04:50:35.279+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/5932387718257253322 2023-10-21T04:50:35.355+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/6145407649125550089 2023-10-21T04:50:35.428+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/6985049432225870291 2023-10-21T04:50:35.548+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7045704526128519604 2023-10-21T04:50:35.578+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 7735644145307932837 2023-10-21T04:50:35.578+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 156.231µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:35.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9996594467034091, throttleDelayMs (new, old, change): (10, 8, 2) 2023-10-21T04:50:35.806+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7147407900589480727 2023-10-21T04:50:35.965+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7392796230534034390 2023-10-21T04:50:35.979+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48392 2023-10-21T04:50:36.020+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48392 2023-10-21T04:50:36.070+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/7525085113233084253 2023-10-21T04:50:36.254+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8378417662949994532 2023-10-21T04:50:36.293+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:48392 2023-10-21T04:50:36.367+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8504686614261377811 2023-10-21T04:50:36.462+05:30 [Info] Indexer::ReadMemstats Time Taken 4.978692ms 2023-10-21T04:50:36.464+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8669649359784409226 2023-10-21T04:50:36.565+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8857269605176042330 2023-10-21T04:50:36.581+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 7735644145307932837 2023-10-21T04:50:36.581+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 276.871µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:36.667+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9994653515313682 2023-10-21T04:50:36.766+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/8910459001587546151 2023-10-21T04:50:36.783+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:36.784+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:36.823+05:30 [Info] switched currmeta from 472 -> 481 force true 2023-10-21T04:50:36.906+05:30 [Info] DDLServiceMgr: processing delete index token /indexing/ddl/commandToken/delete/9665410041638025825 2023-10-21T04:50:36.928+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:36.928+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:36.953+05:30 [Info] switched currmeta from 481 -> 481 force true 2023-10-21T04:50:37.098+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:50:37.098+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:50:37.098+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:50:37.098+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:50:37.098+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:50:37.099+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:60452->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:37.099+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:55332->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:37.099+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35844. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:37.099+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:35844->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:37.100+05:30 [Info] messageListener.start(): message channel closed. Remove peer 91:24:ab:6e:f3:ef:11:eb as follower. 2023-10-21T04:50:37.104+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:55794->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:37.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9995694893005906 2023-10-21T04:50:37.868+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:50:37.868+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:50:38.019+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:36054 2023-10-21T04:50:38.082+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:36060 2023-10-21T04:50:38.121+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:50:38.217+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 128.002613ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:38.247+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 221.189427ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:38.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9999632242569799 2023-10-21T04:50:38.721+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:38.748+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:38.774+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:50:38.774+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:50:38.774+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:50:38.911+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:50:39.039+05:30 [Info] LifecycleMgr.updateIndexReplicaCount() : Index Definition does not exist for 7735644145307932837. No update is performed. 2023-10-21T04:50:39.039+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 139.847µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:39.075+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:50:39.075+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:50:39.075+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:50:39.080+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:36060. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:39.080+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6f:f2:6e:cd:f1:bc:7:8d as follower. 2023-10-21T04:50:39.216+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:50:39.476+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:36054. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:39.477+05:30 [Info] messageListener.start(): message channel closed. Remove peer fe:62:91:39:2e:a6:4b:dc as follower. 2023-10-21T04:50:39.491+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:50:39.491+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:50:39.491+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:50:39.579+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:36126 2023-10-21T04:50:39.604+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:36130 2023-10-21T04:50:39.605+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:50:39.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9997768866018967 2023-10-21T04:50:39.775+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 191.116002ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:39.869+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 260.562854ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:39.869+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:50:39.869+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:50:39.870+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:50:39.870+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:50:39.870+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:50:39.870+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:50:39.870+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:50:39.870+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:50:39.870+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:50:39.870+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:50:39.870+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:36130->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:39.870+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:60676->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:39.871+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:55586->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:39.871+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:56056->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:50:39.871+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:36130. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:39.871+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4d:6d:18:35:47:b3:29:2d as follower. 2023-10-21T04:50:40.013+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:36126. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:40.014+05:30 [Info] messageListener.start(): message channel closed. Remove peer f6:5c:c7:33:88:65:bb:2d as follower. 2023-10-21T04:50:40.276+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:40.371+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:50:40.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9997068126976982 2023-10-21T04:50:40.964+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:50:40.991+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:40.991+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:41.011+05:30 [Info] switched currmeta from 481 -> 481 force true 2023-10-21T04:50:41.170+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:35890. Reason = EOF. Cleaning Pipe. 2023-10-21T04:50:41.171+05:30 [Info] messageListener.start(): message channel closed. Remove peer 25:ca:e8:d0:9b:b4:f7:c9 as follower. 2023-10-21T04:50:41.603+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_1 Forcing Overdue Commit 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 14033078509407174050 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:50:41.603+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 14033078509407174050 PartitionId: 0 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.825µs SnapOpenDur 59.743µs 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 8 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.603+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 8 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.777µs SnapOpenDur 54.108µs 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 1 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.603+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 1 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.575µs SnapOpenDur 30.283µs 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 2 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.603+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 2 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.446µs SnapOpenDur 28.562µs 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 3 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.603+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 3 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.501µs SnapOpenDur 24.379µs 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 4 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.603+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 4 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.474µs SnapOpenDur 23.816µs 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 5 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.603+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 5 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.5µs SnapOpenDur 72.025µs 2023-10-21T04:50:41.603+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 6 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.604+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 6 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.162µs SnapOpenDur 38.757µs 2023-10-21T04:50:41.604+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 7 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:50:41.604+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 7 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.522µs SnapOpenDur 24.836µs 2023-10-21T04:50:41.604+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 544693394183450628 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:50:41.604+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 544693394183450628 PartitionId: 0 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:6 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.127µs SnapOpenDur 55.224µs 2023-10-21T04:50:41.604+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14033078509407174050, PartitionId 0 SnapshotId 21 Creating recovery point ... 2023-10-21T04:50:41.605+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.605+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.605+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.605+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 544693394183450628, PartitionId 0 SnapshotId 3 Creating recovery point ... 2023-10-21T04:50:41.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.606+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:50:41.656+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 14033078509407174050, PartitionId 0 SnapshotId 21 Created recovery point (took 52.068985ms) 2023-10-21T04:50:41.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9922148545688199 2023-10-21T04:50:41.799+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 544693394183450628, PartitionId 0 SnapshotId 3 Created recovery point (took 193.018891ms) 2023-10-21T04:50:41.801+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 544693394183450628, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:50:41.866+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 SnapshotId 2 Created recovery point (took 261.273253ms) 2023-10-21T04:50:41.868+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 4. 2023-10-21T04:50:41.868+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.869+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 4. 2023-10-21T04:50:41.869+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.912+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 SnapshotId 2 Created recovery point (took 306.762312ms) 2023-10-21T04:50:41.916+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.916+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.949+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 SnapshotId 2 Created recovery point (took 343.623449ms) 2023-10-21T04:50:41.950+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.951+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.971+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 SnapshotId 2 Created recovery point (took 365.265531ms) 2023-10-21T04:50:41.972+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.972+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.989+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 SnapshotId 2 Created recovery point (took 383.504801ms) 2023-10-21T04:50:41.990+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:41.991+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:42.018+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 SnapshotId 2 Created recovery point (took 411.880317ms) 2023-10-21T04:50:42.019+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:42.020+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:42.057+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 SnapshotId 2 Created recovery point (took 451.212426ms) 2023-10-21T04:50:42.059+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:42.059+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:42.074+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 SnapshotId 2 Created recovery point (took 468.411701ms) 2023-10-21T04:50:42.076+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:42.077+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:50:42.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9638950587727252 2023-10-21T04:50:42.984+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:42.984+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:42.986+05:30 [Info] switched currmeta from 481 -> 482 force true 2023-10-21T04:50:43.021+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:43.021+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:43.026+05:30 [Info] switched currmeta from 482 -> 483 force true 2023-10-21T04:50:43.412+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:43.412+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:43.417+05:30 [Info] switched currmeta from 483 -> 484 force true 2023-10-21T04:50:43.450+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:43.450+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:43.456+05:30 [Info] switched currmeta from 484 -> 485 force true 2023-10-21T04:50:43.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.751117222542223, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:50:43.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14190651729632750031:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:50:43.969+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[14190651729632750031:[0]]. New rollbacktime: 1697843376847668953, Old rollbacktime: 0 2023-10-21T04:50:44.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.46291057675230696, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:50:44.860+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2047508721584820382:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:50:44.861+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2047508721584820382:[0]]. New rollbacktime: 1697843376907212309, Old rollbacktime: 0 2023-10-21T04:50:45.538+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:50:45.544+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:45.544+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:45.546+05:30 [Info] switched currmeta from 485 -> 485 force true 2023-10-21T04:50:47.759+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:50:47.759+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 21] call. err: operation_canceled 2023-10-21T04:50:47.759+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:50:47.759+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 21] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:50:47.759+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 21] Tasks:[]} 2023-10-21T04:50:47.760+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 21] 2023-10-21T04:50:47.761+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 21] 2023-10-21T04:50:55.165+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:50:55.269+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:50:55.269+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:50:55.292+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:55.292+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:55.297+05:30 [Info] switched currmeta from 485 -> 485 force true 2023-10-21T04:50:55.442+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:50:55.442+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:50:55.482+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:55.485+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:55.491+05:30 [Info] switched currmeta from 485 -> 485 force true 2023-10-21T04:50:55.556+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:55.558+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:55.563+05:30 [Info] switched currmeta from 485 -> 485 force true 2023-10-21T04:50:56.834+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {78118431caff95cf657f177e70958ebf [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:50:56.858+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 8.765µs 2023-10-21T04:50:56.858+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:50:56.858+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:50:56.858+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:50:56.862+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:50:56.862+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:50:56.862+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 4.784561ms 2023-10-21T04:50:56.863+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:50:56.863+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced true 2023-10-21T04:50:56.863+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 22] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:50:56.863+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 8.534µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:56.863+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 21] call. taskList: &{Rev:[0 0 0 0 0 0 0 22] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/78118431caff95cf657f177e70958ebf Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]}]} 2023-10-21T04:50:56.868+05:30 [Info] RebalanceServiceManager::StartTopologyChange: change: {78118431caff95cf657f177e70958ebf [] topology-change-rebalance [{{4325b30e7f9620a998388078247335a8 7060000 } recovery-full} {{ba6304988a6335d65281381d6cfc5e32 7060000 } recovery-full}] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 }]} 2023-10-21T04:50:56.871+05:30 [Info] RebalanceServiceManager::cleanupSettingsShardAffinityPath Cleaning up shard affinity settings path as cluster version is: 9 2023-10-21T04:50:56.876+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 22] 2023-10-21T04:50:56.888+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 22] 2023-10-21T04:50:56.959+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"78118431caff95cf657f177e70958ebf","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:50:56.961+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{4325b30e7f9620a998388078247335a8 78118431caff95cf657f177e70958ebf Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:50:56.983+05:30 [Info] RebalanceServiceManager::registerRebalanceTokenInMetakv: Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 78118431caff95cf657f177e70958ebf Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:50:57.996+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9108/registerRebalanceToken 2023-10-21T04:50:58.003+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9114/registerRebalanceToken 2023-10-21T04:50:58.003+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Skip local service 127.0.0.1:9120 2023-10-21T04:50:58.012+05:30 [Info] RebalanceServiceManager::registerGlobalRebalanceToken Successfully registered rebalance token on 127.0.0.1:9126/registerRebalanceToken 2023-10-21T04:50:58.012+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{4325b30e7f9620a998388078247335a8 78118431caff95cf657f177e70958ebf Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master true localaddr 127.0.0.1:9120 runPlanner true runParam &{false []} clusterVersion 9 2023-10-21T04:50:58.012+05:30 [Info] RebalanceServiceManager::StartTopologyChange: returns Error . isBalanced true. 2023-10-21T04:50:58.012+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 1.143732062s in RebalanceServiceManager::StartTopologyChange: 2023-10-21T04:50:58.012+05:30 [Info] RebalanceServiceManager::monitorStartPhaseInit: Found RebalanceToken &{4325b30e7f9620a998388078247335a8 78118431caff95cf657f177e70958ebf Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD}. 2023-10-21T04:50:58.012+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 22] call. taskList: &{Rev:[0 0 0 0 0 0 0 23] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/78118431caff95cf657f177e70958ebf Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/78118431caff95cf657f177e70958ebf Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]}]} 2023-10-21T04:50:58.013+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 23] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:50:58.013+05:30 [Info] ShardRebalancer::updateProgress goroutine started 2023-10-21T04:50:58.020+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:50:58.020+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 23] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/78118431caff95cf657f177e70958ebf Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]} {Rev:[0 0 0 0 0 0 0 0] ID:rebalance/78118431caff95cf657f177e70958ebf Type:task-rebalance Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]}]} 2023-10-21T04:50:58.025+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:50:58.025+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 23] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:50:58.026+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 23] 2023-10-21T04:50:58.027+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 23] 2023-10-21T04:50:58.128+05:30 [Info] ShardRebalancer::initRebalAsync Global Topology &{[{d7ff2bfd429b21d9c24d822bc13e3acc d7ff2bfd429b21d9c24d822bc13e3acc plasma map[] [{75 bucket_1 _default c1 []} {8 bucket_1 _system _query []} {77 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{12606056968936758026 4 1 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{8303154871847919944 1 0 [{1 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {2 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {3 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {4 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {5 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {6 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {7 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []} {8 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 8 0 0 1 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 7735644145307932837 [{2047508721584820382 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501 11711860383628641160] []}] 1 0 0 0 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{17465522551378543576 4 1 [{0 0 {[{0 0 d7ff2bfd429b21d9c24d822bc13e3acc}]} {[] []} [10838601320854145501] []}] 1 0 0 1 false plasma 0}]}]}] [DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {9b27839d59b06374125d80a2b12a21b6 9b27839d59b06374125d80a2b12a21b6 plasma map[] [{75 bucket_1 _default c1 []} {8 bucket_1 _system _query []} {77 bucket_%2 _default c1 [{bucket_%2 _default c1 idx_partitioned 4390029557575378783 [{4397940545762329850 4 1 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_partitioned_defer 17703248484023971553 [{12396348969790516872 1 0 [{1 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {2 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {3 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {4 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {5 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {6 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {7 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []} {8 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 8 0 0 0 false plasma 0}]} {bucket_%2 _default c1 idx_secondary 7735644145307932837 [{14190651729632750031 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589 6455624821495495854] []}] 1 0 0 1 false plasma 0}]}]} {6 bucket_%2 _system _query [{bucket_%2 _system _query #primary 5277316367626246544 [{10158339249566909515 4 1 [{0 0 {[{0 0 9b27839d59b06374125d80a2b12a21b6}]} {[] []} [18374611443167983589] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {4325b30e7f9620a998388078247335a8 4325b30e7f9620a998388078247335a8 plasma map[] [{60 bucket_%2 _default c1 []} {9 bucket_%2 _system _query []} {27 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{16759912838651900043 4 1 [{1 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{4186965058349029465 1 0 [{1 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {2 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {3 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {4 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {5 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {6 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {7 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []} {8 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []}] 8 0 1 0 false plasma 0}]} {bucket_1 _default c1 idx_secondary 16364390102182676976 [{14033078509407174050 4 1 [{0 0 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215 5629943257627822009] []}] 1 0 0 1 false plasma 0}]}]} {7 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{544693394183450628 4 1 [{0 1 {[{0 0 4325b30e7f9620a998388078247335a8}]} {[] []} [17327901024118572215] []}] 1 0 1 0 false plasma 0}]}]}] [DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false} {ba6304988a6335d65281381d6cfc5e32 ba6304988a6335d65281381d6cfc5e32 plasma map[] [{10 bucket_1 _system _query [{bucket_1 _system _query #primary 668551233566132751 [{18137927630988528516 4 1 [{0 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563] []}] 1 0 1 1 false plasma 0}]}]} {16 bucket_%2 _system _query []} {41 bucket_%2 _default c1 []} {23 bucket_1 _default c1 [{bucket_1 _default c1 idx_partitioned 7790129686881525591 [{9316375249999768625 4 1 [{1 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c1 idx_partitioned_defer 16823504650100619022 [{3751526798537645569 1 0 [{1 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {2 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {3 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {4 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {5 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {6 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {7 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []} {8 1 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []}] 8 0 1 1 false plasma 0}]} {bucket_1 _default c1 idx_secondary 16364390102182676976 [{10417104488879353350 4 1 [{0 0 {[{0 0 ba6304988a6335d65281381d6cfc5e32}]} {[] []} [11114961608238406563 5748926106587204814] []}] 1 0 0 0 false plasma 0}]}]}] [DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] ] 0 0 0 false}] map[]} 2023-10-21T04:50:58.230+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:50:58.235+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9002 2023-10-21T04:50:58.235+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9004 2023-10-21T04:50:58.235+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9001 2023-10-21T04:50:58.235+05:30 [Info] Planner::processCreateToken: There is no create token to process for node: 127.0.0.1:9003 2023-10-21T04:50:58.235+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:50:58.279+05:30 [Info] Planner::processDeleteToken: processing 68 tokens for node 127.0.0.1:9002 2023-10-21T04:50:58.279+05:30 [Info] Planner::processDeleteToken: processing 68 tokens for node 127.0.0.1:9003 2023-10-21T04:50:58.279+05:30 [Info] Planner::processDeleteToken: processing 68 tokens for node 127.0.0.1:9004 2023-10-21T04:50:58.280+05:30 [Info] Planner::processDeleteToken: processing 68 tokens for node 127.0.0.1:9001 2023-10-21T04:50:58.281+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:32850 2023-10-21T04:50:58.288+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9001 2023-10-21T04:50:58.288+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9003 2023-10-21T04:50:58.288+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9004 2023-10-21T04:50:58.288+05:30 [Info] Planner::processDropInstanceToken: There is no drop instance token to process for node: 127.0.0.1:9002 2023-10-21T04:50:58.328+05:30 [Info] Planner::findPlacementForDeletedNodes Deleted Nodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:50:58.328+05:30 [Info] Planner::moveTenantsFromDeletedNodes nonEmptyDeletedNodes [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:50:58.328+05:30 [Info] Planner::moveTenantsFromDeletedNodes pairForDeletedNodes [127.0.0.1:9002 127.0.0.1:9001] 2023-10-21T04:50:58.328+05:30 [Info] Planner::moveTenantsFromDeletedNodes newNodes [] 2023-10-21T04:50:58.328+05:30 [Info] Planner::moveTenantsFromDeletedNodes Num deleted nodes 2 is more than num new/empty nodes 0 2023-10-21T04:50:58.328+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantsToBeMoved from source [127.0.0.1:9001 127.0.0.1:9002] 2023-10-21T04:50:58.328+05:30 [Info] Planner::moveTenantsFromDeletedNodes TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket_%2 MemoryUsage 147617 UnitsUsage 6 2023-10-21T04:50:58.328+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 9002586 Units 0 Tenants 1 2023-10-21T04:50:58.328+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 9003216 Units 0 Tenants 1 2023-10-21T04:50:58.328+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9001 SG Group 2 Memory 9000529 Units 5 2023-10-21T04:50:58.328+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Skip Deleted Index Node 127.0.0.1:9002 SG Group 1 Memory 9001221 Units 6 2023-10-21T04:50:58.328+05:30 [Info] Planner::moveTenantsFromDeletedNodes Found SubClusters Below LWM [[127.0.0.1:9003 127.0.0.1:9004]] 2023-10-21T04:50:58.328+05:30 [Info] Planner::findTenantPlacement TenantUsage - SourceId 127.0.0.1:9002 TenantId bucket_%2 MemoryUsage 147617 UnitsUsage 6 can be placed on [127.0.0.1:9003 127.0.0.1:9004] 2023-10-21T04:50:58.328+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9001 SG Group 2 Memory 8858202 Units 0 2023-10-21T04:50:58.328+05:30 [Info] Planner::findPlacementForDeletedNodes Remove Deleted Node from solution 127.0.0.1:9002 SG Group 1 Memory 8853604 Units 0 2023-10-21T04:50:58.328+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 9144913 Units 5 Tenants 1 2023-10-21T04:50:58.328+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 9150833 Units 6 Tenants 1 2023-10-21T04:50:58.328+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters [[127.0.0.1:9003 127.0.0.1:9004]] 2023-10-21T04:50:58.328+05:30 [Info] Planner::executeTenantAwareRebal Found SubClusters above HWM [] 2023-10-21T04:50:58.328+05:30 [Info] ************ Indexer Layout ************* 2023-10-21T04:50:58.328+05:30 [Info] Using TenantAwarePlanner 2023-10-21T04:50:58.328+05:30 [Info] ---------------------------------------- 2023-10-21T04:50:58.328+05:30 [Info] 2023-10-21T04:50:58.328+05:30 [Info] Indexer serverGroup:Group 2, nodeId:127.0.0.1:9003, nodeUUID:4325b30e7f9620a998388078247335a8, useLiveData:false 2023-10-21T04:50:58.328+05:30 [Info] Indexer total memory:2936147833 (2.7345G), mem:116912 (114.172K), overhead:2936030921 (2.73439G), min:2936147833 (2.7345G), data:116912 (114.172K) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:50:58.328+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:50:58.328+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.328+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.328+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.328+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.328+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.328+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:16759912838651900043, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.329+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.329+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:4186965058349029465, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.329+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.329+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:#primary 0, bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:544693394183450628, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157294383 (150.008M), mem:6912 (6.75K), overhead:157287471 (150.001M), min:157294383 (150.008M), data:6912 (6.75K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16364390102182676976, instId:14033078509407174050, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [17327901024118572215 5629943257627822009], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:12606056968936758026, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.330+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.330+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.330+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.330+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:8303154871847919944, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_%2, scope:_default, collection:c1, defnId:7735644145307932837, instId:2047508721584820382, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501 11711860383628641160], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:17465522551378543576, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [10838601320854145501], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:157293752 (150.007M), mem:6366 (6.2168K), overhead:157287386 (150.001M), min:157293752 (150.007M), data:6366 (6.2168K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] 2023-10-21T04:50:58.331+05:30 [Info] Indexer serverGroup:Group 1, nodeId:127.0.0.1:9004, nodeUUID:ba6304988a6335d65281381d6cfc5e32, useLiveData:false 2023-10-21T04:50:58.331+05:30 [Info] Indexer total memory:2936147840 (2.7345G), mem:116918 (114.178K), overhead:2936030922 (2.73439G), min:2936147840 (2.7345G), data:116918 (114.178K) cpu:0.0000, io:0 (0), scan:0 drain:0 numIndexes:36 shardCompatVersion: 1 2023-10-21T04:50:58.331+05:30 [Info] Indexer isDeleted:false isNew:false exclude: meetConstraint:true usageRatio:0 allowDDLDuringScaleup:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_secondary 0, bucket:bucket_1, scope:_default, collection:c1, defnId:16364390102182676976, instId:10417104488879353350, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:157413450 (150.121M), mem:110000 (107.422K), overhead:157303450 (150.016M), min:157413450 (150.121M), data:110000 (107.422K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.331+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.331+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.331+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.331+05:30 [Info] Index name:idx_partitioned 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:7790129686881525591, instId:9316375249999768625, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned_defer 1 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 1, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned_defer 2 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 2, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned_defer 3 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 3, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned_defer 4 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 4, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned_defer 5 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 5, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.332+05:30 [Info] Index name:idx_partitioned_defer 6 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 6, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.332+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.332+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.332+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned_defer 7 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 7, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned_defer 8 (replica 1), bucket:bucket_1, scope:_default, collection:c1, defnId:16823504650100619022, instId:3751526798537645569, Partition: 8, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563 5748926106587204814], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:#primary 0 (replica 1), bucket:bucket_1, scope:_system, collection:_query, defnId:668551233566132751, instId:18137927630988528516, Partition: 0, new/moved:false shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [11114961608238406563], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157294390 (150.008M), mem:6918 (6.75586K), overhead:157287472 (150.001M), min:157294390 (150.008M), data:6918 (6.75586K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:#primary 0, bucket:bucket_%2, scope:_system, collection:_query, defnId:5277316367626246544, instId:10158339249566909515, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157293898 (150.007M), mem:6492 (6.33984K), overhead:157287406 (150.001M), min:157293898 (150.007M), data:6492 (6.33984K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:4390029557575378783, instId:4397940545762329850, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:157286400 (150M), mem:0 (0), overhead:157286400 (150M), min:157286400 (150M), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.333+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.333+05:30 [Info] Index name:idx_partitioned_defer 1, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 1, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.333+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.333+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_partitioned_defer 2, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 2, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_partitioned_defer 3, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 3, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_partitioned_defer 4, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 4, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_partitioned_defer 5, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 5, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_partitioned_defer 6, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 6, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_partitioned_defer 7, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 7, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_partitioned_defer 8, bucket:bucket_%2, scope:_default, collection:c1, defnId:17703248484023971553, instId:12396348969790516872, Partition: 8, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:0 (0), mem:0 (0), overhead:157286400 (150M), min:0 (0), data:0 (0) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] ------------------------------------------------------------------------------------------------------------------ 2023-10-21T04:50:58.334+05:30 [Info] Index name:idx_secondary 0 (replica 1), bucket:bucket_%2, scope:_default, collection:c1, defnId:7735644145307932837, instId:14190651729632750031, Partition: 0, new/moved:true shardProxy: false, numInstances: 0, alternateShardIds: [], shardIds: [18374611443167983589 6455624821495495854], shardIdsInDefn: map[] 2023-10-21T04:50:58.334+05:30 [Info] Index total memory:157414605 (150.122M), mem:111000 (108.398K), overhead:157303605 (150.016M), min:157414605 (150.122M), data:111000 (108.398K) cpu:0.0000 io:0 (0) scan:0 drain:0 2023-10-21T04:50:58.334+05:30 [Info] Index resident:100% build:100% estimated:false equivCheck:true pendingCreate:false pendingDelete:false 2023-10-21T04:50:58.334+05:30 [Info] **************************************** 2023-10-21T04:50:58.334+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9003 SG Group 2 Memory 9144913 Units 5 Tenants 1 2023-10-21T04:50:58.334+05:30 [Info] Planner::groupIndexNodesIntoSubClusters Index Node 127.0.0.1:9004 SG Group 1 Memory 9150833 Units 6 Tenants 1 2023-10-21T04:50:58.334+05:30 [Info] Generating Shard Transfer Token (ShardToken11:a9:c7:70:34:9f:8d:92) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: Region: InstId: 15757988232943046271 RealInstId: 12606056968936758026 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 9226953440900345886 RealInstId: 8303154871847919944 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 2047508721584820382 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 17465522551378543576 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:50:58.336+05:30 [Info] Generating Shard Transfer Token (ShardTokenba:4e:99:24:ac:d2:11:c2) for rebalance ( MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: Region: InstId: 4130535070377368620 RealInstId: 4397940545762329850 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 4397940545762329850 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 12046213868758366295 RealInstId: 12396348969790516872 Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] Inst: InstId: 12396348969790516872 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: InstId: 14190651729632750031 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 14190651729632750031 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: InstId: 10158339249566909515 RealInstId: 0 Partitions: [0] Versions: [1] Inst: InstId: 10158339249566909515 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: IsEmptyNodeBatch: false IsPendingReady: false ) 2023-10-21T04:50:58.336+05:30 [Info] ShardRebalancer::initRebalAsync Populated destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, region: in all transfer tokens 2023-10-21T04:50:58.336+05:30 [Info] ShardRebalancer::enableEmptyNodeBatching Empty node batching is not applicable for serverless deployment 2023-10-21T04:50:58.339+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardToken11:a9:c7:70:34:9f:8d:92 2023-10-21T04:50:58.354+05:30 [Info] ShardRebalancer::publishShardTransferTokens Published transfer token: ShardTokenba:4e:99:24:ac:d2:11:c2 2023-10-21T04:50:58.354+05:30 [Info] ShardRebalancer::observeRebalance &{4325b30e7f9620a998388078247335a8 78118431caff95cf657f177e70958ebf Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:true 2023-10-21T04:50:58.364+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.364+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenCreated BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.364+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"78118431caff95cf657f177e70958ebf","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:50:58.364+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:50:58.402+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.403+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:50:58.403+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_INITIATED map[bucket_%2:REBALANCE_INITIATED]} 2023-10-21T04:50:58.403+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_INITIATED, bucketTransfer: map[bucket_%2:REBALANCE_INITIATED] 2023-10-21T04:50:58.403+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 22.612µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:58.466+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.494+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:50:58.522+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenScheduledOnSource BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.530+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenScheduleAck BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.537+05:30 [Info] TestCode::TestActionAtTag: currTag: 1, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:50:58.537+05:30 [Info] ShardRebalancer::allShardTransferTokensAcked All transfer tokens moved to ScheduleAck state. Initiating transfer for futher processing 2023-10-21T04:50:58.542+05:30 [Info] ShardRebalancer::updateRebalancePhaseInGlobalRebalToken Registered Global Rebalance Token In Metakv &{4325b30e7f9620a998388078247335a8 78118431caff95cf657f177e70958ebf Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:50:58.542+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[], batchSize: 1 2023-10-21T04:50:58.556+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster Published transfer token batch: map[9b27839d59b06374125d80a2b12a21b6:ShardTokenba:4e:99:24:ac:d2:11:c2 d7ff2bfd429b21d9c24d822bc13e3acc:ShardToken11:a9:c7:70:34:9f:8d:92], currActiveTransfers: map[9b27839d59b06374125d80a2b12a21b6:1 d7ff2bfd429b21d9c24d822bc13e3acc:1] 2023-10-21T04:50:58.556+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"4325b30e7f9620a998388078247335a8","RebalId":"78118431caff95cf657f177e70958ebf","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":1,"RebalPhase":2,"ActiveRebalancer":2} 2023-10-21T04:50:58.556+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:50:58.556+05:30 [Info] ShardRebalancer::updateGlobalRebalancePhase Updating rebalance phase to REBALANCE_TRANSFER_IN_PROGRESS 2023-10-21T04:50:58.556+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:50:58.556+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[]} 2023-10-21T04:50:58.556+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[] 2023-10-21T04:50:58.556+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.557+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 18.398µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:58.561+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenTransferShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.762+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.762+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:1 9b27839d59b06374125d80a2b12a21b6:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:50:58.762+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:50:58.762+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_TRANSFER_DONE]} 2023-10-21T04:50:58.762+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_TRANSFER_DONE] 2023-10-21T04:50:58.762+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 26.669µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:58.763+05:30 [Info] ShardTransferManager::processShardRestoreMessage Initiating command: &{map[10838601320854145501:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/10838601320854145501/shards/shard10838601320854145501 11711860383628641160:/opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/11711860383628641160/shards/shard11711860383628641160] 78118431caff95cf657f177e70958ebf ShardToken11:a9:c7:70:34:9f:8d:92 /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir 0 0xc00cdb2420 0xc00cdb2480 map[] 0xc008314000 0xc009b04120 false} 2023-10-21T04:50:58.763+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/11711860383628641160/shards/shard11711860383628641160 2023-10-21T04:50:58.764+05:30 [Info] plasma DownloadContext : initiating shard restore for download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/10838601320854145501/shards/shard10838601320854145501 2023-10-21T04:50:58.772+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/11711860383628641160/shards/shard11711860383628641160/data: super block snapshot head offset :0 tail offset:1540096 sbgen :241 2023-10-21T04:50:58.772+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/11711860383628641160/shards/shard11711860383628641160/data/recovery: super block snapshot head offset :0 tail offset:1314816 sbgen :241 2023-10-21T04:50:58.773+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/10838601320854145501/shards/shard10838601320854145501/data: super block snapshot head offset :0 tail offset:1900544 sbgen :264 2023-10-21T04:50:58.773+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/10838601320854145501/shards/shard10838601320854145501/data/recovery: super block snapshot head offset :0 tail offset:1716224 sbgen :264 2023-10-21T04:50:58.776+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/10838601320854145501/shards/shard10838601320854145501: metadata saved successfully 2023-10-21T04:50:58.777+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 (10838601320854145501) added 2023-10-21T04:50:58.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:50:58.777+05:30 [Info] plasma shard transfer total bandwidth stats (download), avgBW: 259717857 totalBytes: 3636050 numShards: 2 duration: 14.273851ms 2023-10-21T04:50:58.777+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/10838601320854145501/shards/shard10838601320854145501, newshardId 10838601320854145501, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501, shardsize 3636050, duration 13.403093ms 2023-10-21T04:50:58.777+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 10838601320854145501 bytesWritten: 3636050, totalBytes: 3636050, transferRate: 2.7969615384615386e+08 2023-10-21T04:50:58.777+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 10838601320854145501, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501, err: , elapsed(sec): 0.014499309 2023-10-21T04:50:58.779+05:30 [Info] /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/11711860383628641160/shards/shard11711860383628641160: metadata saved successfully 2023-10-21T04:50:58.786+05:30 [Info] tenant 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 (11711860383628641160) added 2023-10-21T04:50:58.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Recover shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 from metadata completed, tenant :bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b 2023-10-21T04:50:58.789+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : completed shard restore, download path /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir/plasma_storage_v1/78118431caff95cf657f177e70958ebf_ShardToken11_a9_c7_70_34_9f_8d_92/11711860383628641160/shards/shard11711860383628641160, newshardId 11711860383628641160, newpath /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160, shardsize 2874032, duration 26.045649ms 2023-10-21T04:50:58.789+05:30 [Info] ShardTransferManager::processShardRestoreMessage doneCb invoked for shardId: 11711860383628641160, path: /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160, err: , elapsed(sec): 0.02635999 2023-10-21T04:50:58.789+05:30 [Info] ShardRebalancer::startRestoreShard ShardId: 11711860383628641160 bytesWritten: 2874032, totalBytes: 2874032, transferRate: 1.105396923076923e+08 2023-10-21T04:50:58.789+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Invoked doneCb for taskType: 0, taskId: 78118431caff95cf657f177e70958ebf, transferId: ShardToken11:a9:c7:70:34:9f:8d:92 2023-10-21T04:50:58.789+05:30 [Info] ShardTransferManager::cleanupStagingDirOnRestore Clean-up done for staging directory for taskType: 0, taskId: 78118431caff95cf657f177e70958ebf, transferId: ShardToken11:a9:c7:70:34:9f:8d:92, elapsed(sec): 0.000269229 2023-10-21T04:50:58.789+05:30 [Info] ShardTransferManager::processShardRestoreMessage All shards are restored. Sending response errMap: map[10838601320854145501: 11711860383628641160:], shardPaths: map[10838601320854145501:/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 11711860383628641160:/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160], destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir, elapsed(sec): 0.026697552 2023-10-21T04:50:58.789+05:30 [Info] ShardRebalancer::startRestoreShard Received response for restore of shardIds: [10838601320854145501 11711860383628641160], ttid: ShardToken11:a9:c7:70:34:9f:8d:92, elapsed(sec): 0.027017377 2023-10-21T04:50:58.796+05:30 [Info] TestCode::TestActionAtTag: currTag: 3, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:50:58.811+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.815+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:50:58.815+05:30 [Info] ShardRebalancer::startShardRecovery Starting to recover index instance for ttid: ShardToken11:a9:c7:70:34:9f:8d:92 2023-10-21T04:50:58.815+05:30 [Info] StorageMgr::SetRebalanceRunning Setting rebalance flags for all slices with shardIds: map[10838601320854145501:true 11711860383628641160:true] 2023-10-21T04:50:58.815+05:30 [Info] StorageMgr::SetRebalanceRunning Done with setting rebalance flags for all slices. elapsed: 1.439µs 2023-10-21T04:50:58.815+05:30 [Info] ShardTransferManager::handleLockShardCommands Initiating shard locking for shards: [10838601320854145501 11711860383628641160], isLockedForRecovery: true 2023-10-21T04:50:58.815+05:30 [Info] ShardTransferManager::handleLockShardCommands Done with shard locking for shardIds: [10838601320854145501 11711860383628641160], errMap: map[10838601320854145501: 11711860383628641160:], elapsed: 10.673µs 2023-10-21T04:50:58.817+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:50:58.817+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned, defnId: 4390029557575378783, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:50:58.832+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenRestoreShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.832+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:50:58.845+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 12606056968936758026,realInstId: 0, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:50:58.846+05:30 [Info] Indexer::handleRecoverIndex InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:50:58.846+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 510.063µs 2023-10-21T04:50:58.846+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 558.756µs 2023-10-21T04:50:58.846+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:58.847+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 5 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:58.847+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 12606056968936758026, indexDefn DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:50:58.847+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 30.767043ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:58.848+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': added 2023-10-21T04:50:58.851+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data(shard11711860383628641160) : LSSCleaner initialized 2023-10-21T04:50:58.851+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data(shard11711860383628641160) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:50:58.851+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:58.854+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery(shard11711860383628641160) : LSSCleaner initialized for recovery 2023-10-21T04:50:58.854+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery(shard11711860383628641160) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:50:58.854+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery 2023-10-21T04:50:58.857+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:58.858+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data], Shared [true] 2023-10-21T04:50:58.860+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data(shard10838601320854145501) : LSSCleaner initialized 2023-10-21T04:50:58.861+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data(shard10838601320854145501) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:50:58.862+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:58.861+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1314816] Data log Head [0] Tail [1540096] 2023-10-21T04:50:58.865+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery(shard11711860383628641160) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1314816] 2023-10-21T04:50:58.864+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery(shard10838601320854145501) : LSSCleaner initialized for recovery 2023-10-21T04:50:58.866+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery(shard10838601320854145501) : initCompression: Resolved compression algo, using [zstd1] 2023-10-21T04:50:58.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : LSSCtx Created Successfully. Path=/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery 2023-10-21T04:50:58.866+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:58.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.doRecovery: Starting recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data], Shared [true] 2023-10-21T04:50:58.867+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.doRecovery: Recovery log Head[0], Tail[1716224] Data log Head [0] Tail [1900544] 2023-10-21T04:50:58.867+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery(shard10838601320854145501) : recoverFromHeaderReplay: Begin recovering from recovery log, headOffset [0] tailOffset [1716224] 2023-10-21T04:50:58.874+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.884+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex 2023-10-21T04:50:58.877+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:50:58.879+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.885+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:50:58.887+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.888+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex 2023-10-21T04:50:58.893+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.893+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex 2023-10-21T04:50:58.893+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.893+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex 2023-10-21T04:50:58.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.897+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex 2023-10-21T04:50:58.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.897+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex 2023-10-21T04:50:58.897+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.897+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex 2023-10-21T04:50:58.898+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.898+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex 2023-10-21T04:50:58.898+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.898+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex 2023-10-21T04:50:58.898+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.899+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex 2023-10-21T04:50:58.907+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.907+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:50:58.908+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex 2023-10-21T04:50:58.909+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.909+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:50:58.909+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.910+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:50:58.911+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.911+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:50:58.912+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.911+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.912+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:50:58.914+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.919+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:50:58.919+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.919+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:58.919+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.920+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:50:58.922+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex 2023-10-21T04:50:58.922+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.923+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex 2023-10-21T04:50:58.923+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.923+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex 2023-10-21T04:50:58.923+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.923+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex 2023-10-21T04:50:58.924+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.924+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex 2023-10-21T04:50:58.924+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.924+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex 2023-10-21T04:50:58.925+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.925+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:50:58.925+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.925+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:50:58.925+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.925+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:50:58.925+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.925+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:50:58.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.926+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:50:58.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.926+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:50:58.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.926+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:58.926+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.926+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:50:58.930+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.936+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex 2023-10-21T04:50:58.937+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery(shard10838601320854145501) : replayOffset: 1892352 (RP:1884160) (CP:1892352) 2023-10-21T04:50:58.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1892352] took [70.869621ms] 2023-10-21T04:50:58.938+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1055626, resident 1, memSz 1693, memSzIndex 0, numRecords 18 2023-10-21T04:50:58.938+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645885 controller mem 1754 2023-10-21T04:50:58.939+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.939+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645885 controller mem 1754 2023-10-21T04:50:58.939+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1055832, resident 1, memSz 70174, memSzIndex 265, numRecords 1000 2023-10-21T04:50:58.939+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645885 controller mem 1754 2023-10-21T04:50:58.939+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.944+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645841 controller mem 1754 2023-10-21T04:50:58.944+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.945+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645797 controller mem 1754 2023-10-21T04:50:58.945+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.947+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645753 controller mem 1754 2023-10-21T04:50:58.947+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.947+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645709 controller mem 1754 2023-10-21T04:50:58.948+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.948+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645709 controller mem 1754 2023-10-21T04:50:58.953+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.953+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645709 controller mem 1754 2023-10-21T04:50:58.953+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.954+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645665 controller mem 1754 2023-10-21T04:50:58.954+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.956+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645665 controller mem 1754 2023-10-21T04:50:58.957+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.957+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645665 controller mem 1754 2023-10-21T04:50:58.959+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.959+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645621 controller mem 1754 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645577 controller mem 1754 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645533 controller mem 1754 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645533 controller mem 1754 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645533 controller mem 1754 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:58.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 44645533 controller mem 1754 2023-10-21T04:50:58.960+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data(shard10838601320854145501) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1900544] replayOffset [1892352] 2023-10-21T04:50:58.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [93.690775ms] 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.961+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex: Repair 0 added 0 deleted 2023-10-21T04:50:58.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data], Shared [true]. Built [18] plasmas, took [95.392009ms] 2023-10-21T04:50:58.962+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:58.963+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 23 count: 0 metaSz: 2482 rp [1] sn: 24 count: 0 metaSz: 2483 rp [2] sn: 25 count: 0 metaSz: 2483 2023-10-21T04:50:58.963+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:58.964+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex 2023-10-21T04:50:58.990+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenRecoverShard BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:50:58.990+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:0 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:50:59.001+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery(shard11711860383628641160) : replayOffset: 1531904 (RP:1466368) (CP:1531904) 2023-10-21T04:50:59.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.doRecovery: Done recovering from recovery log, replayOffset [1531904] took [136.363067ms] 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699761 controller mem 3474 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699761 controller mem 3474 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1056024, resident 1, memSz 68284, memSzIndex 688, numRecords 1000 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699761 controller mem 3474 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699717 controller mem 3474 2023-10-21T04:50:59.001+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699673 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699629 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699585 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699585 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699585 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699585 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699585 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699541 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699497 controller mem 3474 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.002+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699453 controller mem 3474 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048646, resident 0, memSz 64, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699409 controller mem 3474 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699409 controller mem 3474 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: request quota after header replay 1048576, resident 0, memSz 0, memSzIndex 0, numRecords 0 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: controller ('bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b') quota after header replay 45699409 controller mem 3474 2023-10-21T04:50:59.003+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data(shard11711860383628641160) : recoverFromDataReplay: Begin recovering from data log, headOffset [0] tailOffset [1540096] replayOffset [1531904] 2023-10-21T04:50:59.003+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.doRecovery: Done recovering from data and recovery log, took [138.366029ms] 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.003+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex: Repair 0 added 0 deleted 2023-10-21T04:50:59.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.doRecovery: Done recovery. Recovery log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery], Data log [/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data], Shared [true]. Built [17] plasmas, took [139.12132ms] 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 23 count: 0 metaSz: 2482 rp [1] sn: 24 count: 0 metaSz: 2483 rp [2] sn: 25 count: 0 metaSz: 2483 2023-10-21T04:50:59.004+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 161826 2023-10-21T04:50:59.005+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 161826 2023-10-21T04:50:59.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex 2023-10-21T04:50:59.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swapper started 2023-10-21T04:50:59.010+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex 2023-10-21T04:50:59.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex started 2023-10-21T04:50:59.010+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.011+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex 2023-10-21T04:50:59.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.030+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex 2023-10-21T04:50:59.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex started 2023-10-21T04:50:59.031+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:50:59.035+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.035+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.051+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:59.051+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:59.056+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 5 Warmup took 209.113216ms 2023-10-21T04:50:59.057+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 5 WriterThreads 1 2023-10-21T04:50:59.058+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index Index: 12606056968936758026 Partition: 5 2023-10-21T04:50:59.058+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.058+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 6 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.059+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.059+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 24 count: 0 metaSz: 2482 rp [1] sn: 25 count: 0 metaSz: 2483 rp [2] sn: 26 count: 0 metaSz: 2483 2023-10-21T04:50:59.057+05:30 [Info] switched currmeta from 485 -> 487 force true 2023-10-21T04:50:59.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.061+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.061+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 25 count: 0 metaSz: 2483 rp [1] sn: 26 count: 0 metaSz: 2483 2023-10-21T04:50:59.061+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 161826 2023-10-21T04:50:59.062+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 170040 2023-10-21T04:50:59.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex 2023-10-21T04:50:59.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.097+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex 2023-10-21T04:50:59.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex started 2023-10-21T04:50:59.098+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.098+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 5 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex 2023-10-21T04:50:59.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.107+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex 2023-10-21T04:50:59.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex started 2023-10-21T04:50:59.107+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:50:59.108+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.109+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.129+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:59.129+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:59.139+05:30 [Info] switched currmeta from 487 -> 487 force true 2023-10-21T04:50:59.143+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 6 Warmup took 84.616851ms 2023-10-21T04:50:59.144+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 6 WriterThreads 1 2023-10-21T04:50:59.145+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index Index: 12606056968936758026 Partition: 6 2023-10-21T04:50:59.145+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.146+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 7 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.147+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 24 count: 0 metaSz: 2483 rp [1] sn: 25 count: 0 metaSz: 2483 2023-10-21T04:50:59.148+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 94661 recovery UsedSpace 170040 2023-10-21T04:50:59.148+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.149+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.149+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 24 count: 0 metaSz: 2483 rp [1] sn: 25 count: 0 metaSz: 2483 2023-10-21T04:50:59.149+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 170040 2023-10-21T04:50:59.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex 2023-10-21T04:50:59.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.155+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.156+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex 2023-10-21T04:50:59.156+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex started 2023-10-21T04:50:59.156+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.156+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 6 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex 2023-10-21T04:50:59.159+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.160+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex 2023-10-21T04:50:59.161+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex started 2023-10-21T04:50:59.161+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:50:59.161+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.161+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.217+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 7 Warmup took 70.067734ms 2023-10-21T04:50:59.218+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 7 WriterThreads 1 2023-10-21T04:50:59.218+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index Index: 12606056968936758026 Partition: 7 2023-10-21T04:50:59.218+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.219+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 8 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.224+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.224+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 24 count: 0 metaSz: 2483 rp [1] sn: 25 count: 0 metaSz: 2483 2023-10-21T04:50:59.224+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.224+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.224+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 170040 2023-10-21T04:50:59.224+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 23 count: 0 metaSz: 2482 rp [1] sn: 24 count: 0 metaSz: 2483 rp [2] sn: 25 count: 0 metaSz: 2483 2023-10-21T04:50:59.225+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 161826 2023-10-21T04:50:59.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex 2023-10-21T04:50:59.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.240+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex 2023-10-21T04:50:59.240+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex started 2023-10-21T04:50:59.241+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.241+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 7 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex 2023-10-21T04:50:59.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.242+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex 2023-10-21T04:50:59.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex started 2023-10-21T04:50:59.242+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:50:59.243+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.243+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.264+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 8 Warmup took 44.376037ms 2023-10-21T04:50:59.264+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 8 WriterThreads 1 2023-10-21T04:50:59.264+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index Index: 12606056968936758026 Partition: 8 2023-10-21T04:50:59.264+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.264+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 1 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.264+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.264+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.264+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 25 count: 0 metaSz: 2482 rp [1] sn: 26 count: 0 metaSz: 2483 rp [2] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.265+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 161826 2023-10-21T04:50:59.265+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.265+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.265+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 26 count: 0 metaSz: 2483 rp [1] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.265+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 170040 2023-10-21T04:50:59.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex 2023-10-21T04:50:59.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.285+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex 2023-10-21T04:50:59.285+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex started 2023-10-21T04:50:59.285+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.285+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 8 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex 2023-10-21T04:50:59.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.286+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex 2023-10-21T04:50:59.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex started 2023-10-21T04:50:59.286+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:50:59.287+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.287+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.317+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 1 Warmup took 52.688375ms 2023-10-21T04:50:59.317+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 1 WriterThreads 1 2023-10-21T04:50:59.317+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index Index: 12606056968936758026 Partition: 1 2023-10-21T04:50:59.317+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.318+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 2 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.318+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.318+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 25 count: 0 metaSz: 2482 rp [1] sn: 26 count: 0 metaSz: 2483 rp [2] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.318+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 161826 2023-10-21T04:50:59.318+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.318+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.318+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 26 count: 0 metaSz: 2483 rp [1] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.319+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 170040 2023-10-21T04:50:59.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex 2023-10-21T04:50:59.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.330+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex 2023-10-21T04:50:59.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex started 2023-10-21T04:50:59.330+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.330+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 9 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex 2023-10-21T04:50:59.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.331+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex 2023-10-21T04:50:59.331+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex started 2023-10-21T04:50:59.331+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:50:59.331+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.331+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.352+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 2 Warmup took 34.187365ms 2023-10-21T04:50:59.352+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 2 WriterThreads 1 2023-10-21T04:50:59.352+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index Index: 12606056968936758026 Partition: 2 2023-10-21T04:50:59.352+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.352+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 3 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.353+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.353+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 25 count: 0 metaSz: 2482 rp [1] sn: 26 count: 0 metaSz: 2483 rp [2] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.353+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 161826 2023-10-21T04:50:59.353+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.353+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.353+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 26 count: 0 metaSz: 2483 rp [1] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.354+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 111872 recovery UsedSpace 170040 2023-10-21T04:50:59.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex 2023-10-21T04:50:59.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.362+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex 2023-10-21T04:50:59.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex started 2023-10-21T04:50:59.362+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.363+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex 2023-10-21T04:50:59.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.363+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex 2023-10-21T04:50:59.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex started 2023-10-21T04:50:59.363+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:50:59.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.395+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 3 Warmup took 42.972975ms 2023-10-21T04:50:59.396+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 3 WriterThreads 1 2023-10-21T04:50:59.396+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index Index: 12606056968936758026 Partition: 3 2023-10-21T04:50:59.396+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 12606056968936758026 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.396+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 12606056968936758026 PartitionId: 4 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.396+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.397+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 20 numRPs: 3 rp [0] sn: 25 count: 0 metaSz: 2482 rp [1] sn: 26 count: 0 metaSz: 2483 rp [2] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048646 memSz 64 2023-10-21T04:50:59.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 26 count: 0 metaSz: 2483 rp [1] sn: 27 count: 0 metaSz: 2483 2023-10-21T04:50:59.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 123059 recovery UsedSpace 170040 2023-10-21T04:50:59.398+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 102550 recovery UsedSpace 161826 2023-10-21T04:50:59.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 4 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex 2023-10-21T04:50:59.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.411+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex 2023-10-21T04:50:59.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex started 2023-10-21T04:50:59.411+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 3 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex 2023-10-21T04:50:59.411+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.412+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex 2023-10-21T04:50:59.412+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex started 2023-10-21T04:50:59.412+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.412+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.412+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.412+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:50:59.431+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 12606056968936758026 PartitionId 4 Warmup took 35.277432ms 2023-10-21T04:50:59.431+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 12606056968936758026 partitionId 4 WriterThreads 1 2023-10-21T04:50:59.432+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 12606056968936758026 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index Index: 12606056968936758026 Partition: 4 2023-10-21T04:50:59.432+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 12606056968936758026, shardIdMap: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] err: 2023-10-21T04:50:59.432+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 12606056968936758026, partnShardIdMap: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]], err: 2023-10-21T04:50:59.432+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:50:59.433+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 12606056968936758026 2023-10-21T04:50:59.433+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:50:59.433+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:50:59.434+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:50:59.434+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:50:59.436+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:50:59.437+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 12606056968936758026 Partitions [8 1 2 3 4 5 6 7] 2023-10-21T04:50:59.437+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:8 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.438+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:1 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.439+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:2 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.440+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:3 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.440+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:4 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.442+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:5 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.443+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:6 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.443+05:30 [Info] StorageMgr::openSnapshot IndexInst:12606056968936758026 Partition:7 Attempting to open snapshot (SnapshotInfo: count:0 committed:false) 2023-10-21T04:50:59.445+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 12.815189ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:59.445+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 103.3µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:50:59.467+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:59.467+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:59.475+05:30 [Info] switched currmeta from 487 -> 488 force true 2023-10-21T04:50:59.545+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:59.545+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:59.548+05:30 [Info] switched currmeta from 488 -> 489 force true 2023-10-21T04:50:59.895+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[12606056968936758026:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:50:59.895+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:50:59.896+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_partitioned_defer, defnId: 17703248484023971553, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:50:59.913+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 8303154871847919944,realInstId: 0, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [1 2 3 4 5 6 7 8] 2023-10-21T04:50:59.913+05:30 [Info] Indexer::handleRecoverIndex InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:50:59.918+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 4.330994ms 2023-10-21T04:50:59.918+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 7 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:50:59.921+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 7 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:50:59.918+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 8303154871847919944, indexDefn DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:50:59.923+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 27.139751ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:50:59.924+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.924+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:50:59.918+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 4.746459ms 2023-10-21T04:50:59.925+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:50:59.926+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:50:59.927+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:50:59.929+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.930+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:50:59.931+05:30 [Info] switched currmeta from 489 -> 490 force true 2023-10-21T04:50:59.931+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:50:59.938+05:30 [Info] TestCode::TestActionAtTag: currTag: 4, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:50:59.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:59.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.946+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:59.946+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex started 2023-10-21T04:50:59.946+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.946+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:59.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.949+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:59.949+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex started 2023-10-21T04:50:59.949+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 7 Recovering from recovery point .. 2023-10-21T04:50:59.949+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 7 Unable to find recovery point. Resetting store .. 2023-10-21T04:50:59.949+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:50:59.949+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.949+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:50:59.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:50:59.950+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:50:59.950+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:59.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex stopped 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:50:59.950+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:50:59.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:50:59.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:50:59.960+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex closed 2023-10-21T04:50:59.960+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:50:59.960+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:50:59.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:50:59.961+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:50:59.961+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:59.961+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex stopped 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:50:59.961+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:50:59.970+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:50:59.970+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:50:59.970+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex closed 2023-10-21T04:50:59.971+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:50:59.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex ... 2023-10-21T04:50:59.971+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex ... 2023-10-21T04:50:59.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:50:59.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex sucessfully destroyed 2023-10-21T04:50:59.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex ... 2023-10-21T04:50:59.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex ... 2023-10-21T04:50:59.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:50:59.980+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index destroyed 2023-10-21T04:50:59.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:50:59.984+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:50:59.984+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:59.984+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:50:59.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:50:59.990+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:50:59.990+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:50:59.990+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:50:59.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:59.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:50:59.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:50:59.995+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:59.995+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:50:59.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex started 2023-10-21T04:50:59.995+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.995+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:59.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:50:59.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:50:59.998+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:59.998+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:50:59.998+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex started 2023-10-21T04:50:59.999+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:50:59.999+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 7 Warmup took 76.928997ms 2023-10-21T04:50:59.999+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:50:59.999+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 7 WriterThreads 1 2023-10-21T04:50:59.999+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index Index: 8303154871847919944 Partition: 7 2023-10-21T04:50:59.999+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 8 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.000+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 8 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.000+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.000+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.000+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:51:00.000+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:51:00.001+05:30 DEBU REGU.throttle.runStatSlotResetter() at nodecapstats.go:36 [id 234] clearing stat slot 2 (current slot 1) 2023-10-21T04:51:00.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 17 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:51:00.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.008+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:51:00.008+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex started 2023-10-21T04:51:00.008+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 18 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:51:00.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.009+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:51:00.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex started 2023-10-21T04:51:00.009+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 8 Recovering from recovery point .. 2023-10-21T04:51:00.009+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 8 Unable to find recovery point. Resetting store .. 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:51:00.009+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:51:00.009+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:51:00.009+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex stopped 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.009+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.014+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:51:00.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex closed 2023-10-21T04:51:00.014+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:00.014+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.014+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.014+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.014+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:51:00.014+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:51:00.014+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:51:00.015+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:51:00.015+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:51:00.015+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex stopped 2023-10-21T04:51:00.015+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.015+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.015+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.015+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.021+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:51:00.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex closed 2023-10-21T04:51:00.021+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:00.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex ... 2023-10-21T04:51:00.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 18 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex ... 2023-10-21T04:51:00.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex sucessfully destroyed 2023-10-21T04:51:00.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex ... 2023-10-21T04:51:00.023+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 17 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex ... 2023-10-21T04:51:00.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index destroyed 2023-10-21T04:51:00.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.029+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.029+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.029+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.032+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 27 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:51:00.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.035+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:51:00.035+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:51:00.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex started 2023-10-21T04:51:00.035+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.035+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:51:00.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.043+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:51:00.043+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:51:00.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex started 2023-10-21T04:51:00.043+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 8 Warmup took 43.137794ms 2023-10-21T04:51:00.043+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 8 WriterThreads 1 2023-10-21T04:51:00.043+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index Index: 8303154871847919944 Partition: 8 2023-10-21T04:51:00.043+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 1 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.043+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 1 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.043+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.043+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.044+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 68266 recovery UsedSpace 8236 2023-10-21T04:51:00.045+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.045+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.045+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.045+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.045+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:51:00.062+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:00.062+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:00.064+05:30 [Info] switched currmeta from 490 -> 491 force true 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 10 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.074+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex started 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.074+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex started 2023-10-21T04:51:00.074+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 1 Recovering from recovery point .. 2023-10-21T04:51:00.074+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 1 Unable to find recovery point. Resetting store .. 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:51:00.074+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:51:00.074+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:51:00.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex stopped 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.074+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.075+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.085+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:51:00.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex closed 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:51:00.086+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:51:00.086+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:51:00.086+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex stopped 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.086+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.098+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:51:00.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex closed 2023-10-21T04:51:00.098+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:00.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex ... 2023-10-21T04:51:00.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex ... 2023-10-21T04:51:00.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex sucessfully destroyed 2023-10-21T04:51:00.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex ... 2023-10-21T04:51:00.114+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex ... 2023-10-21T04:51:00.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.116+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index destroyed 2023-10-21T04:51:00.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.120+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.120+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.120+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.124+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.124+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.124+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 28 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:51:00.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.129+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:51:00.129+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:51:00.129+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex started 2023-10-21T04:51:00.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.129+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:51:00.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.133+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:51:00.133+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:51:00.133+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex started 2023-10-21T04:51:00.133+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 1 Warmup took 89.971243ms 2023-10-21T04:51:00.134+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 1 WriterThreads 1 2023-10-21T04:51:00.134+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index Index: 8303154871847919944 Partition: 1 2023-10-21T04:51:00.134+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 2 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.134+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 2 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.134+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.134+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.134+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.134+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.135+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:51:00.135+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.135+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.135+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 11 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.143+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex started 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.143+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:51:00.143+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex started 2023-10-21T04:51:00.143+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.143+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.143+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.143+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.143+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 2 Recovering from recovery point .. 2023-10-21T04:51:00.143+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 2 Unable to find recovery point. Resetting store .. 2023-10-21T04:51:00.143+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:51:00.144+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:51:00.144+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:51:00.144+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex stopped 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.144+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.163+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:51:00.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex closed 2023-10-21T04:51:00.163+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:00.163+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.163+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.163+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:51:00.164+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:51:00.164+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:51:00.164+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:51:00.164+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:51:00.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex stopped 2023-10-21T04:51:00.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.180+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:51:00.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex closed 2023-10-21T04:51:00.180+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:00.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex ... 2023-10-21T04:51:00.180+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex ... 2023-10-21T04:51:00.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex sucessfully destroyed 2023-10-21T04:51:00.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex ... 2023-10-21T04:51:00.190+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 11 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex ... 2023-10-21T04:51:00.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.194+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index destroyed 2023-10-21T04:51:00.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.209+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.209+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.210+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.219+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.219+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.219+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 29 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:51:00.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.225+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:51:00.225+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:51:00.225+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex started 2023-10-21T04:51:00.225+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.225+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:51:00.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.230+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:51:00.230+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:51:00.230+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex started 2023-10-21T04:51:00.230+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 2 Warmup took 96.307793ms 2023-10-21T04:51:00.231+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.231+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.231+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 2 WriterThreads 1 2023-10-21T04:51:00.231+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index Index: 8303154871847919944 Partition: 2 2023-10-21T04:51:00.231+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 3 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.231+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 3 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.231+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.231+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.232+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:51:00.232+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.232+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.232+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 12 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.241+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex started 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.241+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex started 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.241+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 3 Recovering from recovery point .. 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.241+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 3 Unable to find recovery point. Resetting store .. 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:51:00.241+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:51:00.241+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:51:00.241+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:51:00.241+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex stopped 2023-10-21T04:51:00.242+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.242+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.242+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.242+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.242+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.250+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:51:00.250+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex closed 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:51:00.251+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:51:00.251+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:51:00.251+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex stopped 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.251+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.259+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:51:00.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex closed 2023-10-21T04:51:00.259+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:00.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex ... 2023-10-21T04:51:00.259+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex ... 2023-10-21T04:51:00.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex sucessfully destroyed 2023-10-21T04:51:00.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex ... 2023-10-21T04:51:00.267+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 12 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex ... 2023-10-21T04:51:00.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.277+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index destroyed 2023-10-21T04:51:00.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.286+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.286+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.286+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.296+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.297+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.297+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 30 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:51:00.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.305+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:51:00.305+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:51:00.305+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex started 2023-10-21T04:51:00.305+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.305+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:51:00.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.314+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:51:00.314+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:51:00.314+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex started 2023-10-21T04:51:00.314+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 3 Warmup took 83.111219ms 2023-10-21T04:51:00.314+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 3 WriterThreads 1 2023-10-21T04:51:00.314+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index Index: 8303154871847919944 Partition: 3 2023-10-21T04:51:00.314+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 4 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.315+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 4 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.315+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.315+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.315+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:51:00.315+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.315+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.316+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.316+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.316+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:51:00.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:51:00.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 13 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:51:00.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.329+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:51:00.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex started 2023-10-21T04:51:00.329+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.330+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:51:00.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex started 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.330+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 4 Recovering from recovery point .. 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.330+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 4 Unable to find recovery point. Resetting store .. 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:51:00.330+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:51:00.330+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:51:00.330+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex stopped 2023-10-21T04:51:00.330+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.331+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.331+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.331+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.337+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:51:00.337+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex closed 2023-10-21T04:51:00.337+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:51:00.338+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:51:00.338+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:51:00.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex stopped 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.338+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.345+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:51:00.345+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex closed 2023-10-21T04:51:00.346+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:00.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex ... 2023-10-21T04:51:00.346+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex ... 2023-10-21T04:51:00.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex sucessfully destroyed 2023-10-21T04:51:00.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex ... 2023-10-21T04:51:00.349+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 13 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex ... 2023-10-21T04:51:00.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.352+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index destroyed 2023-10-21T04:51:00.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.356+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.357+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.357+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.361+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.361+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 31 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:51:00.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:51:00.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:51:00.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex started 2023-10-21T04:51:00.364+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.365+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:51:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.371+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:51:00.371+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:51:00.371+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex started 2023-10-21T04:51:00.371+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 4 Warmup took 56.251304ms 2023-10-21T04:51:00.371+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 4 WriterThreads 1 2023-10-21T04:51:00.371+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index Index: 8303154871847919944 Partition: 4 2023-10-21T04:51:00.371+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 5 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.371+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.371+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.373+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 5 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.374+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.374+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:51:00.374+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.374+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.374+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.396+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex started 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 14 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.396+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex started 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.396+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 5 Recovering from recovery point .. 2023-10-21T04:51:00.396+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 5 Unable to find recovery point. Resetting store .. 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.396+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:51:00.396+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:51:00.396+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:51:00.396+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:51:00.397+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex stopped 2023-10-21T04:51:00.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:51:00.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex closed 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] 2023-10-21T04:51:00.405+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] 2023-10-21T04:51:00.405+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:51:00.405+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex stopped 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.405+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.413+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] 2023-10-21T04:51:00.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex closed 2023-10-21T04:51:00.413+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:00.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex ... 2023-10-21T04:51:00.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex ... 2023-10-21T04:51:00.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex sucessfully destroyed 2023-10-21T04:51:00.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex ... 2023-10-21T04:51:00.416+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 14 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex ... 2023-10-21T04:51:00.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.430+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index destroyed 2023-10-21T04:51:00.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.443+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.443+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.443+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.448+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.448+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.448+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 32 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:51:00.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.459+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:51:00.459+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:51:00.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex started 2023-10-21T04:51:00.459+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.459+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:51:00.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.475+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:51:00.475+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:51:00.475+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex started 2023-10-21T04:51:00.475+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.475+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 5 Warmup took 101.607753ms 2023-10-21T04:51:00.475+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 5 WriterThreads 1 2023-10-21T04:51:00.475+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index Index: 8303154871847919944 Partition: 5 2023-10-21T04:51:00.475+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 8303154871847919944 Partition: PartitionId: 6 Endpoints: [:9123] , shardIds: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.475+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.475+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 8303154871847919944 PartitionId: 6 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.476+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.476+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 74472 recovery UsedSpace 8236 2023-10-21T04:51:00.476+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.476+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1048576 memSz 0 2023-10-21T04:51:00.476+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 81920 recovery UsedSpace 8236 2023-10-21T04:51:00.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 15 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:51:00.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.485+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:51:00.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex started 2023-10-21T04:51:00.485+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.485+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 16 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:51:00.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.485+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.486+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:51:00.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex started 2023-10-21T04:51:00.486+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 6 Recovering from recovery point .. 2023-10-21T04:51:00.486+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 6 Unable to find recovery point. Resetting store .. 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:51:00.486+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:51:00.486+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:51:00.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex stopped 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.486+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.494+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:51:00.494+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex closed 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:00.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:51:00.495+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:51:00.495+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:51:00.495+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex stopped 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:00.495+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:00.503+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:00.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:51:00.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex closed 2023-10-21T04:51:00.503+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:00.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex ... 2023-10-21T04:51:00.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 16 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex ... 2023-10-21T04:51:00.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex sucessfully destroyed 2023-10-21T04:51:00.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex ... 2023-10-21T04:51:00.506+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 15 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex ... 2023-10-21T04:51:00.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index destroyed 2023-10-21T04:51:00.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.513+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.513+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.513+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:00.518+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.518+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:00.518+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:00.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 33 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:51:00.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:00.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:00.527+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:51:00.527+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:51:00.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex started 2023-10-21T04:51:00.527+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.527+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:51:00.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:00.534+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:00.534+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:51:00.535+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:51:00.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex started 2023-10-21T04:51:00.535+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 8303154871847919944 PartitionId 6 Warmup took 59.003143ms 2023-10-21T04:51:00.535+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 8303154871847919944 partitionId 6 WriterThreads 1 2023-10-21T04:51:00.535+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 8303154871847919944 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index Index: 8303154871847919944 Partition: 6 2023-10-21T04:51:00.535+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 8303154871847919944, shardIdMap: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]] err: 2023-10-21T04:51:00.535+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 8303154871847919944, partnShardIdMap: map[1:[10838601320854145501 11711860383628641160] 2:[10838601320854145501 11711860383628641160] 3:[10838601320854145501 11711860383628641160] 4:[10838601320854145501 11711860383628641160] 5:[10838601320854145501 11711860383628641160] 6:[10838601320854145501 11711860383628641160] 7:[10838601320854145501 11711860383628641160] 8:[10838601320854145501 11711860383628641160]], err: 2023-10-21T04:51:00.535+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_READY RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:00.535+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 8303154871847919944 2023-10-21T04:51:00.535+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 8303154871847919944 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:00.535+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[1:PartitionId: 1 Endpoints: [:9123] 2:PartitionId: 2 Endpoints: [:9123] 3:PartitionId: 3 Endpoints: [:9123] 4:PartitionId: 4 Endpoints: [:9123] 5:PartitionId: 5 Endpoints: [:9123] 6:PartitionId: 6 Endpoints: [:9123] 7:PartitionId: 7 Endpoints: [:9123] 8:PartitionId: 8 Endpoints: [:9123] ] 2023-10-21T04:51:00.536+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:00.536+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 8303154871847919944 Name: idx_partitioned_defer Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:00.536+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:00.536+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8303154871847919944 Partitions [3 4 5 6 7 8 1 2] 2023-10-21T04:51:00.536+05:30 [Info] StorageMgr::openSnapshot IndexInst:8303154871847919944 Partition:4 No Snapshot Found. 2023-10-21T04:51:00.536+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 8303154871847919944 Adding Nil Snapshot. 2023-10-21T04:51:00.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:00.536+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:00.561+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 24.801357ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:00.561+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 121.764µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:51:00.610+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:00.610+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:00.612+05:30 [Info] switched currmeta from 491 -> 492 force true 2023-10-21T04:51:00.642+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:51:00.652+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:00.652+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:00.655+05:30 [Info] switched currmeta from 492 -> 492 force true 2023-10-21T04:51:00.715+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:00.715+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:00.718+05:30 [Info] switched currmeta from 492 -> 493 force true 2023-10-21T04:51:00.945+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[8303154871847919944:INDEX_STATE_READY] reached state: INDEX_STATE_READY 2023-10-21T04:51:00.945+05:30 [Info] Indexer::handleMergePartition Source 9226953440900345886 Target 8303154871847919944 2023-10-21T04:51:00.945+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00764f980 2023-10-21T04:51:00.945+05:30 [Info] MergePartition: sent async request to update index instance 8303154871847919944 rstate moved to ACTIVE 2023-10-21T04:51:00.952+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.911811ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:00.952+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 9226953440900345886, realInst: 8303154871847919944 2023-10-21T04:51:00.954+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 7735644145307932837, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:51:00.954+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 7735644145307932837, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:51:00.978+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 2047508721584820382,realInstId: 0, indexDefn DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:51:00.979+05:30 [Info] Indexer::handleRecoverIndex InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:51:00.979+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 240.823µs 2023-10-21T04:51:00.979+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 284.657µs 2023-10-21T04:51:00.980+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2047508721584820382 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:00.980+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 2047508721584820382 PartitionId: 0 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:00.981+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:51:00.985+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1056024 memSz 1720 2023-10-21T04:51:00.986+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 15 count: 1000 metaSz: 2507 rp [1] sn: 16 count: 1000 metaSz: 2519 2023-10-21T04:51:00.986+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 238809 recovery UsedSpace 24190 2023-10-21T04:51:00.983+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 2047508721584820382, indexDefn DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:51:00.996+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 41.149658ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:00.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:00.995+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:00.997+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:00.997+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1055832 memSz 666 2023-10-21T04:51:01.003+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 3 numRPs: 2 rp [0] sn: 15 count: 1000 metaSz: 2507 rp [1] sn: 16 count: 1000 metaSz: 2519 2023-10-21T04:51:01.001+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 25 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex 2023-10-21T04:51:01.002+05:30 [Info] switched currmeta from 493 -> 495 force true 2023-10-21T04:51:01.003+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:01.004+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 258280 recovery UsedSpace 22108 2023-10-21T04:51:01.004+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:51:01.006+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:51:01.007+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex 2023-10-21T04:51:01.007+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex started 2023-10-21T04:51:01.007+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:01.009+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:01.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 26 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex 2023-10-21T04:51:01.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:01.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:01.012+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex 2023-10-21T04:51:01.012+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex started 2023-10-21T04:51:01.012+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2047508721584820382 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:51:01.012+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:01.012+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:01.019+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:01.019+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:01.023+05:30 [Info] switched currmeta from 495 -> 495 force true 2023-10-21T04:51:01.030+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 2047508721584820382 PartitionId 0 Warmup took 49.463622ms 2023-10-21T04:51:01.030+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2047508721584820382 partitionId 0 WriterThreads 4 2023-10-21T04:51:01.031+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2047508721584820382 PartitionId 0 Stopped Billing WUs 2023-10-21T04:51:01.031+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2047508721584820382 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index Index: 2047508721584820382 Partition: 0 2023-10-21T04:51:01.031+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 2047508721584820382, shardIdMap: map[0:[10838601320854145501 11711860383628641160]] err: 2023-10-21T04:51:01.031+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 2047508721584820382, partnShardIdMap: map[0:[10838601320854145501 11711860383628641160]], err: 2023-10-21T04:51:01.031+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:01.032+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2047508721584820382 2023-10-21T04:51:01.032+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:51:01.032+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:51:01.032+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:01.033+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:51:01.033+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:01.033+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 2047508721584820382 Partitions [0] 2023-10-21T04:51:01.034+05:30 [Info] StorageMgr::openSnapshot IndexInst:2047508721584820382 Partition:0 Attempting to open snapshot (SnapshotInfo: count:1000 committed:false) 2023-10-21T04:51:01.039+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 7.80949ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:01.039+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 76.212µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:51:01.052+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:01.052+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:01.061+05:30 [Info] switched currmeta from 495 -> 496 force true 2023-10-21T04:51:01.118+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:01.118+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:01.125+05:30 [Info] switched currmeta from 496 -> 498 force true 2023-10-21T04:51:01.132+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:01.132+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:01.134+05:30 [Info] switched currmeta from 498 -> 498 force true 2023-10-21T04:51:01.196+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:01.197+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:01.201+05:30 [Info] switched currmeta from 498 -> 499 force true 2023-10-21T04:51:02.015+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[2047508721584820382:INDEX_STATE_RECOVERED 12606056968936758026:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:51:02.015+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [4390029557575378783 7735644145307932837] belonging to collectionId: c, ttid: ShardToken11:a9:c7:70:34:9f:8d:92. Initiating build 2023-10-21T04:51:02.047+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [12606056968936758026 2047508721584820382] &{2} 2023-10-21T04:51:02.047+05:30 [Info] Indexer::handleBuildRecoveredIndexes [12606056968936758026 2047508721584820382] 2023-10-21T04:51:02.047+05:30 [Info] bulkUpdateRState: Index instance 12606056968936758026 rstate moved to PENDING 2023-10-21T04:51:02.047+05:30 [Info] bulkUpdateRState: Index instance 2047508721584820382 rstate moved to PENDING 2023-10-21T04:51:02.047+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [12606056968936758026 2047508721584820382] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:51:02.047+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:51:02.047+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:02.048+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:51:02.060+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:51:02.061+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:02.062+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:02.066+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43324 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:02.064+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:02.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:02.071+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:51:02.075+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:51:02.076+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:02.078+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43324 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:02.079+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:51:02.081+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:02.082+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:02.083+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:51:02.084+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:51:02.085+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId 43325 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} 2023-10-21T04:51:02.087+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:02.088+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:02.088+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:51:02.088+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:4] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:51:02.090+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 12606056968936758026 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:51:02.091+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_1:4] INIT_STREAM:map[bucket_%2:_default:c1:2]] 2023-10-21T04:51:02.091+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 2047508721584820382 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:51:02.093+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:51:02.095+05:30 [Info] switched currmeta from 499 -> 501 force true 2023-10-21T04:51:02.094+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=19,1=31,2=27,3=21,4=25,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=25,17=20,18=20,19=25,20=23,21=28,22=30,23=20,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=39,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=27,57=18,58=26,59=16,60=20,61=24,62=20,63=28] IndexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 43325 CollectionId: c CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} NumVBuckets: 64 2023-10-21T04:51:02.098+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:51:02.098+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:51:02.099+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:02.103+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 Instances [indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:51:02.104+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:02.106+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:02.104+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:51:02.107+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 60.410673ms 2023-10-21T04:51:02.107+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [12606056968936758026 2047508721584820382] 2023-10-21T04:51:02.108+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 92.368974ms len(expediates) 2 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:02.107+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 60.581136ms 2023-10-21T04:51:02.108+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[2047508721584820382:true 12606056968936758026:true] 2023-10-21T04:51:02.114+05:30 [Info] switched currmeta from 501 -> 501 force true 2023-10-21T04:51:02.134+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 26.067373ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:02.141+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.951622ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:02.145+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:02.154+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:02.154+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:02.160+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:37592" doAuth() ... 2023-10-21T04:51:02.160+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:37592" auth successful 2023-10-21T04:51:02.161+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:37592" +1 2023-10-21T04:51:02.179+05:30 [Info] switched currmeta from 501 -> 503 force true 2023-10-21T04:51:02.209+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:02.210+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:02.277+05:30 [Info] switched currmeta from 503 -> 505 force true 2023-10-21T04:51:02.287+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:02.288+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:02.294+05:30 [Info] switched currmeta from 505 -> 505 force true 2023-10-21T04:51:02.355+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:02.355+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:02.358+05:30 [Info] switched currmeta from 505 -> 506 force true 2023-10-21T04:51:02.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:02.391+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:02.398+05:30 [Info] switched currmeta from 506 -> 507 force true 2023-10-21T04:51:02.533+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 99.7326203208504 InstId: 12606056968936758026 RealInstId: 12606056968936758026 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:51:02.534+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 99.7326203208504 InstId: 2047508721584820382 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:51:02.534+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 99.7326203208504 InstId: 4397940545762329850 RealInstId: 4397940545762329850 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:51:02.534+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 99.7326203208504 InstId: 14190651729632750031 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:51:02.534+05:30 [Info] ShardRebalancer::computeProgress 0.7029979946524064 2023-10-21T04:51:03.004+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 220432454230336 19 43325 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.007+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 135940091877708 21 43325 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.007+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 14580309443106 31 43325 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:51:03.010+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 80577332946777 27 43325 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:03.010+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 247041243079227 25 43325 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.014+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 24] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:51:03.014+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 23] call. taskList: &{Rev:[0 0 0 0 0 0 0 24] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/78118431caff95cf657f177e70958ebf Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]} {Rev:[0 0 0 0 0 0 0 1] ID:rebalance/78118431caff95cf657f177e70958ebf Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7029979946524064 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]}]} 2023-10-21T04:51:03.016+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 194695637011621 26 43325 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.016+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 172199206487624 16 43325 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:51:03.017+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 24] 2023-10-21T04:51:03.021+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 24] 2023-10-21T04:51:03.022+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 121484100363817 20 43325 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.022+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 182410957121870 32 43325 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:51:03.022+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 19010051182641 24 43325 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.022+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 68019130447759 21 43325 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.022+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 42756027613197 22 43325 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.027+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 171853572917441 25 43325 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.027+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 1542343912406 21 43325 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.028+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 157626280229441 25 43325 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:51:03.028+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 59183965998125 22 43325 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.028+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 107902021898438 23 43325 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:03.029+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 73693878880740 20 43325 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.033+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 217154782530068 23 43325 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:03.033+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 170065152021297 25 43325 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.036+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 67844364342499 20 43325 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.037+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 16864753682212 30 43325 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:51:03.037+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 194038385561911 28 43325 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:51:03.039+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 206194254469959 20 43325 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.043+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 233111027052204 28 43325 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:51:03.043+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 170842850732116 20 43325 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.047+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 178705476290398 24 43325 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.048+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 252516046855767 27 43325 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:03.053+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 183084439776991 19 43325 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.053+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 52334512457128 20 43325 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.053+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 58389099238647 24 43325 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.053+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 270670782796252 22 43325 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.055+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 86344521159606 25 43325 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.056+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 233119567174084 20 43325 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.058+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 187961911866991 30 43325 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:51:03.058+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 264478627574156 22 43325 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 276412965411368 22 43325 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.060+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 94748374195527 19 43325 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 66391235022367 25 43325 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.063+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 30543404237888 20 43325 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.065+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 168721574304277 24 43325 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.066+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 50379500651629 21 43325 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.069+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 100004485169771 26 43325 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.069+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 148816420612283 27 43325 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:03.075+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 84841900064407 22 43325 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.076+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 199648020746400 23 43325 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:03.076+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 236414409908581 20 43325 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.076+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 199348794242318 39 43325 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:51:03.079+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 131255717944569 26 43325 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.082+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 27094034037488 19 43325 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.082+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 229485491476250 19 43325 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.082+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 28675906453951 21 43325 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.088+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 268905923659105 24 43325 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.096+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 43207585340528 18 43325 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:51:03.096+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 47128363244154 26 43325 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.096+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 7777834518711 31 43325 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:51:03.096+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 75978859756835 17 43325 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:51:03.097+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 70030494603811 21 43325 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.097+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 235447613990794 27 43325 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:51:03.101+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 160015043890055 16 43325 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:51:03.103+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 154979908447073 24 43325 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.103+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 65202412270069 20 43325 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.106+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 170481067504432 20 43325 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.109+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 14950536761368 28 43325 172.31.5.112:9000. HWT [28-28,28]. 2023-10-21T04:51:03.122+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 InstanceIds [12606056968936758026 2047508721584820382] 2023-10-21T04:51:03.122+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 43325 2023-10-21T04:51:03.122+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43325 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 12606056968936758026 INDEX_STATE_CATCHUP 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 2047508721584820382 INDEX_STATE_CATCHUP 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 12606056968936758026 true 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 2047508721584820382 true 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 2047508721584820382 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 43325 BuildTS: [0=19,1=31,2=27,3=21,4=25,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=25,17=20,18=20,19=25,20=23,21=28,22=30,23=20,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=39,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=27,57=18,58=26,59=16,60=20,61=24,62=20,63=28] 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} , fetchKVSeq false 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:51:03.122+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:51:03.123+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43325 STREAM_ACTIVE 2023-10-21T04:51:03.123+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 43325 2023-10-21T04:51:03.123+05:30 [Info] Indexer::processBuildDoneNoCatchup INIT_STREAM bucket_%2:_default:c1 43325 2023-10-21T04:51:03.123+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:51:03.123+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:03.123+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 0 2023-10-21T04:51:03.123+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:03.124+05:30 [Info] send progress stats to clients 2023-10-21T04:51:03.124+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43325 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:03.124+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 12606056968936758026 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:51:03.124+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 2047508721584820382 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:51:03.124+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:51:03.124+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:51:03.124+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:03.124+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43325 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:03.124+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:51:03.124+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:51:03.129+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:51:03.129+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:51:03.129+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:0 1:0 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:0 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:0 60:0 61:0 62:0 63:0] 2023-10-21T04:51:03.130+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:51:03.130+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:37592: use of closed network connection 2023-10-21T04:51:03.130+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:37592" exit: read tcp4 127.0.0.1:9121->127.0.0.1:37592: use of closed network connection 2023-10-21T04:51:03.130+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:37592" closed ! 2023-10-21T04:51:03.130+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:51:03.130+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:03.130+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:03.130+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:03.130+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:03.130+05:30 [Info] Indexer::handleUpdateBuildTs Skipping updateBuildTs message to timekeeper as stream: INIT_STREAM is in state: STREAM_INACTIVE for keyspaceId: bucket_%2:_default:c1 2023-10-21T04:51:03.130+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:51:03.133+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.521116ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:03.144+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.031356ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:03.163+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:51:03.163+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:51:03.163+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:51:03.164+05:30 [Info] Indexer::processBuildDoneNoCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43325 2023-10-21T04:51:03.169+05:30 [Info] Indexer::handleBuildDoneNoCatchupAck INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:51:03.169+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart MAINT_STREAM bucket_%2 2023-10-21T04:51:03.169+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:03.170+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:03.170+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:03.170+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:51:03.170+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:51:03.170+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:03.170+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73679 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:03.170+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_1:{0xc00a13fc80 }] 2023-10-21T04:51:03.170+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:03.170+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:03.170+05:30 [Info] Indexer::resetBucketPauseState MAINT_STREAM bucket_%2 reset bucket state to NIL 2023-10-21T04:51:03.170+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:51:03.170+05:30 [Info] Indexer::startKeyspaceIdStream Stream: MAINT_STREAM KeyspaceId: bucket_%2 SessionId 73680 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} 2023-10-21T04:51:03.175+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:03.175+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:03.176+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2 Added for Stream MAINT_STREAM 2023-10-21T04:51:03.176+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:1 bucket_1:4] INIT_STREAM:map[]] 2023-10-21T04:51:03.176+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:4] INIT_STREAM:map[]] 2023-10-21T04:51:03.176+05:30 [Info] Timekeeper::startTimer MAINT_STREAM bucket_%2 2023-10-21T04:51:03.183+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:03.176+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 73680 CollectionId: CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} NumVBuckets: 64 2023-10-21T04:51:03.184+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0006943c0 } bucket_1:{0xc00a13fc80 }] 2023-10-21T04:51:03.185+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc006853080 2023-10-21T04:51:03.185+05:30 [Info] handleUpdateIndexRState: Index instance 2047508721584820382 rstate moved to ACTIVE 2023-10-21T04:51:03.185+05:30 [Info] Indexer::handleMergePartition Source 15757988232943046271 Target 12606056968936758026 2023-10-21T04:51:03.185+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc00992cde0 2023-10-21T04:51:03.185+05:30 [Info] MergePartition: sent async request to update index instance 12606056968936758026 rstate moved to ACTIVE 2023-10-21T04:51:03.191+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2 Instances [indexInstance:([`emalid`]) partitionScheme:KEY whereExpression:() > singlePartn:> indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:51:03.206+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 19.406671ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:03.208+05:30 [Info] ShardRebalancer::waitForIndexState Missing defnStats for instId 17465522551378543576. Retrying... 2023-10-21T04:51:03.208+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[2047508721584820382:true 12606056968936758026:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:51:03.213+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 6.657591ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:03.213+05:30 [Info] ShardRebalancer::destTokenToMergeOrReady Response reveived for merge of index instance: 15757988232943046271, realInst: 12606056968936758026 2023-10-21T04:51:03.214+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: #primary, defnId: 5277316367626246544, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:51:03.214+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: #primary, defnId: 5277316367626246544, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:51:03.251+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:03.251+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:03.253+05:30 [Info] switched currmeta from 507 -> 511 force true 2023-10-21T04:51:03.260+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:03.260+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:03.267+05:30 [Info] switched currmeta from 511 -> 511 force true 2023-10-21T04:51:03.291+05:30 [Info] clustMgrAgent::OnIndexRecover Notification received for Recover Index: instId 17465522551378543576,realInstId: 0, indexDefn DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:2}, partitions [0] 2023-10-21T04:51:03.292+05:30 [Info] Indexer::handleRecoverIndex InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:51:03.292+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_RECOVER_INDEX message from internalAdminRecvCh channel processing took 268.038µs 2023-10-21T04:51:03.292+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_RECOVER_INDEX message from adminRecvCh channel processing took 313.211µs 2023-10-21T04:51:03.292+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 17465522551378543576 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[0:[10838601320854145501 11711860383628641160]] 2023-10-21T04:51:03.293+05:30 [Info] clustMgrAgent::OnIndexRecover: Successfully initiated for index recovery: instId 17465522551378543576, indexDefn DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:51:03.293+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_CREATE_RECOVER_INDEX_REBAL elapsed 78.890646ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:03.293+05:30 [Info] plasmaSlice::NewPlasmaSlice Id: 0 IndexInstId: 17465522551378543576 PartitionId: 0 using shardIds: [10838601320854145501 11711860383628641160], rebalance running: true 2023-10-21T04:51:03.293+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:51:03.293+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: after recovery: resident ratio 0 requestQuota 1055626 memSz 64 2023-10-21T04:51:03.293+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: RP version: 21 numRPs: 2 rp [0] sn: 23 count: 6 metaSz: 2485 rp [1] sn: 24 count: 6 metaSz: 2485 2023-10-21T04:51:03.293+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: doInit: data UsedSpace 129109 recovery UsedSpace 170562 2023-10-21T04:51:03.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 2 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex 2023-10-21T04:51:03.303+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:51:03.304+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:51:03.304+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex 2023-10-21T04:51:03.304+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex started 2023-10-21T04:51:03.304+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17465522551378543576 PartitionId 0 Recovering from recovery point .. 2023-10-21T04:51:03.304+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:03.304+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:03.307+05:30 [Info] TestCode::TestActionAtTag: currTag: 5, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:03.308+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:03.308+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:03.310+05:30 [Info] switched currmeta from 511 -> 512 force true 2023-10-21T04:51:03.323+05:30 [Info] plasmaSlice::doRecovery SliceId 0 IndexInstId 17465522551378543576 PartitionId 0 Warmup took 29.721074ms 2023-10-21T04:51:03.323+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 17465522551378543576 partitionId 0 WriterThreads 4 2023-10-21T04:51:03.323+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 17465522551378543576 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index Index: 17465522551378543576 Partition: 0 2023-10-21T04:51:03.324+05:30 [Info] Indexer::handleRecoverIndex Sending message to indexer on recovery of inst: 17465522551378543576, shardIdMap: map[0:[10838601320854145501]] err: 2023-10-21T04:51:03.324+05:30 [Info] Indexer::handleInstRecoveryResponse Recovery response received for instId: 17465522551378543576, partnShardIdMap: map[0:[10838601320854145501]], err: 2023-10-21T04:51:03.324+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:03.325+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 17465522551378543576 2023-10-21T04:51:03.325+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:03.326+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:51:03.326+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:03.326+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_RECOVERED Stream: NIL_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:03.327+05:30 [Info] ClustMgr:handleInstAsyncRecoveryDone mType: CLUST_MGR_ASYNC_RECOVERY_DONE indexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 0 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_RECOVERED RState: RebalPending Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:03.327+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex IndexInst 17465522551378543576 Partitions [0] 2023-10-21T04:51:03.329+05:30 [Info] StorageMgr::openSnapshot IndexInst:17465522551378543576 Partition:0 Attempting to open snapshot (SnapshotInfo: count:6 committed:false) 2023-10-21T04:51:03.339+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 10.2873ms len(expediates) 0 len(incomings) 1 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:03.340+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_ASYNC_RECOVERY_DONE elapsed 169.377µs len(expediates) 0 len(incomings) 0 len(outgoings) 1 len(parallels) 0 error 2023-10-21T04:51:03.355+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:03.355+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:03.368+05:30 [Info] switched currmeta from 512 -> 513 force true 2023-10-21T04:51:03.426+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:03.427+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:03.436+05:30 [Info] switched currmeta from 513 -> 515 force true 2023-10-21T04:51:03.452+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:03.452+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:03.454+05:30 [Info] switched currmeta from 515 -> 515 force true 2023-10-21T04:51:03.475+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:03.475+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:03.479+05:30 [Info] switched currmeta from 515 -> 516 force true 2023-10-21T04:51:03.781+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:51:03.835+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 0 220432454230336 19 73680 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.837+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 1 14580309443106 31 73680 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:51:03.840+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 2 80577332946777 27 73680 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:03.840+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 3 135940091877708 21 73680 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.840+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 4 247041243079227 25 73680 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.842+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 6 194695637011621 26 73680 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.846+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 8 182410957121870 32 73680 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:51:03.846+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 5 172199206487624 16 73680 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:51:03.846+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 7 19010051182641 24 73680 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.849+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 11 68019130447759 21 73680 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.849+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 10 121484100363817 20 73680 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.852+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 9 42756027613197 22 73680 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.852+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 12 171853572917441 25 73680 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.855+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 15 1542343912406 21 73680 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.855+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 14 59183965998125 22 73680 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.855+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 13 107902021898438 23 73680 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:03.859+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 16 157626280229441 25 73680 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:51:03.860+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 17 73693878880740 20 73680 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.871+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 18 67844364342499 20 73680 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.874+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 20 217154782530068 23 73680 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:03.877+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 22 16864753682212 30 73680 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:51:03.877+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 21 194038385561911 28 73680 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:51:03.879+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 24 170842850732116 20 73680 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.879+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 19 170065152021297 25 73680 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.879+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 23 206194254469959 20 73680 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.883+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 25 233111027052204 28 73680 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:51:03.884+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 27 178705476290398 24 73680 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.884+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 26 58389099238647 24 73680 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.887+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 29 52334512457128 20 73680 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.887+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 28 252516046855767 27 73680 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:03.888+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 30 183084439776991 19 73680 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.891+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 32 86344521159606 25 73680 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.891+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 31 270670782796252 22 73680 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.895+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 34 264478627574156 22 73680 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.895+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 33 233119567174084 20 73680 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.898+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 36 276412965411368 22 73680 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.898+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 35 187961911866991 30 73680 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:51:03.903+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 38 30543404237888 20 73680 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.906+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 37 94748374195527 19 73680 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.906+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 39 66391235022367 25 73680 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:03.909+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 41 50379500651629 21 73680 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.909+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 40 168721574304277 24 73680 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.909+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 42 100004485169771 26 73680 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.914+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 43 148816420612283 27 73680 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:03.914+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 44 236414409908581 20 73680 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:03.918+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 46 199648020746400 23 73680 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:03.918+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 45 199348794242318 39 73680 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:51:03.921+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 48 131255717944569 26 73680 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.925+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 47 84841900064407 22 73680 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:03.932+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 52 268905923659105 24 73680 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.932+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 53 7777834518711 31 73680 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:51:03.932+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 51 28675906453951 21 73680 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.933+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 54 70030494603811 21 73680 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:03.933+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 50 27094034037488 19 73680 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.941+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 57 43207585340528 18 73680 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:51:03.941+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 58 47128363244154 26 73680 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:03.941+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 59 160015043890055 16 73680 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:51:03.941+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 49 229485491476250 19 73680 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:03.941+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 55 75978859756835 17 73680 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:51:03.941+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 56 235447613990794 27 73680 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:51:03.945+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 60 65202412270069 20 73680 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.947+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 63 14950536761368 28 73680 172.31.5.112:9000. HWT [28-28,28]. 2023-10-21T04:51:03.947+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 62 170481067504432 20 73680 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:03.947+05:30 [Info] TK StreamBegin MAINT_STREAM bucket_%2 61 154979908447073 24 73680 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:03.949+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2 InstanceIds [544693394183450628 16759912838651900043 14033078509407174050 12606056968936758026 2047508721584820382] 2023-10-21T04:51:03.950+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 Cid SessionId 73680 2023-10-21T04:51:03.950+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 73680 2023-10-21T04:51:03.950+05:30 [Info] Timekeeper::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:51:03.950+05:30 [Info] Timekeeper::resetWaitForRecovery Stream MAINT_STREAM KeyspaceId bucket_%2 2023-10-21T04:51:03.950+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:51:03.950+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId MAINT_STREAM keyspaceId bucket_%2 2023-10-21T04:51:03.950+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 73680 2023-10-21T04:51:03.950+05:30 [Info] Indexer::handleRecoveryDone StreamId MAINT_STREAM KeyspaceId bucket_%2 SessionId 73680 STREAM_ACTIVE 2023-10-21T04:51:03.950+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM 2023-10-21T04:51:03.950+05:30 [Info] timekeeper.repairMissingStreamBegin stream MAINT_STREAM done 2023-10-21T04:51:04.125+05:30 [Info] Timekeeper::repairStream Found Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 In State STREAM_INACTIVE. Skipping Repair. 2023-10-21T04:51:04.311+05:30 [Info] ShardRebalancer::waitForIndexState: Indexes: map[17465522551378543576:INDEX_STATE_RECOVERED] reached state: INDEX_STATE_RECOVERED 2023-10-21T04:51:04.311+05:30 [Info] ShardRebalancer::startShardRecovery Successfully posted recoverIndexRebalance requests for active insts defnIds: [5277316367626246544] belonging to collectionId: 9, ttid: ShardToken11:a9:c7:70:34:9f:8d:92. Initiating build 2023-10-21T04:51:04.321+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild Notification Received for Build Index [17465522551378543576] &{2} 2023-10-21T04:51:04.321+05:30 [Info] Indexer::handleBuildRecoveredIndexes [17465522551378543576] 2023-10-21T04:51:04.322+05:30 [Info] bulkUpdateRState: Index instance 17465522551378543576 rstate moved to PENDING 2023-10-21T04:51:04.322+05:30 [Info] Indexer::handleBuildRecoveredIndexes Added Index: [17465522551378543576] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:51:04.322+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:04.322+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:04.322+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:04.323+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:51:04.323+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:04.323+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:04.323+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 969 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:04.323+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:51:04.323+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:51:04.323+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:04.323+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 969 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:04.323+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:51:04.323+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:04.323+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:04.323+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_system:_query reset bucket state to NIL 2023-10-21T04:51:04.323+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:51:04.323+05:30 [Info] Indexer::startKeyspaceIdStream Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId 970 RestartTS bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} 2023-10-21T04:51:04.323+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:04.323+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:04.323+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_system:_query Added for Stream INIT_STREAM 2023-10-21T04:51:04.323+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:2 bucket_1:4] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:51:04.324+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 17465522551378543576 stream INIT_STREAM keyspaceId bucket_%2:_system:_query state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:51:04.324+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:51:04.324+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=19,1=31,2=27,3=21,4=25,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=25,17=20,18=20,19=25,20=23,21=28,22=30,23=20,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=39,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=27,57=18,58=26,59=16,60=20,61=24,62=20,63=28] IndexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 970 CollectionId: 9 CollectionAware: true EnableOSO: false RestartTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} NumVBuckets: 64 2023-10-21T04:51:04.324+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:51:04.324+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:51:04.326+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:04.327+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from internalAdminRecvCh channel processing took 5.277537ms 2023-10-21T04:51:04.328+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_RECOVERED_INDEXES message from adminRecvCh channel processing took 6.550516ms 2023-10-21T04:51:04.329+05:30 [Info] clustMgrAgent::OnRecoveredIndexBuild returns for Build Index [17465522551378543576] 2023-10-21T04:51:04.330+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_BUILD_RECOVERED_INDEXES_REBAL elapsed 18.58769ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:04.335+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_system:_query Check pending stream merge. 2023-10-21T04:51:04.341+05:30 [Info] ShardRebalancer::startShardRecovery Waiting for index state to become active for insts: map[17465522551378543576:true] 2023-10-21T04:51:04.352+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:04.352+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:04.357+05:30 [Info] switched currmeta from 516 -> 517 force true 2023-10-21T04:51:04.358+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 26.575564ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:04.385+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:04.391+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:04.392+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:04.411+05:30 [Info] switched currmeta from 517 -> 518 force true 2023-10-21T04:51:04.470+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:04.470+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:04.475+05:30 [Info] switched currmeta from 518 -> 519 force true 2023-10-21T04:51:04.511+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:04.511+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:04.516+05:30 [Info] switched currmeta from 519 -> 520 force true 2023-10-21T04:51:04.573+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:04.573+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:04.579+05:30 [Info] switched currmeta from 520 -> 521 force true 2023-10-21T04:51:04.579+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:04.579+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[3 4 5 6 7 8 1 2] 14190651729632750031:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:04.579+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[3 4 5 6 7 8 1 2] 14190651729632750031:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:04.579+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[3 4 5 6 7 8 1 2] 14190651729632750031:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:04.579+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[4397940545762329850:[3 4 5 6 7 8 1 2] 14190651729632750031:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:04.893+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[12606056968936758026:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:51:04.893+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2047508721584820382:[0] 12606056968936758026:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:51:04.893+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2047508721584820382:[0] 12606056968936758026:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:51:04.893+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2047508721584820382:[0] 12606056968936758026:[3 4 5 6 7 8 1 2]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:51:04.950+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2 StreamId MAINT_STREAM Vbuckets []. 2023-10-21T04:51:04.950+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream MAINT_STREAM and KeyspaceId bucket_%2 2023-10-21T04:51:05.327+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_system:_query Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:51:05.339+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:37774" doAuth() ... 2023-10-21T04:51:05.339+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:37774" auth successful 2023-10-21T04:51:05.339+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:37774" +1 2023-10-21T04:51:05.391+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:05.602+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:05.603+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:05.609+05:30 [Info] switched currmeta from 521 -> 523 force true 2023-10-21T04:51:05.629+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:05.629+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:05.635+05:30 [Info] switched currmeta from 523 -> 523 force true 2023-10-21T04:51:05.737+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:51:05.749+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:05.749+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:05.752+05:30 [Info] switched currmeta from 523 -> 523 force true 2023-10-21T04:51:05.828+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 1 14580309443106 31 970 172.31.5.112:9000. HWT [31-31,31]. 2023-10-21T04:51:05.831+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 3 135940091877708 21 970 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:05.831+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 2 80577332946777 27 970 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:05.833+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 5 172199206487624 16 970 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:51:05.836+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 7 19010051182641 24 970 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:05.836+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 6 194695637011621 26 970 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:05.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 10 121484100363817 20 970 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:05.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 0 220432454230336 19 970 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:05.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 9 42756027613197 22 970 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:05.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 8 182410957121870 32 970 172.31.5.112:9000. HWT [0-32,32]. 2023-10-21T04:51:05.842+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 4 247041243079227 25 970 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:05.845+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 11 68019130447759 21 970 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:05.846+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 12 171853572917441 25 970 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:05.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 14 59183965998125 22 970 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:05.849+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 13 107902021898438 23 970 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:05.851+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 15 1542343912406 21 970 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:05.855+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 18 67844364342499 20 970 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:05.855+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 17 73693878880740 20 970 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:05.857+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 19 170065152021297 25 970 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:05.859+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 21 194038385561911 28 970 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:51:05.861+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 22 16864753682212 30 970 172.31.5.112:9000. HWT [0-30,30]. 2023-10-21T04:51:05.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 16 157626280229441 25 970 172.31.5.112:9000. HWT [25-25,25]. 2023-10-21T04:51:05.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 23 206194254469959 20 970 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:05.864+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 20 217154782530068 23 970 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:05.865+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 24 170842850732116 20 970 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:05.868+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 25 233111027052204 28 970 172.31.5.112:9000. HWT [0-28,28]. 2023-10-21T04:51:05.868+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 27 178705476290398 24 970 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:05.868+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 26 58389099238647 24 970 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:05.874+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 29 52334512457128 20 970 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:05.875+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 28 252516046855767 27 970 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:05.876+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 30 183084439776991 19 970 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:05.877+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 31 270670782796252 22 970 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:05.881+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 33 233119567174084 20 970 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:05.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 32 86344521159606 25 970 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:05.883+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 34 264478627574156 22 970 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:05.886+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 35 187961911866991 30 970 172.31.5.112:9000. HWT [30-30,30]. 2023-10-21T04:51:05.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 36 276412965411368 22 970 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:05.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 37 94748374195527 19 970 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:05.890+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 38 30543404237888 20 970 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:05.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 40 168721574304277 24 970 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:05.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 39 66391235022367 25 970 172.31.5.112:9000. HWT [0-25,25]. 2023-10-21T04:51:05.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 41 50379500651629 21 970 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:05.898+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 42 100004485169771 26 970 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:05.900+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 43 148816420612283 27 970 172.31.5.112:9000. HWT [0-27,27]. 2023-10-21T04:51:05.902+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 45 199348794242318 39 970 172.31.5.112:9000. HWT [39-39,39]. 2023-10-21T04:51:05.902+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 44 236414409908581 20 970 172.31.5.112:9000. HWT [0-20,20]. 2023-10-21T04:51:05.906+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 46 199648020746400 23 970 172.31.5.112:9000. HWT [0-23,23]. 2023-10-21T04:51:05.908+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 47 84841900064407 22 970 172.31.5.112:9000. HWT [0-22,22]. 2023-10-21T04:51:05.909+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 48 131255717944569 26 970 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:05.912+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 49 229485491476250 19 970 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:05.914+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 50 27094034037488 19 970 172.31.5.112:9000. HWT [0-19,19]. 2023-10-21T04:51:05.920+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 53 7777834518711 31 970 172.31.5.112:9000. HWT [0-31,31]. 2023-10-21T04:51:05.926+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 51 28675906453951 21 970 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:05.926+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 52 268905923659105 24 970 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:05.928+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 55 75978859756835 17 970 172.31.5.112:9000. HWT [0-17,17]. 2023-10-21T04:51:05.928+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 54 70030494603811 21 970 172.31.5.112:9000. HWT [0-21,21]. 2023-10-21T04:51:05.935+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 58 47128363244154 26 970 172.31.5.112:9000. HWT [0-26,26]. 2023-10-21T04:51:05.935+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 56 235447613990794 27 970 172.31.5.112:9000. HWT [27-27,27]. 2023-10-21T04:51:05.935+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 57 43207585340528 18 970 172.31.5.112:9000. HWT [0-18,18]. 2023-10-21T04:51:05.935+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 59 160015043890055 16 970 172.31.5.112:9000. HWT [0-16,16]. 2023-10-21T04:51:05.938+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 60 65202412270069 20 970 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:05.939+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 61 154979908447073 24 970 172.31.5.112:9000. HWT [0-24,24]. 2023-10-21T04:51:05.941+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 62 170481067504432 20 970 172.31.5.112:9000. HWT [20-20,20]. 2023-10-21T04:51:05.944+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_system:_query 63 14950536761368 28 970 172.31.5.112:9000. HWT [28-28,28]. 2023-10-21T04:51:05.952+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_system:_query InstanceIds [17465522551378543576] 2023-10-21T04:51:05.952+05:30 [Info] Indexer::startKeyspaceIdStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query Cid 9 SessionId 970 2023-10-21T04:51:05.952+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 970 2023-10-21T04:51:05.952+05:30 [Info] Timekeeper::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:51:05.952+05:30 [Info] Timekeeper::resetWaitForRecovery Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:51:05.952+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 17465522551378543576 INDEX_STATE_CATCHUP 2023-10-21T04:51:05.952+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 17465522551378543576 true 2023-10-21T04:51:05.952+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 17465522551378543576 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query Session: 970 BuildTS: [0=19,1=31,2=27,3=21,4=25,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=25,17=20,18=20,19=25,20=23,21=28,22=30,23=20,24=20,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=30,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=39,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=27,57=18,58=26,59=16,60=20,61=24,62=20,63=28] 2023-10-21T04:51:05.952+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_system:_query, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 31 31 31} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 25 0 25} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 39 39 39} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 16 0 16} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} , fetchKVSeq false 2023-10-21T04:51:05.952+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:51:05.953+05:30 [Info] Timekeeper::handleRecoveryDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:51:05.953+05:30 [Info] Indexer::handleRecoveryDone StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 970 STREAM_ACTIVE 2023-10-21T04:51:05.953+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 970 2023-10-21T04:51:05.953+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_system:_query 970 2023-10-21T04:51:05.953+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:05.953+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:05.953+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:05.953+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:05.954+05:30 [Info] send progress stats to clients 2023-10-21T04:51:05.954+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:51:05.954+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_system:_query StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:51:05.954+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_%2:_system:_query:1]] 2023-10-21T04:51:05.954+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalPending Stream: INIT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 970 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:51:05.956+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:51:05.956+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:51:05.956+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [0=4,1=32,2=4,3=4,4=26,5=4,6=4,7=4,8=4,9=4,10=4,11=4,12=4,13=4,14=4,15=4,16=25,17=4,18=20,19=4,20=4,21=4,22=4,23=20,24=20,25=4,26=4,27=4,28=4,29=4,30=4,31=4,32=4,33=4,34=4,35=30,36=4,37=4,38=4,39=4,40=4,41=4,42=4,43=4,44=4,45=40,46=4,47=4,48=4,49=4,50=4,51=4,52=4,53=4,54=4,55=4,56=27,57=4,58=4,59=17,60=20,61=4,62=20,63=28] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:05.958+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_%2 Instances [indexInstance:([]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:51:05.962+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_%2 2023-10-21T04:51:05.962+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 970 2023-10-21T04:51:05.962+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 970 2023-10-21T04:51:05.962+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_system:_query set minMergeTs 2023-10-21T04:51:05.962+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:51:05.962+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_system:_query 17465522551378543576 false 2023-10-21T04:51:05.962+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_system:_query 17465522551378543576 2023-10-21T04:51:05.962+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_system:_query. 2023-10-21T04:51:05.962+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 970 2023-10-21T04:51:05.966+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.601394ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:06.018+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:06.018+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:06.022+05:30 [Info] switched currmeta from 523 -> 524 force true 2023-10-21T04:51:06.398+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:06.483+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:06.483+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:06.486+05:30 [Info] switched currmeta from 524 -> 525 force true 2023-10-21T04:51:06.633+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is enabled 2023-10-21T04:51:06.954+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_system:_query StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:51:06.954+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_system:_query 2023-10-21T04:51:06.955+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 17465522551378543576 Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query SessionId: 970, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType INMEM_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 32 31 32} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 26 0 26} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 25 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 20 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 20 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 30 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 40 39 40} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 27 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 17 0 17} { 60 3b4d1e9f31f5 2 20 20 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 20 20} { 63 d98f16ed018 2 28 28 28} 2023-10-21T04:51:06.955+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_system:_query 17465522551378543576 INDEX_STATE_ACTIVE 2023-10-21T04:51:06.955+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query State Changed to INACTIVE 2023-10-21T04:51:06.955+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:51:06.955+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:51:06.955+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:06.955+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId: 970 2023-10-21T04:51:06.955+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:06.955+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:06.955+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalPending Version: 1 ReplicaId: 1 2023-10-21T04:51:06.956+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 970 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:06.956+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 17465522551378543576 from stream INIT_STREAM keyspaceId bucket_%2:_system:_query 2023-10-21T04:51:06.956+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_system:_query 2023-10-21T04:51:06.956+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_system:_query Deleted from Stream INIT_STREAM 2023-10-21T04:51:06.956+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:06.956+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_system:_query BuildTS: [] IndexList: [] Async: false SessionId: 970 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:06.956+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:51:06.956+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query total received key-versions: map[0:0 1:1 2:0 3:0 4:1 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:1 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:1 60:0 61:0 62:0 63:0] 2023-10-21T04:51:06.956+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_system:_query latest sequence numbers: map[0:0 1:32 2:0 3:0 4:26 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:0 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:40 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:0 59:17 60:0 61:0 62:0 63:0] 2023-10-21T04:51:06.957+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:51:06.957+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:37774: use of closed network connection 2023-10-21T04:51:06.957+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:37774" exit: read tcp4 127.0.0.1:9121->127.0.0.1:37774: use of closed network connection 2023-10-21T04:51:06.957+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:37774" closed ! 2023-10-21T04:51:06.957+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:51:06.957+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:37774" already gone 2023-10-21T04:51:06.957+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:06.957+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:06.957+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:06.957+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:06.957+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:51:06.958+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalPending Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:06.958+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_system:_query Stream: INIT_STREAM SessionId 970 2023-10-21T04:51:06.958+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_system:_query] 2023-10-21T04:51:06.985+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_system:_query]. Err genServer.closed 2023-10-21T04:51:06.985+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:51:06.985+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_system:_query] Treating genServer.closed As Success 2023-10-21T04:51:06.985+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 970 2023-10-21T04:51:06.985+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query SessionId 970 2023-10-21T04:51:06.985+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_system:_query 2023-10-21T04:51:07.011+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 53.26296ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:07.068+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 12606056968936758026 RealInstId: 12606056968936758026 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9003 2023-10-21T04:51:07.068+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 2047508721584820382 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:51:07.068+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 17465522551378543576 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9003 2023-10-21T04:51:07.068+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_partitioned Progress: 100 InstId: 4397940545762329850 RealInstId: 4397940545762329850 Partitions: [1 2 3 4 5 6 7 8] Destination: 127.0.0.1:9004 2023-10-21T04:51:07.068+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_default:c1:idx_secondary Progress: 100 InstId: 14190651729632750031 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:51:07.068+05:30 [Info] ShardRebalancer::getBuildProgress Index: bucket_%2:_system:_query:#primary Progress: 100 InstId: 10158339249566909515 RealInstId: 0 Partitions: [0] Destination: 127.0.0.1:9004 2023-10-21T04:51:07.068+05:30 [Info] ShardRebalancer::computeProgress 0.7030000000000001 2023-10-21T04:51:07.070+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:07.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:07.074+05:30 [Info] switched currmeta from 525 -> 526 force true 2023-10-21T04:51:07.402+05:30 [Info] TestCode::TestActionAtTag: currTag: 6, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:07.402+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 17465522551378543576 Defn: DefnId: 5277316367626246544 Name: #primary Using: plasma Bucket: bucket_%2 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: true respCh: 0xc0059041e0 2023-10-21T04:51:07.403+05:30 [Info] handleUpdateIndexRState: Index instance 17465522551378543576 rstate moved to ACTIVE 2023-10-21T04:51:07.408+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 5.277963ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:07.408+05:30 [Info] ShardRebalancer::waitForIndexState All indexes: map[17465522551378543576:true] are active and caught up. Waiting for pending merge to finish 2023-10-21T04:51:07.408+05:30 [Info] ShardRebalancer::startShardRecovery Finished recovery of all indexes in ttid: ShardToken11:a9:c7:70:34:9f:8d:92, elapsed(sec): 8.59312379 2023-10-21T04:51:07.417+05:30 [Info] ShardRebalancer::startShardRecovery Done with recovery for ttid: ShardToken11:a9:c7:70:34:9f:8d:92 2023-10-21T04:51:07.417+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:51:07.435+05:30 [Info] TestCode::TestActionAtTag: currTag: 7, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:07.561+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:07.561+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:07.565+05:30 [Info] switched currmeta from 526 -> 528 force true 2023-10-21T04:51:07.570+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:07.570+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:07.571+05:30 [Info] switched currmeta from 528 -> 528 force true 2023-10-21T04:51:07.626+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:07.626+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:07.629+05:30 [Info] switched currmeta from 528 -> 529 force true 2023-10-21T04:51:07.673+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenReady BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:51:07.679+05:30 [Info] TestCode::TestActionAtTag: currTag: 7, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:07.684+05:30 [Info] TestCode::TestActionAtTag: currTag: 8, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:07.686+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenad:a9:48:d8:e4:72:ee:80 MasterId: SourceId: DestId: RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenDropOnSource BuildSource: S3 TransferMode: Move SourceTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:51:07.686+05:30 [Info] plasmaSlice::SetStopWriteUnitBilling SliceId 0 IndexInstId 2047508721584820382 PartitionId 0 Started Billing WUs 2023-10-21T04:51:08.014+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 24] call. taskList: &{Rev:[0 0 0 0 0 0 0 25] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/78118431caff95cf657f177e70958ebf Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]} {Rev:[0 0 0 0 0 0 0 2] ID:rebalance/78118431caff95cf657f177e70958ebf Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]}]} 2023-10-21T04:51:08.014+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 25] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:51:08.016+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 25] 2023-10-21T04:51:08.017+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 25] 2023-10-21T04:51:08.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:51:08.393+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[17465522551378543576:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:51:08.546+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:08.546+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:08.546+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:08.547+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:08.547+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:08.547+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10158339249566909515:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:51:08.988+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:08.988+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:09.012+05:30 [Info] switched currmeta from 529 -> 534 force true 2023-10-21T04:51:09.133+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:09.134+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:09.176+05:30 [Info] switched currmeta from 534 -> 534 force true 2023-10-21T04:51:09.380+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:09.380+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:09.421+05:30 [Info] switched currmeta from 534 -> 538 force true 2023-10-21T04:51:09.578+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:09.578+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:09.627+05:30 [Info] switched currmeta from 538 -> 542 force true 2023-10-21T04:51:09.846+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:09.846+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:09.902+05:30 [Info] switched currmeta from 542 -> 544 force true 2023-10-21T04:51:10.081+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:10.082+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:10.124+05:30 [Info] switched currmeta from 544 -> 545 force true 2023-10-21T04:51:10.222+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:10.222+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:10.232+05:30 [Info] switched currmeta from 545 -> 545 force true 2023-10-21T04:51:10.617+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:51:10.617+05:30 [Info] ShardTransferManager::handleRestoreShardDone Initiating RestoreShardDone for shards: [10838601320854145501 11711860383628641160] 2023-10-21T04:51:10.617+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data(shard10838601320854145501) : all deamons started 2023-10-21T04:51:10.617+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery(shard10838601320854145501) : all deamons started 2023-10-21T04:51:10.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:51:10.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:51:10.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : RestoreShardDone completed 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data(shard11711860383628641160) : all deamons started 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery(shard11711860383628641160) : all deamons started 2023-10-21T04:51:10.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : cleaning up unrecovered instances 2023-10-21T04:51:10.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : clean up unrecovered instances done 2023-10-21T04:51:10.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : RestoreShardDone completed 2023-10-21T04:51:10.618+05:30 [Info] ShardTransferManager::handleRestoreShardDone Finished RestoreShardDone for shards: [10838601320854145501 11711860383628641160], elapsed: 360.324µs 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery(shard10838601320854145501) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[10838601320854145501:true 11711860383628641160:true] 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data(shard10838601320854145501) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 4 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data(shard10838601320854145501) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 5 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 6 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery(shard10838601320854145501) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 7 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 8 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data(shard11711860383628641160) : LSSCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 1 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery(shard11711860383628641160) : RecoveryCleaner.lssCleanerDaemon: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data(shard11711860383628641160) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 2 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 12606056968936758026, partnId: 3 2023-10-21T04:51:10.618+05:30 [Info] LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery(shard11711860383628641160) : LSSCtx.runtimeStats: Starting daemon 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 6 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 7 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 8 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 1 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 2 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 3 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 4 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 8303154871847919944, partnId: 5 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 2047508721584820382, partnId: 0 2023-10-21T04:51:10.618+05:30 [Info] PlasmaSlice::ClearRebalRunning Clearing rebalance running for instId: 17465522551378543576, partnId: 0 2023-10-21T04:51:10.618+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.64µs 2023-10-21T04:51:10.618+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Initiating shard unlock for shards: [10838601320854145501 11711860383628641160] 2023-10-21T04:51:10.618+05:30 [Info] ShardTransferManager::handleUnlockShardCommands Done with shard unlock for shardIds: [10838601320854145501 11711860383628641160], errMap: map[10838601320854145501: 11711860383628641160:], elapsed: 11.168µs 2023-10-21T04:51:10.619+05:30 [Info] Indexer:updateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:51:10.619+05:30 [Info] ClustMgr:handleUpdateRebalancePhase &{REBALANCE_TRANSFER_IN_PROGRESS map[bucket_%2:REBALANCE_DONE]} 2023-10-21T04:51:10.619+05:30 [Info] IndexManager.UpdateRebalancePhase(): making request for updating rebalance phase, global: REBALANCE_TRANSFER_IN_PROGRESS, bucketTransfer: map[bucket_%2:REBALANCE_DONE] 2023-10-21T04:51:10.619+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REBALANCE_PHASE elapsed 27.587µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:10.635+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardToken11:a9:c7:70:34:9f:8d:92 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: d7ff2bfd429b21d9c24d822bc13e3acc (127.0.0.1:9001) DestId: 4325b30e7f9620a998388078247335a8 (127.0.0.1:9003) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardTokenba:4e:99:24:ac:d2:11:c2 Shards: [10838601320854145501 11711860383628641160] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 15757988232943046271 RealInstId: 12606056968936758026 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 9226953440900345886 RealInstId: 8303154871847919944 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 2047508721584820382 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 17465522551378543576 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:51:10.648+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:-1 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:0 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:51:10.648+05:30 [Info] ShardRebalancer::checkAllTokensDone Tranfser token: ShardTokenba:4e:99:24:ac:d2:11:c2 is in state: ShardTokenReady 2023-10-21T04:51:10.653+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardToken11:a9:c7:70:34:9f:8d:92 2023-10-21T04:51:10.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9790579228512172, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:51:10.739+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenCommit BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:51:10.952+05:30 [Info] ShardRebalancer::decodeTransferToken TransferToken ShardTokenba:4e:99:24:ac:d2:11:c2 MasterId: 4325b30e7f9620a998388078247335a8 SourceId: 9b27839d59b06374125d80a2b12a21b6 (127.0.0.1:9002) DestId: ba6304988a6335d65281381d6cfc5e32 (127.0.0.1:9004) RebalId: 78118431caff95cf657f177e70958ebf ShardTokenState: ShardTokenDeleted BuildSource: Dcp TransferMode: Move SiblingTokenId: ShardToken11:a9:c7:70:34:9f:8d:92 Shards: [18374611443167983589 6455624821495495854] Destination: /opt/build/goproj/src/github.com/couchbase/indexing/secondary/tests/serverlesstests/shard_rebalance_storage_dir InstId: 4130535070377368620 RealInstId: 4397940545762329850 Name: idx_partitioned Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 12046213868758366295 RealInstId: 12396348969790516872 Name: idx_partitioned_defer Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [1 2 3 4 5 6 7 8] Versions: [1 1 1 1 1 1 1 1] InstId: 14190651729632750031 RealInstId: 0 Name: idx_secondary Bucket: bucket_%2 ScopeId: _default/0 Collection/Id: c1/c Partitions: [0] Versions: [1] InstId: 10158339249566909515 RealInstId: 0 Name: #primary Bucket: bucket_%2 ScopeId: _system/8 Collection/Id: _query/9 Partitions: [0] Versions: [1] IsEmptyNodeBatch: false IsPendingReady: false 2023-10-21T04:51:10.956+05:30 [Info] ShardRebalancer::initiatePerNodeTransferAsMaster activeTransferCount: map[4325b30e7f9620a998388078247335a8:-1 9b27839d59b06374125d80a2b12a21b6:0 ba6304988a6335d65281381d6cfc5e32:-1 d7ff2bfd429b21d9c24d822bc13e3acc:0], batchSize: 1 2023-10-21T04:51:10.993+05:30 [Info] TestCode::TestActionAtTag: currTag: 9, actionAtTag: 9, actionOnNode: , currNode: 127.0.0.1:9003, action: 2 2023-10-21T04:51:11.059+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:51:11.059+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 25] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/78118431caff95cf657f177e70958ebf Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]} {Rev:[0 0 0 0 0 0 0 2] ID:rebalance/78118431caff95cf657f177e70958ebf Type:task-rebalance Status:task-running IsCancelable:true Progress:0.7030000000000001 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:78118431caff95cf657f177e70958ebf]}]} 2023-10-21T04:51:11.070+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:51:11.070+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 25] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:51:11.139+05:30 [Info] GenericServiceManager::CancelTask: called with id: rebalance/78118431caff95cf657f177e70958ebf, rev: [] 2023-10-21T04:51:11.139+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: rebalance/78118431caff95cf657f177e70958ebf, rev: [] 2023-10-21T04:51:11.139+05:30 [Info] ShardRebalancer::Cancel Exiting 2023-10-21T04:51:11.139+05:30 [Info] ShardRebalancer::processDropShards Cancel Received 2023-10-21T04:51:11.139+05:30 [Info] ShardRebalancer::updateProgress Cancel Received. progress reporter stapped 2023-10-21T04:51:11.139+05:30 [Info] ShardRebalancer::updateProgress:asyncCollectProgress cancel received. progress collector stopped 2023-10-21T04:51:11.153+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:51:11.189+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:11.189+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:11.256+05:30 [Info] switched currmeta from 545 -> 545 force true 2023-10-21T04:51:11.320+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:51:11.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.999556874190792, throttleDelayMs (new, old, change): (8, 3, 5) 2023-10-21T04:51:11.793+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:11.793+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:11.809+05:30 [Info] switched currmeta from 545 -> 545 force true 2023-10-21T04:51:12.127+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to false 2023-10-21T04:51:12.241+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:51:12.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.99971633878596, throttleDelayMs (new, old, change): (10, 8, 2) 2023-10-21T04:51:13.245+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:13.245+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:13.259+05:30 [Info] switched currmeta from 545 -> 545 force true 2023-10-21T04:51:13.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 1 2023-10-21T04:51:14.038+05:30 [Info] ShardRebalancer::processShardTransferTokenAsMaster No Tokens Found. Mark Done. 2023-10-21T04:51:14.038+05:30 [Info] ShardRebalancer::processShardTokens Received empty or deleted transfer token /indexing/rebalance/ShardTokenba:4e:99:24:ac:d2:11:c2 2023-10-21T04:51:14.038+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster true 2023-10-21T04:51:14.038+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:51:14.051+05:30 [Info] RebalanceServiceManager::cleanupGlobalRToken Delete Global Rebalance Token {4325b30e7f9620a998388078247335a8 78118431caff95cf657f177e70958ebf Rebalance 127.0.0.1 1 REBALANCE_TRANSFER_IN_PROGRESS SHARD} 2023-10-21T04:51:14.062+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:51:14.062+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:51:14.084+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:51:14.084+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:51:14.084+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.552µs 2023-10-21T04:51:14.084+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:51:14.084+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:51:14.084+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:51:14.086+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:51:14.086+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:51:14.086+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:51:14.086+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:51:14.086+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 26.882µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:14.088+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:51:14.088+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:51:14.088+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced false, isMaster true, forceUnbalanced true, err: 2023-10-21T04:51:14.088+05:30 [Warn] util::traceLockLog: Long hold of svcMgrMu.Lock() 2.949479476s in RebalanceServiceManager::cancelActualTask: 2023-10-21T04:51:14.088+05:30 [Info] GenericServiceManager::CancelTask: return from id: rebalance/78118431caff95cf657f177e70958ebf, rev: [] call. err: 2023-10-21T04:51:14.089+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 25] call. taskList: &{Rev:[0 0 0 0 0 0 0 26] Tasks:[]} 2023-10-21T04:51:14.089+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 26] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:true Messages:[]} 2023-10-21T04:51:14.089+05:30 [Info] DDLServiceMgr::rebalanceDone: Called 2023-10-21T04:51:14.094+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/78118431caff95cf657f177e70958ebf, rev: [] 2023-10-21T04:51:14.094+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/78118431caff95cf657f177e70958ebf, rev: [] 2023-10-21T04:51:14.094+05:30 [Info] PauseServiceManager::endTask: called with err for taskId prepare/78118431caff95cf657f177e70958ebf 2023-10-21T04:51:14.094+05:30 [Info] PauseServiceManager::endTask task with ID prepare/78118431caff95cf657f177e70958ebf already cleaned up 2023-10-21T04:51:14.094+05:30 [Info] PauseServiceManager::PauseResumeCancelTask: removed all tasks with ID prepare/78118431caff95cf657f177e70958ebf 2023-10-21T04:51:14.094+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/78118431caff95cf657f177e70958ebf, rev: [] call. err: 2023-10-21T04:51:14.095+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:51:14.095+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 26] Tasks:[]} 2023-10-21T04:51:14.100+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:51:14.100+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 26] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:51:14.104+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 26] 2023-10-21T04:51:14.105+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 26] 2023-10-21T04:51:14.277+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9106 2023-10-21T04:51:14.277+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:51:14.345+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:51:14.536+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:51:14.536+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9112 2023-10-21T04:51:14.536+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:51:14.584+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:51:14.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9997756404808068 2023-10-21T04:51:14.687+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:51:14.687+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9118 2023-10-21T04:51:14.687+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:51:14.746+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:37392 2023-10-21T04:51:14.746+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:51:14.916+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 164.196256ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:14.916+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:51:14.916+05:30 [Info] DDLServiceMgr::rebalanceDone:: connecting to node 127.0.0.1:9124 2023-10-21T04:51:14.916+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:51:15.022+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:51:15.105+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:51:15.105+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:51:15.105+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:51:15.105+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:51:15.105+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:51:15.109+05:30 [Info] MetadataProvider: Updating indexer version to 9 2023-10-21T04:51:15.109+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:51:15.109+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:51:15.109+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:51:15.109+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:51:15.109+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:51:15.109+05:30 [Info] DDLServiceMgr::rebalanceDone: Returned 2023-10-21T04:51:15.109+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:56868->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:15.110+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:57330->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:15.110+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:37392->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:15.110+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:33772->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:15.110+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:37392. Reason = EOF. Cleaning Pipe. 2023-10-21T04:51:15.110+05:30 [Info] messageListener.start(): message channel closed. Remove peer cf:f2:37:56:a4:69:c4:47 as follower. 2023-10-21T04:51:15.116+05:30 [Info] RebalanceServiceManager::handleRebalanceCleanupStatus Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:51:15.181+05:30 [Info] RebalanceServiceManager::handleListRebalanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33132 2023-10-21T04:51:15.417+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:51:15.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9331360304724419, throttleDelayMs (new, old, change): (8, 10, -2) 2023-10-21T04:51:16.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.5572955154748965, throttleDelayMs (new, old, change): (3, 8, -5) 2023-10-21T04:51:17.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.2543170358945738, throttleDelayMs (new, old, change): (0, 3, -3) 2023-10-21T04:51:28.910+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 14033078509407174050 &{0} 2023-10-21T04:51:28.912+05:30 [Info] Indexer::handleDropIndex - IndexInstId 14033078509407174050 2023-10-21T04:51:28.922+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:28.923+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:28.923+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 14033078509407174050 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:28.924+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [14033078509407174050] 2023-10-21T04:51:28.924+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [14033078509407174050] 2023-10-21T04:51:28.924+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:28.924+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [14033078509407174050] 2023-10-21T04:51:28.924+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:28.924+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 14033078509407174050 Defn: DefnId: 16364390102182676976 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:28.925+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 12.766174ms 2023-10-21T04:51:28.925+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 14033078509407174050, PartitionId 0, IndexDefnId 16364390102182676976. 2023-10-21T04:51:28.925+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:28.925+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:28.925+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:28.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex] 2023-10-21T04:51:28.925+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex 2023-10-21T04:51:28.925+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex] 2023-10-21T04:51:28.925+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex] 2023-10-21T04:51:28.925+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex 2023-10-21T04:51:28.925+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex stopped 2023-10-21T04:51:28.925+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:28.925+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:28.925+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:28.925+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:28.925+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 14033078509407174050 2023-10-21T04:51:28.927+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [14033078509407174050] 2023-10-21T04:51:28.932+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:51:28.932+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51546 2023-10-21T04:51:28.935+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 10.106978ms 2023-10-21T04:51:28.935+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 23.040453ms 2023-10-21T04:51:28.938+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:28.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex] 2023-10-21T04:51:28.938+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex closed 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Mainstore#14033078509407174050:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:28.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex] 2023-10-21T04:51:28.939+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex 2023-10-21T04:51:28.939+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex] 2023-10-21T04:51:28.939+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex] 2023-10-21T04:51:28.939+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex 2023-10-21T04:51:28.939+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex stopped 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:28.939+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:28.954+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:28.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex] 2023-10-21T04:51:28.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex closed 2023-10-21T04:51:28.956+05:30 [Info] bucket_1/idx_secondary/Backstore#14033078509407174050:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:28.956+05:30 [Info] Indexer::cleanupIndexData IndexInst 14033078509407174050 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:28.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex ... 2023-10-21T04:51:28.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex ... 2023-10-21T04:51:28.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:51:28.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/mainIndex sucessfully destroyed 2023-10-21T04:51:28.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex ... 2023-10-21T04:51:28.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex ... 2023-10-21T04:51:28.967+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:28.967+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:28.971+05:30 [Info] switched currmeta from 545 -> 546 force true 2023-10-21T04:51:28.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:51:28.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_14033078509407174050_0.index destroyed 2023-10-21T04:51:28.975+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 14033078509407174050, PartitionId 0, IndexDefnId 16364390102182676976. 2023-10-21T04:51:28.976+05:30 [Info] Indexer::cleanupIndexData IndexInst 14033078509407174050 Partition 0 Destroy Done 2023-10-21T04:51:28.977+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 73.898927ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:28.992+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:28.992+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:29.012+05:30 [Info] switched currmeta from 546 -> 548 force true 2023-10-21T04:51:29.030+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:29.030+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:29.038+05:30 [Info] switched currmeta from 548 -> 549 force true 2023-10-21T04:51:29.055+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:29.055+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:29.066+05:30 [Info] switched currmeta from 549 -> 549 force true 2023-10-21T04:51:29.154+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 2047508721584820382 &{0} 2023-10-21T04:51:29.154+05:30 [Info] Indexer::handleDropIndex - IndexInstId 2047508721584820382 2023-10-21T04:51:29.155+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:51:29.155+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:29.155+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2047508721584820382 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:51:29.155+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [2047508721584820382] 2023-10-21T04:51:29.155+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [2047508721584820382] 2023-10-21T04:51:29.155+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:29.156+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [2047508721584820382] 2023-10-21T04:51:29.156+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:29.156+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 2047508721584820382 Defn: DefnId: 7735644145307932837 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:29.156+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.863804ms 2023-10-21T04:51:29.156+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 2047508721584820382, PartitionId 0, IndexDefnId 7735644145307932837. 2023-10-21T04:51:29.156+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:29.156+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:29.157+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:29.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex] 2023-10-21T04:51:29.157+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex 2023-10-21T04:51:29.157+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex] 2023-10-21T04:51:29.157+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex] 2023-10-21T04:51:29.157+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex 2023-10-21T04:51:29.157+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex stopped 2023-10-21T04:51:29.157+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:29.157+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:29.157+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:29.157+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:29.157+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 2047508721584820382 2023-10-21T04:51:29.158+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [2047508721584820382] 2023-10-21T04:51:29.161+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:51:29.161+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73680 2023-10-21T04:51:29.161+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 4.879685ms 2023-10-21T04:51:29.161+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 6.81732ms 2023-10-21T04:51:29.163+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:29.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex] 2023-10-21T04:51:29.163+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex closed 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:51:29.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex] 2023-10-21T04:51:29.164+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex] 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex] 2023-10-21T04:51:29.164+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex 2023-10-21T04:51:29.164+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex stopped 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:51:29.164+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:51:29.169+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:51:29.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex] 2023-10-21T04:51:29.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex closed 2023-10-21T04:51:29.169+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:51:29.169+05:30 [Info] Indexer::cleanupIndexData IndexInst 2047508721584820382 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:51:29.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex ... 2023-10-21T04:51:29.170+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex ... 2023-10-21T04:51:29.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:51:29.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/mainIndex sucessfully destroyed 2023-10-21T04:51:29.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex ... 2023-10-21T04:51:29.179+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex ... 2023-10-21T04:51:29.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:51:29.182+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_2047508721584820382_0.index destroyed 2023-10-21T04:51:29.182+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 2047508721584820382, PartitionId 0, IndexDefnId 7735644145307932837. 2023-10-21T04:51:29.182+05:30 [Info] Indexer::cleanupIndexData IndexInst 2047508721584820382 Partition 0 Destroy Done 2023-10-21T04:51:29.190+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 42.164957ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:29.261+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:29.262+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:29.268+05:30 [Info] switched currmeta from 549 -> 553 force true 2023-10-21T04:51:29.360+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:29.360+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:29.364+05:30 [Info] switched currmeta from 553 -> 553 force true 2023-10-21T04:51:30.797+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:51:30.802+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:30.802+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:30.804+05:30 [Info] switched currmeta from 553 -> 553 force true 2023-10-21T04:51:31.729+05:30 [Info] updator: updating service map. server group=Group 2, indexerVersion=9 nodeAddr 127.0.0.1:9003 clusterVersion 9 excludeNode storageMode 2 2023-10-21T04:51:31.738+05:30 [Info] memstats {"Alloc":197878696, "TotalAlloc":1196365136, "Sys":270293304, "Lookups":0, "Mallocs":10755536,"Frees":9692611, "HeapAlloc":197878696, "HeapSys":246939648, "HeapIdle":43712512, "HeapInuse":203227136,"HeapReleased":1048576, "HeapObjects":1062925,"MSpanInuse":1413600, "MSpanSys": 1844160, "StackInuse": 4718592,"GCSys":14111480, "LastGC":1697844022927139496,"PauseTotalNs":9427448, "PauseNs":[4696788, 199702, 150894], "NumGC":16} 2023-10-21T04:51:32.787+05:30 [Info] connected with 4 indexers 2023-10-21T04:51:32.787+05:30 [Info] client stats current counts: current: 12, not current: 0 2023-10-21T04:51:32.864+05:30 [Info] num concurrent scans {0} 2023-10-21T04:51:32.864+05:30 [Info] average scan response {0 ms} 2023-10-21T04:51:33.356+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] active conns 0, free conns 1 2023-10-21T04:51:33.357+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] active conns 0, free conns 1 2023-10-21T04:51:33.620+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] active conns 0, free conns 1 2023-10-21T04:51:33.655+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] active conns 0, free conns 1 2023-10-21T04:51:33.782+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:51:34.311+05:30 [Info] janitor: running cleanup. 2023-10-21T04:51:34.311+05:30 [Info] janitor: Processing delete tokens for /indexing/ddl/commandToken/delete/{[8378417662949994532 17476654447546601131 16632040625897953619 12599498416578472779 8910459001587546151 10606939961365971246 12582561312368048736 16961575949607441659 5932387718257253322 4860549750398470299 13268712595133993155 16706481854123898870 249887442666643345 4532813151040661705 128675491261474353 16337157693634799600 17797267771846799851 4987105685507282285 8504686614261377811 3381524789655566900 4790214984019693894 12824340686309573428 14292583894719865678 15973521466438086342 7392796230534034390 14741949230702299196 8857269605176042330 5465835793930010953 18046371353728341033 573521267164877540 3674078063353481907 10761885902580101111 8669649359784409226 1252359852432998131 16162012140118277065 7045704526128519604 17917116660857103172 12140458060295645238 4535197604682045916 12195715103909237684 12338036531800130039 6145407649125550089 2608585389864151823 15968368593314043432 18026660730857541767 6985049432225870291 11619165464560280276 9665410041638025825 2048006213723315482 4841852412665776737 11276338709617092948 14549775068266281786 11365633301282701856 4138047918192025493 18323460706027642575 7735644145307932837 4286621850367430051 12027054384338714804 16483001637761719171 16367703496921140088 1301336828089581719 13856704970496601958 7147407900589480727 14889401051926000176 16364390102182676976 17656144409930763141 15945117906390072636 7525085113233084253 15431181238515168303 12833670934079048728]} 2023-10-21T04:51:36.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9999538991338035, throttleDelayMs (new, old, change): (5, 0, 5) 2023-10-21T04:51:37.160+05:30 [Info] Indexer::ReadMemstats Time Taken 1.154329ms 2023-10-21T04:51:37.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9998005676075811, throttleDelayMs (new, old, change): (10, 5, 5) 2023-10-21T04:51:38.665+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9956132454106933 2023-10-21T04:51:39.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.645384183387451, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:51:40.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.3047425566392393, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:51:44.104+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 26] call. err: operation_canceled 2023-10-21T04:51:44.105+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:51:44.105+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 26] Tasks:[]} 2023-10-21T04:51:44.107+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 26] 2023-10-21T04:51:44.107+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:51:44.107+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 26] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:51:44.108+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 26] 2023-10-21T04:51:49.649+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 12549405239467949221 2023-10-21T04:51:49.649+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 204.992µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:49.712+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51350 2023-10-21T04:51:49.720+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51350 2023-10-21T04:51:49.766+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51350 2023-10-21T04:51:49.838+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12549405239467949221, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:51:49.838+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12549405239467949221, replicaId: 1, alternateShardIds: map[] 2023-10-21T04:51:49.877+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 2657603273182005502, indexDefn DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:51:49.877+05:30 [Info] Indexer::handleCreateIndex InstId: 2657603273182005502 Defn: DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:51:49.878+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 2657603273182005502 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:51:49.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:51:49.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data/recovery) 2023-10-21T04:51:49.883+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:49.884+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:49.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:51:49.887+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data/recovery) 2023-10-21T04:51:49.888+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:51:49.888+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:51:49.899+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex 2023-10-21T04:51:49.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215 2023-10-21T04:51:49.916+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215/data 2023-10-21T04:51:49.917+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex 2023-10-21T04:51:49.917+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex 2023-10-21T04:51:49.917+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex started 2023-10-21T04:51:49.917+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:49.917+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:49.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex 2023-10-21T04:51:49.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009 2023-10-21T04:51:49.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009/data 2023-10-21T04:51:49.920+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex 2023-10-21T04:51:49.920+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex 2023-10-21T04:51:49.920+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex started 2023-10-21T04:51:49.921+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 2657603273182005502 partitionId 0 WriterThreads 4 2023-10-21T04:51:49.921+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 2657603273182005502 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index Index: 2657603273182005502 Partition: 0 2023-10-21T04:51:49.921+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 2657603273182005502 2023-10-21T04:51:49.921+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:49.921+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:51:49.921+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:49.921+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:49.921+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 44.004433ms 2023-10-21T04:51:49.921+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 44.048999ms 2023-10-21T04:51:49.921+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:51:49.922+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:51:49.922+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 2657603273182005502, partnShardIdMap: map[0:[17327901024118572215 5629943257627822009]], indexDefn DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:51:49.934+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:49.934+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:49.937+05:30 [Info] switched currmeta from 553 -> 557 force true 2023-10-21T04:51:49.942+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:49.942+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:49.944+05:30 [Info] switched currmeta from 557 -> 558 force true 2023-10-21T04:51:49.955+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [2657603273182005502] &{0} 2023-10-21T04:51:49.955+05:30 [Info] Indexer::handleBuildIndex [2657603273182005502] 2023-10-21T04:51:49.958+05:30 [Info] bulkUpdateRState: Index instance 2657603273182005502 rstate moved to ACTIVE 2023-10-21T04:51:49.958+05:30 [Info] Indexer::handleBuildIndex Added Index: [2657603273182005502] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:51:49.958+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:49.959+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:49.959+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:49.960+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_1:_default:c1 2023-10-21T04:51:49.960+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:49.960+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:49.960+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91311 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:49.960+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:51:49.960+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:51:49.960+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:49.960+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91311 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:49.960+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:51:49.960+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:49.960+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:49.960+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_1:_default:c1 reset bucket state to NIL 2023-10-21T04:51:49.960+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_1 NilBucketState 2023-10-21T04:51:49.960+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:49.960+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:49.960+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_1:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:51:49.960+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:4] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:51:49.960+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 2657603273182005502 stream INIT_STREAM keyspaceId bucket_1:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:51:49.960+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:51:49.960+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=18,1=22,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=23,12=23,13=24,14=19,15=25,16=24,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=25,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=18,46=21,47=28,48=20,49=23,50=20,51=27,52=18,53=30,54=22,55=25,56=22,57=27,58=25,59=21,60=24,61=24,62=17,63=24] IndexList: [ InstId: 2657603273182005502 Defn: DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 91312 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:51:49.961+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:51:49.961+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:51:49.962+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2657603273182005502 Defn: DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:49.962+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 7.022497ms 2023-10-21T04:51:49.962+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 7.06253ms 2023-10-21T04:51:49.962+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [2657603273182005502] 2023-10-21T04:51:49.971+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_1:_default:c1 Check pending stream merge. 2023-10-21T04:51:49.978+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:49.978+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:49.980+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 12549405239467949221 2023-10-21T04:51:49.980+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 142.148699ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:49.985+05:30 [Info] switched currmeta from 558 -> 561 force true 2023-10-21T04:51:49.994+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:51:49.996+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 15.672958ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:50.085+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:50.085+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:50.104+05:30 [Info] switched currmeta from 561 -> 562 force true 2023-10-21T04:51:50.324+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39050" doAuth() ... 2023-10-21T04:51:50.324+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39050" auth successful 2023-10-21T04:51:50.324+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:39050" +1 2023-10-21T04:51:50.413+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38292 2023-10-21T04:51:50.457+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 28.987181ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:50.498+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38306 2023-10-21T04:51:50.504+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 29.768µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:50.519+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38292. Reason = EOF. Cleaning Pipe. 2023-10-21T04:51:50.519+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5b:37:e5:9a:c8:86:48:e5 as follower. 2023-10-21T04:51:50.571+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 52.070787ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:50.684+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:51:50.684+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:51:50.698+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:51:50.732+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38330 2023-10-21T04:51:50.741+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38306. Reason = EOF. Cleaning Pipe. 2023-10-21T04:51:50.741+05:30 [Info] messageListener.start(): message channel closed. Remove peer 97:ee:52:5:eb:80:a6:74 as follower. 2023-10-21T04:51:50.762+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 21.5224ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:50.775+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:51:50.775+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:51:50.775+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:51:50.797+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:51:50.806+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38330. Reason = EOF. Cleaning Pipe. 2023-10-21T04:51:50.806+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6d:b9:6a:e2:21:bc:a7:3c as follower. 2023-10-21T04:51:50.838+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:51:50.838+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:51:50.838+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:51:50.859+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38344 2023-10-21T04:51:50.859+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:51:50.880+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 17.391654ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:50.880+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:51:50.880+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:51:50.880+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:51:50.885+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:51:50.893+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:50.894+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:50.896+05:30 [Info] switched currmeta from 562 -> 562 force true 2023-10-21T04:51:50.918+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:51:50.926+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 0 67576817345502 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.927+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 1 44231811555836 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.930+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 3 39196762516158 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.934+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 5 95215784105904 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.937+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 6 143637217187246 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.937+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 7 4065155603859 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.946+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:51:50.946+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:51:50.946+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:51:50.946+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:51:50.946+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:51:50.946+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:51:50.946+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:51:50.946+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:51:50.946+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:51:50.947+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:51:50.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 4 95570275214302 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 8 91960699361891 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.947+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 2 197901620499212 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.948+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:58280->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:50.948+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:57810->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:50.949+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:38344->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:50.949+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38344. Reason = EOF. Cleaning Pipe. 2023-10-21T04:51:50.949+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6a:b7:f5:86:d1:74:db:5b as follower. 2023-10-21T04:51:50.949+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:34722->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:51:50.956+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 9 147449900650979 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.957+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 10 27353924173936 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.957+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 11 77783296532664 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.957+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:51:50.957+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 13 198938078557080 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.958+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 15 129413110409107 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.959+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 14 122353375037127 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.959+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 12 233597919143467 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.959+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 17 25078414014136 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.959+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 16 33320884322178 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.968+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 21 7646518130509 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.968+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 18 132000914750455 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.968+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 19 79253753007122 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.968+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 20 101186302195296 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.969+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 23 230675090583633 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.969+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 22 215732063158788 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.977+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 25 232497361012987 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.978+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 24 113904165092305 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.979+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 26 201780742117921 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.984+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 29 241043521322196 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.984+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 27 131315346703752 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.987+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 28 61904892722472 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.991+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 30 248212138718363 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.991+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 31 201745596055144 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.994+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 33 258788849496338 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.994+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 32 19399479457331 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.999+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 37 63194637348236 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.999+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 36 280972182784873 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:50.999+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 35 173020619068939 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.003+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 40 186283152043223 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.003+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 39 119373194724283 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.003+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 38 13155294694475 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.003+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 34 190878005091193 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.009+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 42 209574230498887 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.009+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 41 277109307527403 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.011+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 43 270106388647522 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.012+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 44 255609313786736 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.014+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 45 49352389572227 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.016+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 47 62156602796507 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.023+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 48 80989250905985 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.032+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 51 204579891639872 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.032+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 50 193080054939721 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.032+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 46 83754344146634 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.032+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 49 279208201382660 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.032+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 53 212804514607186 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.032+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 52 150369066093368 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.043+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 57 218103594015078 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.043+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 54 102294135990395 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.043+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 55 65071891036109 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 62 184273998690001 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 60 223555064174984 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 56 152904045160406 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 59 90104317736129 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 61 257224089448052 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.049+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 58 35015089382353 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.055+05:30 [Info] TK StreamBegin INIT_STREAM bucket_1:_default:c1 63 202830985870043 0 91312 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:51:51.062+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_1:_default:c1 InstanceIds [2657603273182005502] 2023-10-21T04:51:51.063+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 Cid c SessionId 91312 2023-10-21T04:51:51.063+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:51:51.063+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:51:51.063+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs Not Snapshot Aligned. Continue both streams for keyspaceId bucket_1:_default:c1. INIT PendTsCount 0. MAINT PendTsCount 0. 2023-10-21T04:51:51.063+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: FlushTs bucket: bucket_1, scopeId: , collectionId: , vbuckets: 63 Crc64: 4466742297020474358 snapType NO_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 32 0 32} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 0 0 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 0 0 0} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 0 0 0 0} { 12 d474c0fda62b 2 0 0 0} { 13 b4eee1813398 0 0 0 0} { 14 6f479d74c6c7 2 21 0 21} { 15 75b35679a793 0 0 0 0} { 16 1e4e1f793f82 0 0 0 0} { 17 16cf05f576b8 0 0 0 0} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 0 0 0 0} { 20 5c0745993660 0 0 0 0} { 21 6f457ff374d 0 0 0 0} { 22 c43508fdc204 2 0 0 0} { 23 d1cc3ad53051 0 0 0 0} { 24 6798614c7bd1 0 0 0 0} { 25 d374829d48fb 0 0 0 0} { 26 b784bd74da21 0 0 0 0} { 27 776e3c980588 0 0 0 0} { 28 384d5b23fd28 0 0 0 0} { 29 db3a5176bcd4 0 0 0 0} { 30 e1bf645b749b 0 0 0 0} { 31 b77c8e967e68 0 0 0 0} { 32 11a4cb14f633 0 0 0 0} { 33 eb5df9723112 0 0 0 0} { 34 ad9a3fb05b79 0 0 0 0} { 35 9d5c8089d60b 0 0 0 0} { 36 ff8aef258b69 0 0 0 0} { 37 3979a5e92d8c 0 0 0 0} { 38 bf6f4aacc4b 0 0 0 0} { 39 6c91bcde03bb 0 0 0 0} { 40 a96c6d1aecd7 0 0 0 0} { 41 fc0789d6d0eb 0 0 0 0} { 42 be9b4d901e47 0 0 0 0} { 43 f5a90b58ce62 0 0 0 0} { 44 e879ae597b70 0 0 0 0} { 45 2ce2bfa63683 0 0 0 0} { 46 4c2c93ee66ca 0 0 0 0} { 47 3887f63ae9db 0 0 0 0} { 48 49a8c787e781 0 0 0 0} { 49 fdf039a96b04 0 0 0 0} { 50 af9af4168049 0 0 0 0} { 51 ba1077c4f640 0 0 0 0} { 52 88c286c50b38 0 0 0 0} { 53 c18b697ff052 0 0 0 0} { 54 5d0935a3687b 0 0 0 0} { 55 3b2ebaf337cd 0 0 0 0} { 56 8b10bf4be7d6 0 0 0 0} { 57 c65d333ed966 0 0 0 0} { 58 1fd895f6e3d1 0 0 0 0} { 59 51f30bd9f4c1 0 0 0 0} { 60 cb5278301988 0 0 0 0} { 61 e9f1a67aae74 0 0 0 0} { 62 a798a23bb6d1 0 0 0 0} HWT bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 0 snapType NO_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 32 0 32} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 0 0 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 22 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 0 0 0 0} { 12 d474c0fda62b 2 14 0 23} { 13 b4eee1813398 0 0 0 0} { 14 6f479d74c6c7 2 21 0 21} { 15 75b35679a793 0 0 0 0} { 16 1e4e1f793f82 0 0 0 0} { 17 16cf05f576b8 0 0 0 0} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 0 0 0 0} { 20 5c0745993660 0 0 0 0} { 21 6f457ff374d 0 0 0 0} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 0 0 0 0} { 24 6798614c7bd1 0 0 0 0} { 25 d374829d48fb 0 0 0 0} { 26 b784bd74da21 0 0 0 0} { 27 776e3c980588 0 0 0 0} { 28 384d5b23fd28 0 0 0 0} { 29 db3a5176bcd4 0 0 0 0} { 30 e1bf645b749b 0 0 0 0} { 31 b77c8e967e68 0 0 0 0} { 32 11a4cb14f633 0 0 0 0} { 33 eb5df9723112 0 0 0 0} { 34 ad9a3fb05b79 0 0 0 0} { 35 9d5c8089d60b 0 0 0 0} { 36 ff8aef258b69 0 0 0 0} { 37 3979a5e92d8c 0 0 0 0} { 38 bf6f4aacc4b 0 0 0 0} { 39 6c91bcde03bb 0 0 0 0} { 40 a96c6d1aecd7 0 0 0 0} { 41 fc0789d6d0eb 0 0 0 0} { 42 be9b4d901e47 0 0 0 0} { 43 f5a90b58ce62 0 0 0 0} { 44 e879ae597b70 0 0 0 0} { 45 2ce2bfa63683 0 0 0 0} { 46 4c2c93ee66ca 0 0 0 0} { 47 3887f63ae9db 0 0 0 0} { 48 49a8c787e781 0 0 0 0} { 49 fdf039a96b04 0 0 0 0} { 50 af9af4168049 0 0 0 0} { 51 ba1077c4f640 0 0 0 0} { 52 88c286c50b38 0 0 0 0} { 53 c18b697ff052 0 0 0 0} { 54 5d0935a3687b 0 0 0 0} { 55 3b2ebaf337cd 0 0 0 0} { 56 8b10bf4be7d6 0 0 0 0} { 57 c65d333ed966 0 0 0 0} { 58 1fd895f6e3d1 0 0 0 0} { 59 51f30bd9f4c1 0 0 0 0} { 60 cb5278301988 0 0 0 0} { 61 e9f1a67aae74 0 0 0 0} { 62 a798a23bb6d1 0 0 0 0} { 63 b87944dcbadb 0 0 0 0} 2023-10-21T04:51:51.063+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:51:51.063+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:51:51.063+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:51:51.063+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:51:51.066+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [0=18,1=22,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=23,12=23,13=24,14=19,15=25,16=24,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=25,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=18,46=21,47=28,48=20,49=23,50=20,51=27,52=18,53=30,54=22,55=25,56=22,57=27,58=25,59=21,60=24,61=24,62=17,63=24] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:51.075+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:51:51.264+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:51:51.382+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:51:51.512+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_1:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:51:51.517+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2657603273182005502 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:51:51.518+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2657603273182005502 PartitionId: 0 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.602µs SnapOpenDur 327.722µs 2023-10-21T04:51:51.518+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 2657603273182005502 INDEX_STATE_CATCHUP 2023-10-21T04:51:51.518+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 2657603273182005502 true 2023-10-21T04:51:51.518+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 2657603273182005502 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 Session: 91312 BuildTS: [0=18,1=22,2=23,3=19,4=23,5=25,6=23,7=28,8=21,9=28,10=22,11=23,12=23,13=24,14=19,15=25,16=24,17=18,18=21,19=21,20=32,21=24,22=18,23=23,24=21,25=18,26=20,27=21,28=19,29=24,30=17,31=25,32=23,33=20,34=21,35=21,36=24,37=27,38=25,39=26,40=18,41=24,42=23,43=25,44=23,45=18,46=21,47=28,48=20,49=23,50=20,51=27,52=18,53=30,54=22,55=25,56=22,57=27,58=25,59=21,60=24,61=24,62=17,63=24] 2023-10-21T04:51:51.518+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2657603273182005502, PartitionId 0 SnapshotId 19 Creating recovery point ... 2023-10-21T04:51:51.520+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 91312 2023-10-21T04:51:51.521+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_1:_default:c1 91312 2023-10-21T04:51:51.521+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:51.521+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:51.521+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:51.521+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2657603273182005502 Defn: DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:51.521+05:30 [Info] send progress stats to clients 2023-10-21T04:51:51.521+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:51:51.521+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:51:51.521+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:5] INIT_STREAM:map[bucket_1:_default:c1:1]] 2023-10-21T04:51:51.521+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [ InstId: 2657603273182005502 Defn: DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 91312 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:51:51.529+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:51:51.536+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.480737ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:51.574+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_1 2023-10-21T04:51:51.574+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91312 2023-10-21T04:51:51.574+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91312 2023-10-21T04:51:51.574+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_1:_default:c1 set minMergeTs 2023-10-21T04:51:51.574+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:51:51.574+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_1:_default:c1 2657603273182005502 false 2023-10-21T04:51:51.574+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_1:_default:c1 2657603273182005502 2023-10-21T04:51:51.574+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_1:_default:c1. 2023-10-21T04:51:51.574+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_1 91312 2023-10-21T04:51:51.581+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:51.581+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:51.585+05:30 [Info] switched currmeta from 562 -> 564 force true 2023-10-21T04:51:51.597+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:51.597+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:51.602+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2657603273182005502, PartitionId 0 SnapshotId 19 Created recovery point (took 83.104224ms) 2023-10-21T04:51:51.609+05:30 [Info] switched currmeta from 564 -> 564 force true 2023-10-21T04:51:52.064+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_1:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:51:52.064+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_1:_default:c1 2023-10-21T04:51:52.064+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 2657603273182005502 Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 SessionId: 91312, INIT_STREAM:LastFlushTs: bucket: bucket_1, scopeId: , collectionId: , vbuckets: 64 Crc64: 13481327001995404766 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 3d75f43163de 2 18 0 18} { 1 283a856a61fc 2 32 0 32} { 2 b3fd8fca530c 2 23 0 23} { 3 23a6351b2abe 2 19 0 19} { 4 56ebb04e23de 2 23 0 23} { 5 569926fd17b0 2 25 0 25} { 6 82a3254e95ae 2 23 0 23} { 7 3b27e27dd93 2 28 0 28} { 8 53a344d19a63 2 21 0 21} { 9 861adaf351e3 2 28 0 28} { 10 18e0d4ef0070 2 22 0 22} { 11 46be55c260b8 2 25 0 25} { 12 d474c0fda62b 2 23 0 23} { 13 b4eee1813398 2 24 0 24} { 14 6f479d74c6c7 2 21 0 21} { 15 75b35679a793 2 25 0 25} { 16 1e4e1f793f82 2 36 0 36} { 17 16cf05f576b8 2 18 0 18} { 18 780ddba23bf7 2 21 0 21} { 19 4814b3cb8c12 2 21 0 21} { 20 5c0745993660 2 32 0 32} { 21 6f457ff374d 2 24 0 24} { 22 c43508fdc204 2 18 0 18} { 23 d1cc3ad53051 2 23 0 23} { 24 6798614c7bd1 2 21 0 21} { 25 d374829d48fb 2 18 0 18} { 26 b784bd74da21 2 20 0 20} { 27 776e3c980588 2 21 0 21} { 28 384d5b23fd28 2 19 0 19} { 29 db3a5176bcd4 2 24 0 24} { 30 e1bf645b749b 2 17 0 17} { 31 b77c8e967e68 2 26 0 26} { 32 11a4cb14f633 2 23 0 23} { 33 eb5df9723112 2 21 0 21} { 34 ad9a3fb05b79 2 21 0 21} { 35 9d5c8089d60b 2 21 0 21} { 36 ff8aef258b69 2 24 0 24} { 37 3979a5e92d8c 2 27 0 27} { 38 bf6f4aacc4b 2 25 0 25} { 39 6c91bcde03bb 2 26 0 26} { 40 a96c6d1aecd7 2 18 0 18} { 41 fc0789d6d0eb 2 24 0 24} { 42 be9b4d901e47 2 23 0 23} { 43 f5a90b58ce62 2 25 0 25} { 44 e879ae597b70 2 23 0 23} { 45 2ce2bfa63683 2 28 0 28} { 46 4c2c93ee66ca 2 23 0 23} { 47 3887f63ae9db 2 28 0 28} { 48 49a8c787e781 2 20 0 20} { 49 fdf039a96b04 2 23 0 23} { 50 af9af4168049 2 22 0 22} { 51 ba1077c4f640 2 27 0 27} { 52 88c286c50b38 2 18 0 18} { 53 c18b697ff052 2 30 0 30} { 54 5d0935a3687b 2 22 0 22} { 55 3b2ebaf337cd 2 27 0 27} { 56 8b10bf4be7d6 2 26 0 26} { 57 c65d333ed966 2 27 0 27} { 58 1fd895f6e3d1 2 25 0 25} { 59 51f30bd9f4c1 2 21 0 21} { 60 cb5278301988 2 24 0 24} { 61 e9f1a67aae74 2 24 0 24} { 62 a798a23bb6d1 2 17 0 17} { 63 b87944dcbadb 2 24 0 24} 2023-10-21T04:51:52.064+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_1:_default:c1 2657603273182005502 INDEX_STATE_ACTIVE 2023-10-21T04:51:52.064+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 State Changed to INACTIVE 2023-10-21T04:51:52.064+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:51:52.064+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:51:52.064+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 5 CurrMaint 0 New 0 2023-10-21T04:51:52.064+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId: 91312 2023-10-21T04:51:52.064+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:52.064+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:51:52.064+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:51:52.065+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91312 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:52.065+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 2657603273182005502 from stream INIT_STREAM keyspaceId bucket_1:_default:c1 2023-10-21T04:51:52.065+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_1:_default:c1 2023-10-21T04:51:52.065+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:51:52.065+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:51:52.065+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_1:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 91312 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:51:52.065+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:51:52.065+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 total received key-versions: map[0:11 1:15 2:16 3:12 4:16 5:18 6:16 7:21 8:14 9:21 10:15 11:16 12:16 13:17 14:12 15:18 16:17 17:11 18:14 19:14 20:25 21:17 22:11 23:16 24:14 25:11 26:13 27:14 28:12 29:17 30:10 31:18 32:16 33:13 34:14 35:14 36:17 37:20 38:18 39:19 40:11 41:17 42:16 43:18 44:16 45:11 46:14 47:21 48:13 49:16 50:13 51:20 52:11 53:23 54:15 55:18 56:15 57:20 58:18 59:14 60:17 61:17 62:10 63:17] 2023-10-21T04:51:52.065+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_1:_default:c1 latest sequence numbers: map[0:18 1:22 2:23 3:19 4:23 5:25 6:23 7:28 8:21 9:28 10:22 11:23 12:23 13:24 14:19 15:25 16:24 17:18 18:21 19:21 20:32 21:24 22:18 23:23 24:21 25:18 26:20 27:21 28:19 29:24 30:17 31:25 32:23 33:20 34:21 35:21 36:24 37:27 38:25 39:26 40:18 41:24 42:23 43:25 44:23 45:18 46:21 47:28 48:20 49:23 50:20 51:27 52:18 53:30 54:22 55:25 56:22 57:27 58:25 59:21 60:24 61:24 62:17 63:24] 2023-10-21T04:51:52.065+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:51:52.065+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:39050: use of closed network connection 2023-10-21T04:51:52.065+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:39050" exit: read tcp4 127.0.0.1:9121->127.0.0.1:39050: use of closed network connection 2023-10-21T04:51:52.065+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39050" closed ! 2023-10-21T04:51:52.065+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:51:52.065+05:30 [Error] DATP[->dataport ":9121"] fatal remote "127.0.0.1:39050" already gone 2023-10-21T04:51:52.066+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:52.066+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:52.066+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:51:52.066+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:51:52.066+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_1 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:51:52.066+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 2657603273182005502 Defn: DefnId: 12549405239467949221 Name: idx_secondary Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 1 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:51:52.067+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_1:_default:c1 Stream: INIT_STREAM SessionId 91312 2023-10-21T04:51:52.067+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_default:c1] 2023-10-21T04:51:52.078+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 11.149152ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:51:52.082+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1:_default:c1]. Err genServer.closed 2023-10-21T04:51:52.082+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:51:52.082+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_1:_default:c1] Treating genServer.closed As Success 2023-10-21T04:51:52.082+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91312 2023-10-21T04:51:52.082+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 SessionId 91312 2023-10-21T04:51:52.082+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_1:_default:c1 2023-10-21T04:51:52.118+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:52.118+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:52.120+05:30 [Info] switched currmeta from 564 -> 565 force true 2023-10-21T04:51:52.166+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:52.166+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:52.170+05:30 [Info] switched currmeta from 565 -> 566 force true 2023-10-21T04:51:52.894+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2657603273182005502:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:51:52.895+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2657603273182005502:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:51:52.895+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2657603273182005502:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:51:52.895+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2657603273182005502:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:51:52.895+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[2657603273182005502:[0]]. New rollbacktime: 1697843900235320103, Old rollbacktime: 0 2023-10-21T04:51:53.041+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10858167748885264727:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:51:53.041+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10858167748885264727:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:51:53.041+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10858167748885264727:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:51:53.042+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[10858167748885264727:[0]]. New rollbacktime: 1697843588669388475, Old rollbacktime: 0 2023-10-21T04:51:53.573+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 2657603273182005502 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:51:53.573+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 2657603273182005502 PartitionId: 0 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.547µs SnapOpenDur 89.537µs 2023-10-21T04:51:53.573+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 4 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.573+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 4 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 4.83µs SnapOpenDur 59.291µs 2023-10-21T04:51:53.573+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 5 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.573+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 5 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.255µs SnapOpenDur 43.547µs 2023-10-21T04:51:53.573+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 6 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.573+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 6 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.362µs SnapOpenDur 39.071µs 2023-10-21T04:51:53.573+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 7 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.573+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 7 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.352µs SnapOpenDur 36.592µs 2023-10-21T04:51:53.573+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 8 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.573+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 8 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.208µs SnapOpenDur 37.544µs 2023-10-21T04:51:53.573+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 1 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.574+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 1 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.397µs SnapOpenDur 34.71µs 2023-10-21T04:51:53.574+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 2 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.574+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 2 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.43µs SnapOpenDur 33.249µs 2023-10-21T04:51:53.574+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 16759912838651900043 PartitionId 3 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:51:53.574+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 16759912838651900043 PartitionId: 3 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:0 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.532µs SnapOpenDur 40.724µs 2023-10-21T04:51:53.574+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 544693394183450628 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:51:53.574+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 544693394183450628 PartitionId: 0 SliceId: 0 Crc64: 13481327001995404766 (SnapshotInfo: count:6 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.139µs SnapOpenDur 89.412µs 2023-10-21T04:51:53.574+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2657603273182005502, PartitionId 0 SnapshotId 20 Creating recovery point ... 2023-10-21T04:51:53.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.575+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 544693394183450628, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:51:53.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.576+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 SnapshotId 3 Creating recovery point ... 2023-10-21T04:51:53.692+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 SnapshotId 3 Created recovery point (took 117.092827ms) 2023-10-21T04:51:53.693+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.694+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.726+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 544693394183450628, PartitionId 0 SnapshotId 4 Created recovery point (took 150.262703ms) 2023-10-21T04:51:53.727+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 544693394183450628, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:51:53.802+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 2657603273182005502, PartitionId 0 SnapshotId 20 Created recovery point (took 227.819032ms) 2023-10-21T04:51:53.818+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 SnapshotId 3 Created recovery point (took 243.535697ms) 2023-10-21T04:51:53.819+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.820+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.836+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 SnapshotId 3 Created recovery point (took 261.090466ms) 2023-10-21T04:51:53.838+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.839+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.872+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 SnapshotId 3 Created recovery point (took 296.616893ms) 2023-10-21T04:51:53.874+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.874+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.905+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 SnapshotId 3 Created recovery point (took 330.069503ms) 2023-10-21T04:51:53.906+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.907+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.932+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 SnapshotId 3 Created recovery point (took 356.141324ms) 2023-10-21T04:51:53.933+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.934+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.954+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 SnapshotId 3 Created recovery point (took 377.957588ms) 2023-10-21T04:51:53.958+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.958+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.976+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 SnapshotId 3 Created recovery point (took 400.143737ms) 2023-10-21T04:51:53.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:53.978+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:51:55.879+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:51:55.884+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:51:55.884+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:51:55.886+05:30 [Info] switched currmeta from 566 -> 566 force true 2023-10-21T04:52:00.547+05:30 [Info] LifecycleMgr.handlePrepareCreateIndex() : Accept 12924535004952781813 2023-10-21T04:52:00.547+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_PREPARE_CREATE_INDEX elapsed 175.841µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:00.619+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51350 2023-10-21T04:52:00.626+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51350 2023-10-21T04:52:00.684+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:51350 2023-10-21T04:52:00.754+05:30 [Info] testcode::IgnoreAlternateShardIds (Before), defn.Name: idx_secondary, defnId: 12924535004952781813, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:52:00.754+05:30 [Info] testcode::IgnoreAlternateShardIds (After), defn.Name: idx_secondary, defnId: 12924535004952781813, replicaId: 0, alternateShardIds: map[] 2023-10-21T04:52:00.774+05:30 [Info] clustMgrAgent::OnIndexCreate: Notification received for Create Index: instId 1786384967016908289, indexDefn DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] , reqCtx &{ReqSource:0}, partitions [0] 2023-10-21T04:52:00.786+05:30 [Info] Indexer::handleCreateIndex InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} 2023-10-21T04:52:00.786+05:30 [Info] Indexer::initPartnInstance Initialized Partition: Index: 1786384967016908289 Partition: PartitionId: 0 Endpoints: [:9123] , shardIds: map[] 2023-10-21T04:52:00.799+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:00.800+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data/recovery) 2023-10-21T04:52:00.800+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:52:00.800+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:52:00.810+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:00.816+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Map plasma instance (/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex) to LSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data) and RecoveryLSS (/opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data/recovery) 2023-10-21T04:52:00.817+05:30 [Info] initInMemCompression: Resolved compression algo, using algo [zstd1] 2023-10-21T04:52:00.817+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: doInit: data UsedSpace 0 recovery UsedSpace 0 2023-10-21T04:52:00.821+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Assign plasmaId 34 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex 2023-10-21T04:52:00.822+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160 2023-10-21T04:52:00.823+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160/data 2023-10-21T04:52:00.826+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex 2023-10-21T04:52:00.827+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex 2023-10-21T04:52:00.827+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex started 2023-10-21T04:52:00.828+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:52:00.826+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Assign plasmaId 35 to instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex 2023-10-21T04:52:00.828+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex to Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501 2023-10-21T04:52:00.830+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Add instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex to LSS /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501/data 2023-10-21T04:52:00.830+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.registerInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex 2023-10-21T04:52:00.830+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.addToSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex 2023-10-21T04:52:00.831+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex started 2023-10-21T04:52:00.831+05:30 [Info] plasmaSlice:NewplasmaSlice Created New Slice Id 0 IndexInstId 1786384967016908289 partitionId 0 WriterThreads 4 2023-10-21T04:52:00.831+05:30 [Info] Indexer::initPartnInstance Initialized Slice: Index: 1786384967016908289 Slice: SliceId: 0 File: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index Index: 1786384967016908289 Partition: 0 2023-10-21T04:52:00.830+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:52:00.832+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.mvccPurgerDaemon: Starting MVCC Purger 2023-10-21T04:52:00.832+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.monitor: Starting monitor 2023-10-21T04:52:00.834+05:30 [Info] StorageMgr::updateIndexSnapMapForIndex, New IndexSnapshotContainer is being created for indexInst: 1786384967016908289 2023-10-21T04:52:00.838+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:00.839+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, updated paritionMap: map[0:PartitionId: 0 Endpoints: [:9123] ] 2023-10-21T04:52:00.843+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:00.847+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CREATED Stream: NIL_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:00.848+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from adminRecvCh channel processing took 61.906312ms 2023-10-21T04:52:00.847+05:30 [Info] clustMgrAgent::OnIndexCreate: Success for Create Index: instId 1786384967016908289, partnShardIdMap: map[0:[10838601320854145501 11711860383628641160]], indexDefn DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] 2023-10-21T04:52:00.847+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_CREATE_INDEX_DDL message from internalAdminRecvCh channel processing took 61.451192ms 2023-10-21T04:52:00.870+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:00.870+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:00.878+05:30 [Info] switched currmeta from 566 -> 571 force true 2023-10-21T04:52:00.936+05:30 [Info] clustMgrAgent::OnIndexBuild Notification Received for Build Index [1786384967016908289] &{0} 2023-10-21T04:52:00.940+05:30 [Info] Indexer::handleBuildIndex [1786384967016908289] 2023-10-21T04:52:00.943+05:30 [Info] bulkUpdateRState: Index instance 1786384967016908289 rstate moved to ACTIVE 2023-10-21T04:52:00.943+05:30 [Info] Indexer::handleBuildIndex Added Index: [1786384967016908289] to Stream: INIT_STREAM State: INDEX_STATE_INITIAL 2023-10-21T04:52:00.943+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:00.943+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:00.943+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_INITIAL Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:00.943+05:30 [Info] Indexer::prepareStreamKeyspaceIdForFreshStart INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:52:00.944+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:00.944+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:00.944+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43325 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:00.944+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:52:00.944+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:52:00.944+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:52:00.944+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43325 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:00.944+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Stream Already Closed INIT_STREAM 2023-10-21T04:52:00.944+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:00.944+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:00.944+05:30 [Info] Indexer::resetBucketPauseState INIT_STREAM bucket_%2:_default:c1 reset bucket state to NIL 2023-10-21T04:52:00.944+05:30 [Info] Timekeeper::handleUpdateBucketPauseState bucket_%2 NilBucketState 2023-10-21T04:52:00.945+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:00.945+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:00.946+05:30 [Info] StreamState::initKeyspaceIdInStream New KeyspaceId bucket_%2:_default:c1 Added for Stream INIT_STREAM 2023-10-21T04:52:00.946+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:3 bucket_1:5] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:52:00.950+05:30 [Info] Timekeeper::addIndextoStream add BuildInfo index 1786384967016908289 stream INIT_STREAM keyspaceId bucket_%2:_default:c1 state INDEX_STATE_INITIAL waitForRecovery false 2023-10-21T04:52:00.950+05:30 [Info] Timekeeper::startTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:52:00.950+05:30 [Info] MutationMgr::handleOpenStream Message: MsgStreamUpdate Type: OPEN_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=19,1=22,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=15,17=20,18=18,19=25,20=23,21=28,22=30,23=18,24=19,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=28,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=30,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=25,57=18,58=26,59=16,60=18,61=24,62=18,63=26] IndexList: [ InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: true SessionId: 43326 CollectionId: c CollectionAware: true EnableOSO: true RestartTs: NumVBuckets: 64 2023-10-21T04:52:00.951+05:30 [Info] DATP[->dataport ":9121"] started ... 2023-10-21T04:52:00.952+05:30 [Info] MutationStreamReader: Setting Stream Workers INIT_STREAM 32 2023-10-21T04:52:00.953+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_INITIAL RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:52:00.953+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from internalAdminRecvCh channel processing took 13.548414ms 2023-10-21T04:52:00.953+05:30 [Info] clustMgrAgent::OnIndexBuild returns for Build Index [1786384967016908289] 2023-10-21T04:52:00.954+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_BUILD_INDEX_DDL message from adminRecvCh channel processing took 14.174234ms 2023-10-21T04:52:00.959+05:30 [Info] KVSender::sendMutationTopicRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9121" > ] 2023-10-21T04:52:00.969+05:30 [Info] LifecycleMgr.handleCommitCreateIndex() : Create token posted for 12924535004952781813 2023-10-21T04:52:00.970+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_COMMIT_CREATE_INDEX elapsed 215.506683ms len(expediates) 1 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:00.971+05:30 [Info] Timekeeper::generateNewStabilityTS INIT_STREAM bucket_%2:_default:c1 Check pending stream merge. 2023-10-21T04:52:00.984+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 14.191607ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:01.019+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39404" doAuth() ... 2023-10-21T04:52:01.019+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39404" auth successful 2023-10-21T04:52:01.019+05:30 [Info] DATP[->dataport ":9121"] new connection "127.0.0.1:39404" +1 2023-10-21T04:52:01.034+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:01.034+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:01.064+05:30 [Info] switched currmeta from 571 -> 575 force true 2023-10-21T04:52:01.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:01.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:01.074+05:30 [Info] switched currmeta from 575 -> 575 force true 2023-10-21T04:52:01.262+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38656 2023-10-21T04:52:01.284+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 17.792503ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:01.424+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_REPLICA_COUNT elapsed 39.411µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:01.457+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38656. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:01.457+05:30 [Info] messageListener.start(): message channel closed. Remove peer 4a:87:ce:0:26:79:e5:dd as follower. 2023-10-21T04:52:01.509+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 0 220432454230336 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.511+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 2 80577332946777 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.514+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 4 247041243079227 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.518+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 6 194695637011621 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.523+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 1 14580309443106 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.523+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 3 135940091877708 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.523+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 5 172199206487624 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.530+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 8 182410957121870 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.531+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 7 19010051182641 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.534+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 9 42756027613197 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.534+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 10 121484100363817 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.560+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 11 68019130447759 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.564+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 12 171853572917441 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.572+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 14 59183965998125 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.572+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 13 107902021898438 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.579+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 16 157626280229441 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.579+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 15 1542343912406 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 18 67844364342499 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 20 217154782530068 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.585+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 17 73693878880740 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.588+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 22 16864753682212 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 24 170842850732116 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 21 194038385561911 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.599+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 19 170065152021297 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.607+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 25 233111027052204 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.607+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 26 58389099238647 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.610+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 23 206194254469959 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.642+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 29 52334512457128 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.643+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 30 183084439776991 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.643+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 27 178705476290398 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.643+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 31 270670782796252 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.643+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 28 252516046855767 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.643+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 32 86344521159606 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.646+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 34 264478627574156 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.646+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 36 276412965411368 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.646+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 35 187961911866991 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.646+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 33 233119567174084 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.646+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 38 30543404237888 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.649+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 40 168721574304277 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.654+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 42 100004485169771 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.654+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 37 94748374195527 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.658+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 44 236414409908581 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.667+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 46 199648020746400 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.681+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 39 66391235022367 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.681+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 50 27094034037488 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.681+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 41 50379500651629 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.681+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 48 131255717944569 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.682+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 52 268905923659105 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.684+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 54 70030494603811 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.690+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 56 235447613990794 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.691+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9124 2023-10-21T04:52:01.691+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9124 2023-10-21T04:52:01.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 45 199348794242318 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.696+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 47 84841900064407 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.697+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 58 47128363244154 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.697+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 43 148816420612283 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.698+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 60 65202412270069 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.700+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9124 2023-10-21T04:52:01.702+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 62 170481067504432 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.706+05:30 [Info] KVSender::sendMutationTopicRequest Success Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 bucket_%2:_default:c1 InstanceIds [1786384967016908289] 2023-10-21T04:52:01.706+05:30 [Info] Indexer::sendStreamUpdateForBuildIndex Stream Request Success For Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 Cid c SessionId 43326 2023-10-21T04:52:01.706+05:30 [Info] Indexer::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:52:01.706+05:30 [Info] Timekeeper::handleStreamRequestDone StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:52:01.706+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Final return. Index may not belong to flushed keyspace. INIT_STREAM, bucket_%2:_default:c1, initFlushTs bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 55 Crc64: 6791607392133076976 snapType INMEM_SNAP_OSO OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 0 0 0 0} { 2 4948df4ea759 0 0 0 0} { 3 7ba304e3194c 0 0 0 0} { 4 e0aec589e23b 0 0 0 0} { 5 9c9d40888648 0 0 0 0} { 6 b1131c488ca5 0 0 0 0} { 7 114a1f591031 0 0 0 0} { 8 a5e6dc4d694e 0 0 0 0} { 9 26e2e9d6800d 0 0 0 0} { 10 6e7d38a5e629 0 0 0 0} { 11 3ddcf01bb78f 0 0 0 0} { 12 9c4cc72ac8c1 0 0 0 0} { 13 6222e5af90c6 0 0 0 0} { 14 35d3d7450c2d 0 0 0 0} { 15 1671adc6bd6 0 0 0 0} { 16 8f5c3a70ba41 0 0 0 0} { 17 4306317909e4 0 0 0 0} { 18 3db43f3c94e3 0 0 0 0} { 19 9aac60f99331 0 0 0 0} { 20 c58049ac8a14 0 0 0 0} { 21 b07a150b5137 0 0 0 0} { 22 f56a1a70724 0 0 0 0} { 23 bb88574adb47 0 0 0 0} { 24 9b61736e2854 0 0 0 0} { 25 d40363f5eeac 0 0 0 0} { 26 351ac584d4f7 0 0 0 0} { 27 a2881c6f075e 0 0 0 0} { 28 e5a97945de57 0 0 0 0} { 29 2f99140645a8 0 0 0 0} { 30 a683aaff2edf 0 0 0 0} { 31 f62c73dc11dc 0 0 0 0} { 32 4e87a68313b6 0 0 0 0} { 33 d40560fdd9c4 0 0 0 0} { 34 f08abaa5b18c 0 0 0 0} { 35 aaf34afcf66f 0 0 0 0} { 36 fb65689d5a28 0 0 0 0} { 37 562c532fdd47 0 0 0 0} { 38 1bc770c28440 0 0 0 0} { 39 3c61e9face1f 0 0 0 0} { 40 99738d809a15 0 0 0 0} { 41 2dd1e43d846d 0 0 0 0} { 42 5af41bd0826b 0 0 0 0} { 43 875905e328bb 0 0 0 0} { 44 d70484ec2165 0 0 0 0} { 45 b54e8211350e 0 0 0 0} { 46 b5942d5b50a0 0 0 0 0} { 47 4d29cb50be97 0 0 0 0} { 48 77605a715cf9 0 0 0 0} { 50 18a452463af0 0 0 0 0} { 52 f4918a0ec561 0 0 0 0} { 54 3fb13ec11a23 0 0 0 0} { 56 d6236b65278a 0 0 0 0} { 58 2adced599a7a 0 0 0 0} { 60 3b4d1e9f31f5 0 0 0 0} , fetchKVSeq false 2023-10-21T04:52:01.706+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairStream to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:52:01.706+05:30 [Info] Timekeeper::handleStreamRequestDone Call RepairMissingStreamBegin to check for vbucket for repair. StreamId INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:52:01.706+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM 2023-10-21T04:52:01.713+05:30 [Info] Timekeeper::handleUpdateBuildTs Message: MsgStreamUpdate Type: INDEXER_UPDATE_BUILD_TS Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [0=19,1=22,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=15,17=20,18=18,19=25,20=23,21=28,22=30,23=18,24=19,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=28,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=30,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=25,57=18,58=26,59=16,60=18,61=24,62=18,63=26] IndexList: [] Async: false SessionId: 0 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:01.722+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 55 75978859756835 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.722+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 57 43207585340528 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.722+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 53 7777834518711 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.722+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 59 160015043890055 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.723+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 51 28675906453951 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.723+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 49 229485491476250 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.731+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 61 154979908447073 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.731+05:30 [Info] TK StreamBegin INIT_STREAM bucket_%2:_default:c1 63 14950536761368 0 43326 172.31.5.112:9000. HWT [0-0,0]. 2023-10-21T04:52:01.742+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9124. 2023-10-21T04:52:01.743+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9106 2023-10-21T04:52:01.744+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9106 2023-10-21T04:52:01.753+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9106 2023-10-21T04:52:01.771+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38690 2023-10-21T04:52:01.792+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:52:01.815+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 15.773575ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:01.823+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9106. 2023-10-21T04:52:01.824+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9112 2023-10-21T04:52:01.825+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9112 2023-10-21T04:52:01.834+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9112 2023-10-21T04:52:01.887+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38702 2023-10-21T04:52:01.898+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9112. 2023-10-21T04:52:01.898+05:30 [Info] DDLServiceMgr:handleCreateCommand: connecting to node 127.0.0.1:9118 2023-10-21T04:52:01.898+05:30 [Info] MetadataProvider.WatchMetadata(): indexer 127.0.0.1:9118 2023-10-21T04:52:01.913+05:30 [Info] IndexManager:ServerAuth auth successful for connection 127.0.0.1:9118:127.0.0.1:38704 2023-10-21T04:52:01.914+05:30 [Info] watcher:ClientAuth successful for 127.0.0.1:9118 2023-10-21T04:52:01.920+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 28.289398ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:01.931+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38690. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:01.931+05:30 [Info] messageListener.start(): message channel closed. Remove peer 5c:fb:1c:cf:46:f4:85:b as follower. 2023-10-21T04:52:01.945+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_SERVICE_MAP elapsed 25.648452ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:01.945+05:30 [Info] WatchMetadata(): successfully reach indexer at 127.0.0.1:9118. 2023-10-21T04:52:01.945+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9124 connected=true 2023-10-21T04:52:01.945+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9106 connected=true 2023-10-21T04:52:01.946+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9112 connected=true 2023-10-21T04:52:01.946+05:30 [Info] MetadataProvider.CheckIndexerStatus(): adminport=127.0.0.1:9118 connected=true 2023-10-21T04:52:01.946+05:30 [Info] MetadataProvider is terminated. Cleaning up ... 2023-10-21T04:52:01.946+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:52:01.946+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:52:01.946+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:52:01.946+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:52:01.946+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:35056->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:01.946+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:58170->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:01.946+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:58640->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:01.947+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:38704->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:01.947+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38704. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:01.947+05:30 [Info] messageListener.start(): message channel closed. Remove peer 6a:8f:58:47:9d:d4:ca:2 as follower. 2023-10-21T04:52:01.954+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:38702. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:01.954+05:30 [Info] messageListener.start(): message channel closed. Remove peer 8c:6e:4f:b9:6d:62:74:ad as follower. 2023-10-21T04:52:02.317+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:52:02.428+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:52:02.446+05:30 [Info] Fetch new stats upon request by life cycle manager 2023-10-21T04:52:02.632+05:30 [Info] Timekeeper::setSnapshotType INIT_STREAM bucket_%2:_default:c1 setting snapshot type as DISK_SNAP due to BuildCompletionTS 2023-10-21T04:52:02.637+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1786384967016908289 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:52:02.637+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1786384967016908289 PartitionId: 0 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:1000 committed:true) SnapType DISK_SNAP SnapAligned true SnapCreateDur 2.957µs SnapOpenDur 460.11µs 2023-10-21T04:52:02.637+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1786384967016908289, PartitionId 0 SnapshotId 26 Creating recovery point ... 2023-10-21T04:52:02.639+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 1786384967016908289 INDEX_STATE_CATCHUP 2023-10-21T04:52:02.639+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 1786384967016908289 true 2023-10-21T04:52:02.639+05:30 [Info] Timekeeper::checkInitialBuildDone Initial Build Done Index: 1786384967016908289 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 Session: 43326 BuildTS: [0=19,1=22,2=27,3=21,4=24,5=16,6=26,7=24,8=32,9=22,10=20,11=21,12=25,13=23,14=22,15=21,16=15,17=20,18=18,19=25,20=23,21=28,22=30,23=18,24=19,25=28,26=24,27=24,28=27,29=20,30=19,31=22,32=25,33=20,34=22,35=28,36=22,37=19,38=20,39=25,40=24,41=21,42=26,43=27,44=20,45=30,46=23,47=22,48=26,49=19,50=19,51=21,52=24,53=31,54=21,55=17,56=25,57=18,58=26,59=16,60=18,61=24,62=18,63=26] 2023-10-21T04:52:02.639+05:30 [Info] Indexer::handleInitialBuildDone KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 43326 2023-10-21T04:52:02.639+05:30 [Info] Indexer::processBuildDoneCatchup INIT_STREAM bucket_%2:_default:c1 43326 2023-10-21T04:52:02.640+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:02.640+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:02.640+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_CATCHUP Stream: INIT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:02.640+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:52:02.641+05:30 [Info] send progress stats to clients 2023-10-21T04:52:02.643+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:52:02.643+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2:_default:c1 StreamId: INIT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: false 2023-10-21T04:52:02.643+05:30 [Info] Timekeeper::addIndextoStream IndexCount map[MAINT_STREAM:map[bucket_%2:4 bucket_1:5] INIT_STREAM:map[bucket_%2:_default:c1:1]] 2023-10-21T04:52:02.643+05:30 [Info] MutationMgr::handleAddIndexListToStream Message: MsgStreamUpdate Type: ADD_INDEX_LIST_TO_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [ InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CATCHUP RState: RebalActive Stream: INIT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 43326 CollectionId: CollectionAware: false EnableOSO: false RestartTs: NumVBuckets: 64 2023-10-21T04:52:02.647+05:30 [Info] KVSender::sendAddInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_%2 Instances [indexInstance:([`age`]) partitionScheme:SINGLE whereExpression:() > singlePartn:endpoints:"127.0.0.1:9123" > ] 2023-10-21T04:52:02.650+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 8.879114ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:02.657+05:30 [Info] KVSender::sendAddInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId bucket_%2 2023-10-21T04:52:02.657+05:30 [Info] Indexer::processBuildDoneCatchup Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43326 2023-10-21T04:52:02.657+05:30 [Info] Indexer::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43326 2023-10-21T04:52:02.657+05:30 [Info] Indexer::setMinMergeTsForCatchup INIT_STREAM bucket_%2:_default:c1 set minMergeTs 2023-10-21T04:52:02.657+05:30 [Info] Timekeeper::handleInitBuildDoneAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:52:02.657+05:30 [Info] Timekeeper::setAddInstPending INIT_STREAM bucket_%2:_default:c1 1786384967016908289 false 2023-10-21T04:52:02.657+05:30 [Info] Timekeeper::setMergeTs INIT_STREAM bucket_%2:_default:c1 1786384967016908289 2023-10-21T04:52:02.657+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: INIT_STREAM In Repair. INIT_STREAM cannot be merged. Continue both streams for keyspaceId bucket_%2:_default:c1. 2023-10-21T04:52:02.657+05:30 [Info] Indexer::processPendingBuildDone MAINT_STREAM bucket_%2 43326 2023-10-21T04:52:02.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9653303454929937, throttleDelayMs (new, old, change): (2, 0, 2) 2023-10-21T04:52:02.685+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:02.685+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:02.688+05:30 [Info] switched currmeta from 575 -> 576 force true 2023-10-21T04:52:02.701+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1786384967016908289, PartitionId 0 SnapshotId 26 Created recovery point (took 63.464098ms) 2023-10-21T04:52:02.706+05:30 [Info] StreamState::adjustNonSnapAlignedVbs KeyspaceId bucket_%2:_default:c1 StreamId INIT_STREAM Vbuckets []. 2023-10-21T04:52:02.706+05:30 [Info] Timekeeper::repairStream Nothing to repair for Stream INIT_STREAM and KeyspaceId bucket_%2:_default:c1 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Index Ready To Merge using MaintTs Index: 1786384967016908289 Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 SessionId: 43326, INIT_STREAM:LastFlushTs: bucket: bucket_%2, scopeId: , collectionId: , vbuckets: 64 Crc64: 11980419538711688961 snapType DISK_SNAP OpenOSOSnap false - {vbno, vbuuid, manifest, seqno, snapshot-start, snapshot-end} { 0 c87b6e21a140 2 19 0 19} { 1 d42be2a4222 2 32 0 32} { 2 4948df4ea759 2 27 0 27} { 3 7ba304e3194c 2 21 0 21} { 4 e0aec589e23b 2 26 0 26} { 5 9c9d40888648 2 16 0 16} { 6 b1131c488ca5 2 26 0 26} { 7 114a1f591031 2 24 0 24} { 8 a5e6dc4d694e 2 32 0 32} { 9 26e2e9d6800d 2 22 0 22} { 10 6e7d38a5e629 2 20 0 20} { 11 3ddcf01bb78f 2 21 0 21} { 12 9c4cc72ac8c1 2 25 0 25} { 13 6222e5af90c6 2 23 0 23} { 14 35d3d7450c2d 2 22 0 22} { 15 1671adc6bd6 2 21 0 21} { 16 8f5c3a70ba41 2 25 0 25} { 17 4306317909e4 2 20 0 20} { 18 3db43f3c94e3 2 20 0 20} { 19 9aac60f99331 2 25 0 25} { 20 c58049ac8a14 2 23 0 23} { 21 b07a150b5137 2 28 0 28} { 22 f56a1a70724 2 30 0 30} { 23 bb88574adb47 2 20 0 20} { 24 9b61736e2854 2 20 0 20} { 25 d40363f5eeac 2 28 0 28} { 26 351ac584d4f7 2 24 0 24} { 27 a2881c6f075e 2 24 0 24} { 28 e5a97945de57 2 27 0 27} { 29 2f99140645a8 2 20 0 20} { 30 a683aaff2edf 2 19 0 19} { 31 f62c73dc11dc 2 22 0 22} { 32 4e87a68313b6 2 25 0 25} { 33 d40560fdd9c4 2 20 0 20} { 34 f08abaa5b18c 2 22 0 22} { 35 aaf34afcf66f 2 30 0 30} { 36 fb65689d5a28 2 22 0 22} { 37 562c532fdd47 2 19 0 19} { 38 1bc770c28440 2 20 0 20} { 39 3c61e9face1f 2 25 0 25} { 40 99738d809a15 2 24 0 24} { 41 2dd1e43d846d 2 21 0 21} { 42 5af41bd0826b 2 26 0 26} { 43 875905e328bb 2 27 0 27} { 44 d70484ec2165 2 20 0 20} { 45 b54e8211350e 2 40 0 40} { 46 b5942d5b50a0 2 23 0 23} { 47 4d29cb50be97 2 22 0 22} { 48 77605a715cf9 2 26 0 26} { 49 d0b74133571a 2 19 0 19} { 50 18a452463af0 2 19 0 19} { 51 1a14a137c1bf 2 21 0 21} { 52 f4918a0ec561 2 24 0 24} { 53 712eb1048b7 2 31 0 31} { 54 3fb13ec11a23 2 21 0 21} { 55 451a34f23d23 2 17 0 17} { 56 d6236b65278a 2 27 0 27} { 57 274c0cc69c70 2 18 0 18} { 58 2adced599a7a 2 26 0 26} { 59 918867d98b87 2 17 0 17} { 60 3b4d1e9f31f5 2 20 0 20} { 61 8cf41263eb61 2 24 0 24} { 62 9b0d37783330 2 20 0 20} { 63 d98f16ed018 2 28 0 28} 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::changeIndexStateForKeyspaceId INIT_STREAM bucket_%2:_default:c1 1786384967016908289 INDEX_STATE_ACTIVE 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::checkInitStreamReadyToMerge: Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 State Changed to INACTIVE 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:52:02.707+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 2 CurrMaint 0 New 0 2023-10-21T04:52:02.707+05:30 [Info] Indexer::handleMergeInitStream keyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId: 43326 2023-10-21T04:52:02.707+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:02.707+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:02.707+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_ACTIVE Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43326 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::removeKeyspaceFromStream remove index 1786384967016908289 from stream INIT_STREAM keyspaceId bucket_%2:_default:c1 2023-10-21T04:52:02.707+05:30 [Info] Timekeeper::stopTimer INIT_STREAM bucket_%2:_default:c1 2023-10-21T04:52:02.708+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2:_default:c1 Deleted from Stream INIT_STREAM 2023-10-21T04:52:02.708+05:30 [Info] Timekeeper::resetTsQueueStats Stream INIT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:52:02.708+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: INIT_STREAM KeyspaceId: bucket_%2:_default:c1 BuildTS: [] IndexList: [] Async: false SessionId: 43326 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:02.708+05:30 [Info] MutationStreamReader:Shutdown StreamReader INIT_STREAM 2023-10-21T04:52:02.708+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 total received key-versions: map[0:12 1:15 2:20 3:14 4:17 5:9 6:19 7:17 8:25 9:15 10:13 11:14 12:18 13:16 14:15 15:14 16:8 17:13 18:11 19:18 20:16 21:21 22:23 23:11 24:12 25:21 26:17 27:17 28:20 29:13 30:12 31:15 32:18 33:13 34:15 35:21 36:15 37:12 38:13 39:18 40:17 41:14 42:19 43:20 44:13 45:23 46:16 47:15 48:19 49:12 50:12 51:14 52:17 53:24 54:14 55:10 56:18 57:11 58:19 59:9 60:11 61:17 62:11 63:19] 2023-10-21T04:52:02.708+05:30 [Info] DATP[->dataport ":9121"] keyspaceId: bucket_%2:_default:c1 latest sequence numbers: map[0:19 1:22 2:27 3:21 4:24 5:16 6:26 7:24 8:32 9:22 10:20 11:21 12:25 13:23 14:22 15:21 16:15 17:20 18:18 19:25 20:23 21:28 22:30 23:18 24:19 25:28 26:24 27:24 28:27 29:20 30:19 31:22 32:25 33:20 34:22 35:28 36:22 37:19 38:20 39:25 40:24 41:21 42:26 43:27 44:20 45:30 46:23 47:22 48:26 49:19 50:19 51:21 52:24 53:31 54:21 55:17 56:25 57:18 58:26 59:16 60:18 61:24 62:18 63:26] 2023-10-21T04:52:02.708+05:30 [Error] DATP[->dataport ":9121"] Accept() Error: accept tcp4 0.0.0.0:9121: use of closed network connection 2023-10-21T04:52:02.708+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9121->127.0.0.1:39404: use of closed network connection 2023-10-21T04:52:02.708+05:30 [Error] DATP[->dataport ":9121"] worker "127.0.0.1:39404" exit: read tcp4 127.0.0.1:9121->127.0.0.1:39404: use of closed network connection 2023-10-21T04:52:02.708+05:30 [Info] DATP[->dataport ":9121"] connection "127.0.0.1:39404" closed ! 2023-10-21T04:52:02.708+05:30 [Info] DATP[->dataport ":9121"] ... stopped 2023-10-21T04:52:02.709+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:02.709+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:02.709+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:02.709+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:02.709+05:30 [Info] Timekeeper::handleFlushStateChange Received Flush State Change for KeyspaceId: bucket_%2 StreamId: MAINT_STREAM Type: TK_ENABLE_FLUSH ResetPendingMerge: true 2023-10-21T04:52:02.709+05:30 [Info] ClustMgr:handleUpdateTopologyForIndex mType: CLUST_MGR_UPDATE_TOPOLOGY_FOR_INDEX indexList: [ InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_ACTIVE RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0} ] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:52:02.709+05:30 [Info] Indexer::handleMergeInitStream Merge Done KeyspaceId: bucket_%2:_default:c1 Stream: INIT_STREAM SessionId 43326 2023-10-21T04:52:02.711+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1] 2023-10-21T04:52:02.739+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:02.739+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:02.741+05:30 [Info] switched currmeta from 576 -> 577 force true 2023-10-21T04:52:02.749+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_UPDATE_INDEX_INST elapsed 39.714616ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:02.761+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:02.762+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:02.775+05:30 [Info] switched currmeta from 577 -> 578 force true 2023-10-21T04:52:02.782+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic INIT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2:_default:c1]. Err genServer.closed 2023-10-21T04:52:02.782+05:30 [Error] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:52:02.782+05:30 [Info] KVSender::deleteKeyspacesFromStream INIT_STREAM [bucket_%2:_default:c1] Treating genServer.closed As Success 2023-10-21T04:52:02.782+05:30 [Info] Indexer::handleMergeInitStream Success Stream INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43326 2023-10-21T04:52:02.782+05:30 [Info] Indexer::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 SessionId 43326 2023-10-21T04:52:02.782+05:30 [Info] Timekeeper::handleMergeStreamAck StreamId INIT_STREAM KeyspaceId bucket_%2:_default:c1 2023-10-21T04:52:02.849+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:02.849+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:02.867+05:30 [Info] switched currmeta from 578 -> 579 force true 2023-10-21T04:52:03.067+05:30 DEBU REGU.impl.(*storageLimitingImpl).reconcileStorageReports() at storagelimiting.go:361 [id 244] aggregate storage info: map[bucket_%2:map[kv:2496704 index:15412723] bucket_1:map[kv:2590912 index:16895724]] 2023-10-21T04:52:03.176+05:30 [Info] Timekeeper:: MAINT_STREAM bucket_%2 Forcing Overdue Commit 2023-10-21T04:52:03.177+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 5 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.177+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 1786384967016908289 PartitionId 0 Creating New Snapshot SnapshotInfo: count:1000 committed:true 2023-10-21T04:52:03.177+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.177+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 1786384967016908289 PartitionId: 0 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:1000 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.959µs SnapOpenDur 122.778µs 2023-10-21T04:52:03.177+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 17465522551378543576 PartitionId 0 Creating New Snapshot SnapshotInfo: count:6 committed:true 2023-10-21T04:52:03.177+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1786384967016908289, PartitionId 0 SnapshotId 27 Creating recovery point ... 2023-10-21T04:52:03.177+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17465522551378543576, PartitionId 0 SnapshotId 4 Creating recovery point ... 2023-10-21T04:52:03.177+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 5 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 3.223µs SnapOpenDur 104.102µs 2023-10-21T04:52:03.177+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 6 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.177+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 17465522551378543576 PartitionId: 0 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:6 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.562µs SnapOpenDur 72.802µs 2023-10-21T04:52:03.177+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 6 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.177+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 6 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.327µs SnapOpenDur 45.825µs 2023-10-21T04:52:03.177+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 7 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.177+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 7 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 1.736µs SnapOpenDur 32.826µs 2023-10-21T04:52:03.177+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 8 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.177+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 8 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.016µs SnapOpenDur 74.639µs 2023-10-21T04:52:03.177+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 7 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.177+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 8 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.178+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 1 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.178+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 1 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.178+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 1 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 4.549µs SnapOpenDur 86.993µs 2023-10-21T04:52:03.178+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 2 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.178+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 2 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.264µs SnapOpenDur 45.219µs 2023-10-21T04:52:03.178+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 3 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.178+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 3 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.293µs SnapOpenDur 34.376µs 2023-10-21T04:52:03.178+05:30 [Info] plasmaSlice::OpenSnapshot SliceId 0 IndexInstId 12606056968936758026 PartitionId 4 Creating New Snapshot SnapshotInfo: count:0 committed:true 2023-10-21T04:52:03.178+05:30 [Info] StorageMgr::handleCreateSnapshot Added New Snapshot Index: 12606056968936758026 PartitionId: 4 SliceId: 0 Crc64: 11980419538711688961 (SnapshotInfo: count:0 committed:true) SnapType FORCE_COMMIT SnapAligned true SnapCreateDur 2.803µs SnapOpenDur 45.405µs 2023-10-21T04:52:03.178+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 3 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.178+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 2 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.178+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 4 SnapshotId 2 Creating recovery point ... 2023-10-21T04:52:03.208+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17465522551378543576, PartitionId 0 SnapshotId 4 Created recovery point (took 31.350672ms) 2023-10-21T04:52:03.209+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 17465522551378543576, PartitionId 0 Cleanup mainstore recovery point SnapshotInfo: count:6 committed:false. num RPs 3. 2023-10-21T04:52:03.267+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 8 SnapshotId 2 Created recovery point (took 89.019135ms) 2023-10-21T04:52:03.268+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 8 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.268+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 8 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.313+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5 SnapshotId 2 Created recovery point (took 135.860993ms) 2023-10-21T04:52:03.313+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 4. 2023-10-21T04:52:03.314+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.314+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 4. 2023-10-21T04:52:03.315+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.320+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 1786384967016908289, PartitionId 0 SnapshotId 27 Created recovery point (took 143.436763ms) 2023-10-21T04:52:03.326+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 6 SnapshotId 2 Created recovery point (took 149.205086ms) 2023-10-21T04:52:03.327+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 6 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.327+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 6 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.334+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 7 SnapshotId 2 Created recovery point (took 156.573372ms) 2023-10-21T04:52:03.335+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 7 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.335+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 7 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.350+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 1 SnapshotId 2 Created recovery point (took 171.862186ms) 2023-10-21T04:52:03.351+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 1 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.351+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 1 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.376+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 3 SnapshotId 2 Created recovery point (took 197.510259ms) 2023-10-21T04:52:03.377+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 3 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.378+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 3 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.391+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 2 SnapshotId 2 Created recovery point (took 212.680685ms) 2023-10-21T04:52:03.397+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 2 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.398+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 2 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.433+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 4 SnapshotId 2 Created recovery point (took 255.251ms) 2023-10-21T04:52:03.434+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 4 Cleanup mainstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.435+05:30 [Info] PlasmaSlice Slice Id 0, IndexInstId 12606056968936758026, PartitionId 4 Cleanup backstore recovery point SnapshotInfo: count:0 committed:false. num RPs 3. 2023-10-21T04:52:03.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.786959521607632, throttleDelayMs (new, old, change): (0, 2, -2) 2023-10-21T04:52:03.707+05:30 [Info] timekeeper.repairMissingStreamBegin stream INIT_STREAM done 2023-10-21T04:52:03.785+05:30 [Info] RebalanceServiceManager::rebalanceJanitor Running Periodic Cleanup 2023-10-21T04:52:03.894+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1786384967016908289:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:52:03.894+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1786384967016908289:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:52:03.894+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1786384967016908289:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:52:03.894+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1786384967016908289:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:52:03.894+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[1786384967016908289:[0]]. New rollbacktime: 1697843682957894080, Old rollbacktime: 0 2023-10-21T04:52:04.043+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9928478567258509811:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:52:04.043+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9928478567258509811:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:52:04.043+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9928478567258509811:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:52:04.043+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9928478567258509811:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:52:04.043+05:30 [Info] Rollback time has changed for the following indexes {Index inst : [partitions]}: map[9928478567258509811:[0]]. New rollbacktime: 1697843682512381626, Old rollbacktime: 0 2023-10-21T04:52:05.964+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:52:05.976+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:05.976+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:05.979+05:30 [Info] switched currmeta from 579 -> 579 force true 2023-10-21T04:52:10.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9707439931217061, throttleDelayMs (new, old, change): (2, 0, 2) 2023-10-21T04:52:11.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 1, throttleDelayMs (new, old, change): (7, 2, 5) 2023-10-21T04:52:12.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 1, throttleDelayMs (new, old, change): (10, 7, 3) 2023-10-21T04:52:13.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.8726144205580232, throttleDelayMs (new, old, change): (5, 10, -5) 2023-10-21T04:52:14.107+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 26] call. err: operation_canceled 2023-10-21T04:52:14.107+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:52:14.107+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 26] Tasks:[]} 2023-10-21T04:52:14.109+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:52:14.109+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 26] Nodes:[4325b30e7f9620a998388078247335a8 ba6304988a6335d65281381d6cfc5e32] IsBalanced:false Messages:[]} 2023-10-21T04:52:14.109+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 26] 2023-10-21T04:52:14.110+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 26] 2023-10-21T04:52:14.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.5952704453287399, throttleDelayMs (new, old, change): (0, 5, -5) 2023-10-21T04:52:14.972+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 8303154871847919944 &{0} 2023-10-21T04:52:14.972+05:30 [Info] Indexer::handleDropIndex - IndexInstId 8303154871847919944 2023-10-21T04:52:14.972+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [8303154871847919944] 2023-10-21T04:52:14.974+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [8303154871847919944] 2023-10-21T04:52:14.974+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:14.974+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [8303154871847919944] 2023-10-21T04:52:14.975+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:52:14.975+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:52:14.975+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex 2023-10-21T04:52:14.975+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex stopped 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.975+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.975+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.976+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.976+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.976+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.976+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:52:14.977+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:52:14.976+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 8303154871847919944 Defn: DefnId: 17703248484023971553 Name: idx_partitioned_defer Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:52:14.977+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex 2023-10-21T04:52:14.977+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.922015ms 2023-10-21T04:52:14.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex stopped 2023-10-21T04:52:14.977+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 4.968196ms 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.977+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 8303154871847919944 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.976+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:52:14.977+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:52:14.977+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex 2023-10-21T04:52:14.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex stopped 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.976+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.977+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.977+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:52:14.977+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex 2023-10-21T04:52:14.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex stopped 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.976+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.976+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.976+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex 2023-10-21T04:52:14.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex stopped 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.976+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 8303154871847919944, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:52:14.978+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex 2023-10-21T04:52:14.978+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex stopped 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.978+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:52:14.979+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:52:14.979+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex 2023-10-21T04:52:14.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex stopped 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:52:14.979+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:52:14.979+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex 2023-10-21T04:52:14.979+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex stopped 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.979+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:14.995+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:14.995+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex] 2023-10-21T04:52:14.996+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex closed 2023-10-21T04:52:14.996+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:14.996+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:14.996+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:14.997+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:14.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:52:14.997+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:52:14.997+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:52:14.997+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:52:14.997+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex 2023-10-21T04:52:14.997+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex stopped 2023-10-21T04:52:14.997+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:14.997+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:14.997+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:14.997+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.016+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex] 2023-10-21T04:52:15.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex closed 2023-10-21T04:52:15.017+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex] 2023-10-21T04:52:15.017+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex closed 2023-10-21T04:52:15.017+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.017+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex ... 2023-10-21T04:52:15.018+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex ... 2023-10-21T04:52:15.019+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.019+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.019+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.020+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.020+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:52:15.020+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:52:15.020+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:52:15.020+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:52:15.021+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex 2023-10-21T04:52:15.021+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex stopped 2023-10-21T04:52:15.021+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.021+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.021+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.021+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.026+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex ... 2023-10-21T04:52:15.027+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex ... 2023-10-21T04:52:15.032+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex] 2023-10-21T04:52:15.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex closed 2023-10-21T04:52:15.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.033+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:52:15.034+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 69.683943ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:15.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.035+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_6.index destroyed 2023-10-21T04:52:15.035+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 6, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.035+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 6 Destroy Done 2023-10-21T04:52:15.035+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:52:15.036+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:52:15.036+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:52:15.036+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex 2023-10-21T04:52:15.036+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex stopped 2023-10-21T04:52:15.036+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.036+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.036+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.036+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.040+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex] 2023-10-21T04:52:15.040+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex closed 2023-10-21T04:52:15.036+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.042+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.043+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.046+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.052+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.053+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex] 2023-10-21T04:52:15.056+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:15.059+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex] 2023-10-21T04:52:15.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex closed 2023-10-21T04:52:15.062+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.068+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.068+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex] 2023-10-21T04:52:15.062+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.062+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:15.061+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex closed 2023-10-21T04:52:15.068+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:52:15.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex closed 2023-10-21T04:52:15.069+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.071+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.073+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.074+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:52:15.075+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:52:15.075+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:52:15.075+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex 2023-10-21T04:52:15.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex] 2023-10-21T04:52:15.074+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:52:15.075+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 7 Close Done 2023-10-21T04:52:15.075+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex stopped 2023-10-21T04:52:15.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex] 2023-10-21T04:52:15.076+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex closed 2023-10-21T04:52:15.083+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.084+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.084+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.084+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.084+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex ... 2023-10-21T04:52:15.085+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex ... 2023-10-21T04:52:15.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.087+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:52:15.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex closed 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.088+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.088+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:52:15.088+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex 2023-10-21T04:52:15.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex stopped 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.088+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] 2023-10-21T04:52:15.088+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex 2023-10-21T04:52:15.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex stopped 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.088+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:52:15.088+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:52:15.088+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex 2023-10-21T04:52:15.088+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex stopped 2023-10-21T04:52:15.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.089+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex ... 2023-10-21T04:52:15.090+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex ... 2023-10-21T04:52:15.093+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_7.index destroyed 2023-10-21T04:52:15.094+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 7, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.094+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 7 Destroy Done 2023-10-21T04:52:15.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex ... 2023-10-21T04:52:15.094+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex ... 2023-10-21T04:52:15.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex ... 2023-10-21T04:52:15.097+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex ... 2023-10-21T04:52:15.098+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex] 2023-10-21T04:52:15.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex closed 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Mainstore#8303154871847919944:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:52:15.100+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_8.index destroyed 2023-10-21T04:52:15.107+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 8, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.107+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 8 Destroy Done 2023-10-21T04:52:15.107+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:52:15.107+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex 2023-10-21T04:52:15.107+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex stopped 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.107+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.130+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex] 2023-10-21T04:52:15.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex closed 2023-10-21T04:52:15.131+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.131+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex ... 2023-10-21T04:52:15.131+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex ... 2023-10-21T04:52:15.136+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex] 2023-10-21T04:52:15.137+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex closed 2023-10-21T04:52:15.137+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.137+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.138+05:30 [Info] switched currmeta from 579 -> 583 force true 2023-10-21T04:52:15.146+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex] 2023-10-21T04:52:15.147+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex closed 2023-10-21T04:52:15.147+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.147+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex ... 2023-10-21T04:52:15.151+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex ... 2023-10-21T04:52:15.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex ... 2023-10-21T04:52:15.152+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex ... 2023-10-21T04:52:15.152+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_1.index destroyed 2023-10-21T04:52:15.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex] 2023-10-21T04:52:15.160+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex closed 2023-10-21T04:52:15.161+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.161+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.161+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 1, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.161+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 1 Destroy Done 2023-10-21T04:52:15.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex ... 2023-10-21T04:52:15.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex ... 2023-10-21T04:52:15.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex ... 2023-10-21T04:52:15.165+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex ... 2023-10-21T04:52:15.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.169+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_3.index destroyed 2023-10-21T04:52:15.169+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 3, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.169+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 3 Destroy Done 2023-10-21T04:52:15.171+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:15.171+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:15.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.173+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex ... 2023-10-21T04:52:15.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex ... 2023-10-21T04:52:15.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex ... 2023-10-21T04:52:15.174+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex ... 2023-10-21T04:52:15.176+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.187+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_4.index destroyed 2023-10-21T04:52:15.188+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 4, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.188+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 4 Destroy Done 2023-10-21T04:52:15.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex] 2023-10-21T04:52:15.188+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex closed 2023-10-21T04:52:15.188+05:30 [Info] bucket_%2/idx_partitioned_defer/Backstore#8303154871847919944:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.188+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex ... 2023-10-21T04:52:15.192+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex ... 2023-10-21T04:52:15.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex ... 2023-10-21T04:52:15.195+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex ... 2023-10-21T04:52:15.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.197+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_5.index destroyed 2023-10-21T04:52:15.197+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 5, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.197+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 5 Destroy Done 2023-10-21T04:52:15.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex ... 2023-10-21T04:52:15.199+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex ... 2023-10-21T04:52:15.209+05:30 [Info] switched currmeta from 583 -> 584 force true 2023-10-21T04:52:15.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.217+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_defer_8303154871847919944_2.index destroyed 2023-10-21T04:52:15.217+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 8303154871847919944, PartitionId 2, IndexDefnId 17703248484023971553. 2023-10-21T04:52:15.217+05:30 [Info] Indexer::cleanupIndexData IndexInst 8303154871847919944 Partition 2 Destroy Done 2023-10-21T04:52:15.359+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 12606056968936758026 &{0} 2023-10-21T04:52:15.359+05:30 [Info] Indexer::handleDropIndex - IndexInstId 12606056968936758026 2023-10-21T04:52:15.359+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:52:15.359+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:15.359+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 12606056968936758026 Name: idx_partitioned Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:52:15.359+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [12606056968936758026] 2023-10-21T04:52:15.360+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [12606056968936758026] 2023-10-21T04:52:15.360+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:15.360+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [12606056968936758026] 2023-10-21T04:52:15.360+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex 2023-10-21T04:52:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex stopped 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.361+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.361+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.361+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex 2023-10-21T04:52:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex stopped 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.361+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.361+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.361+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.361+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex] 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex] 2023-10-21T04:52:15.362+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:15.362+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 12606056968936758026 Defn: DefnId: 4390029557575378783 Name: idx_partitioned Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 1 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:15.362+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.707009ms 2023-10-21T04:52:15.362+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex] 2023-10-21T04:52:15.361+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex 2023-10-21T04:52:15.362+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex stopped 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.362+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.363+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 12606056968936758026, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex] 2023-10-21T04:52:15.363+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex] 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex] 2023-10-21T04:52:15.363+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex 2023-10-21T04:52:15.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex stopped 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.363+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 12606056968936758026 2023-10-21T04:52:15.363+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex] 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex] 2023-10-21T04:52:15.363+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex 2023-10-21T04:52:15.363+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex stopped 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.363+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.364+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [12606056968936758026] 2023-10-21T04:52:15.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex] 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex] 2023-10-21T04:52:15.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex 2023-10-21T04:52:15.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex stopped 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex] 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex] 2023-10-21T04:52:15.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex 2023-10-21T04:52:15.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex stopped 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex] 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex] 2023-10-21T04:52:15.364+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex 2023-10-21T04:52:15.364+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex stopped 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.364+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.365+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.365+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.376+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:52:15.376+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73680 2023-10-21T04:52:15.376+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 13.590293ms 2023-10-21T04:52:15.376+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 17.542979ms 2023-10-21T04:52:15.378+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex] 2023-10-21T04:52:15.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex closed 2023-10-21T04:52:15.378+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.378+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.378+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.378+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.378+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex] 2023-10-21T04:52:15.378+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex 2023-10-21T04:52:15.378+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex] 2023-10-21T04:52:15.379+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex] 2023-10-21T04:52:15.379+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex 2023-10-21T04:52:15.379+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex stopped 2023-10-21T04:52:15.379+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.379+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.379+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.379+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.397+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex] 2023-10-21T04:52:15.398+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex closed 2023-10-21T04:52:15.398+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.398+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.398+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.399+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.413+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex] 2023-10-21T04:52:15.413+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex closed 2023-10-21T04:52:15.414+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.421+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex ... 2023-10-21T04:52:15.421+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex ... 2023-10-21T04:52:15.399+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex] 2023-10-21T04:52:15.429+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex 2023-10-21T04:52:15.429+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex] 2023-10-21T04:52:15.429+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex] 2023-10-21T04:52:15.429+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex 2023-10-21T04:52:15.429+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex stopped 2023-10-21T04:52:15.430+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.430+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.430+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.430+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.437+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.438+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:15.448+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:15.442+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex ... 2023-10-21T04:52:15.450+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex ... 2023-10-21T04:52:15.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.454+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_1.index destroyed 2023-10-21T04:52:15.454+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 1, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.455+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 1 Destroy Done 2023-10-21T04:52:15.458+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 193.787014ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:15.444+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex] 2023-10-21T04:52:15.462+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex closed 2023-10-21T04:52:15.463+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.463+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 3 Close Done 2023-10-21T04:52:15.448+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.463+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex] 2023-10-21T04:52:15.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex closed 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex] 2023-10-21T04:52:15.464+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex] 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex] 2023-10-21T04:52:15.464+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex 2023-10-21T04:52:15.464+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex stopped 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.464+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.465+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.467+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex ... 2023-10-21T04:52:15.472+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex ... 2023-10-21T04:52:15.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.486+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.488+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.488+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex ... 2023-10-21T04:52:15.489+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex ... 2023-10-21T04:52:15.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex] 2023-10-21T04:52:15.492+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex closed 2023-10-21T04:52:15.493+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.493+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.493+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.493+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.493+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex] 2023-10-21T04:52:15.502+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex] 2023-10-21T04:52:15.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex closed 2023-10-21T04:52:15.503+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.503+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.503+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.503+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.503+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex] 2023-10-21T04:52:15.505+05:30 [Info] switched currmeta from 584 -> 587 force true 2023-10-21T04:52:15.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_3.index destroyed 2023-10-21T04:52:15.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex] 2023-10-21T04:52:15.527+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex closed 2023-10-21T04:52:15.528+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.528+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex ... 2023-10-21T04:52:15.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex ... 2023-10-21T04:52:15.530+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex 2023-10-21T04:52:15.531+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex] 2023-10-21T04:52:15.531+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex] 2023-10-21T04:52:15.531+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex 2023-10-21T04:52:15.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex stopped 2023-10-21T04:52:15.532+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.532+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.532+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.533+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.533+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex 2023-10-21T04:52:15.541+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex] 2023-10-21T04:52:15.541+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex] 2023-10-21T04:52:15.541+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex 2023-10-21T04:52:15.541+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex stopped 2023-10-21T04:52:15.541+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.541+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.541+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.541+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.542+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 3, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.542+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 3 Destroy Done 2023-10-21T04:52:15.555+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.557+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex] 2023-10-21T04:52:15.557+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex closed 2023-10-21T04:52:15.558+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.558+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.564+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.565+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex] 2023-10-21T04:52:15.567+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex closed 2023-10-21T04:52:15.568+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.568+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.568+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.572+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:15.572+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:15.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex ... 2023-10-21T04:52:15.582+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex ... 2023-10-21T04:52:15.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex] 2023-10-21T04:52:15.583+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex closed 2023-10-21T04:52:15.584+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.584+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.585+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.590+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex] 2023-10-21T04:52:15.590+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex ... 2023-10-21T04:52:15.591+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex ... 2023-10-21T04:52:15.594+05:30 [Info] switched currmeta from 587 -> 587 force true 2023-10-21T04:52:15.595+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.602+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.605+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex] 2023-10-21T04:52:15.606+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex closed 2023-10-21T04:52:15.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex] 2023-10-21T04:52:15.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex closed 2023-10-21T04:52:15.607+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.607+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.607+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex ... 2023-10-21T04:52:15.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex ... 2023-10-21T04:52:15.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex] 2023-10-21T04:52:15.607+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.608+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.608+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.608+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex] 2023-10-21T04:52:15.607+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.608+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_6.index destroyed 2023-10-21T04:52:15.608+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 6, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.608+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 6 Destroy Done 2023-10-21T04:52:15.609+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex 2023-10-21T04:52:15.609+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex] 2023-10-21T04:52:15.609+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex] 2023-10-21T04:52:15.609+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex 2023-10-21T04:52:15.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex stopped 2023-10-21T04:52:15.609+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.609+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.609+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.609+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.608+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex ... 2023-10-21T04:52:15.609+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex ... 2023-10-21T04:52:15.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.613+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index/docIndex sucessfully destroyed 2023-10-21T04:52:15.613+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_7.index destroyed 2023-10-21T04:52:15.614+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 7, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.614+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 7 Destroy Done 2023-10-21T04:52:15.614+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex 2023-10-21T04:52:15.614+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex] 2023-10-21T04:52:15.614+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex] 2023-10-21T04:52:15.615+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex 2023-10-21T04:52:15.615+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex 2023-10-21T04:52:15.615+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex] 2023-10-21T04:52:15.615+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex] 2023-10-21T04:52:15.615+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex 2023-10-21T04:52:15.615+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex] 2023-10-21T04:52:15.616+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex closed 2023-10-21T04:52:15.616+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.616+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex stopped 2023-10-21T04:52:15.617+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.617+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.617+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.617+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.617+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex stopped 2023-10-21T04:52:15.618+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.618+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.618+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.619+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.620+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex] 2023-10-21T04:52:15.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex closed 2023-10-21T04:52:15.620+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.620+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.620+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex ... 2023-10-21T04:52:15.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex ... 2023-10-21T04:52:15.621+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex ... 2023-10-21T04:52:15.622+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex ... 2023-10-21T04:52:15.627+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.627+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.628+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex ... 2023-10-21T04:52:15.631+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex ... 2023-10-21T04:52:15.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.632+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_8.index destroyed 2023-10-21T04:52:15.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex] 2023-10-21T04:52:15.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex closed 2023-10-21T04:52:15.633+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 8, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.633+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 8 Destroy Done 2023-10-21T04:52:15.633+05:30 [Info] bucket_%2/idx_partitioned/Backstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.634+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.633+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex ... 2023-10-21T04:52:15.635+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex ... 2023-10-21T04:52:15.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_2.index destroyed 2023-10-21T04:52:15.638+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 2, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.640+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 2 Destroy Done 2023-10-21T04:52:15.647+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.654+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex ... 2023-10-21T04:52:15.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex ... 2023-10-21T04:52:15.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex ... 2023-10-21T04:52:15.655+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex ... 2023-10-21T04:52:15.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.661+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.670+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.670+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_4.index destroyed 2023-10-21T04:52:15.670+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 4, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.671+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 4 Destroy Done 2023-10-21T04:52:15.671+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex ... 2023-10-21T04:52:15.672+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex ... 2023-10-21T04:52:15.679+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.680+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_partitioned_12606056968936758026_5.index destroyed 2023-10-21T04:52:15.680+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 12606056968936758026, PartitionId 5, IndexDefnId 4390029557575378783. 2023-10-21T04:52:15.680+05:30 [Info] Indexer::cleanupIndexData IndexInst 12606056968936758026 Partition 5 Destroy Done 2023-10-21T04:52:15.702+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 16759912838651900043 &{0} 2023-10-21T04:52:15.703+05:30 [Info] Indexer::handleDropIndex - IndexInstId 16759912838651900043 2023-10-21T04:52:15.704+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:52:15.708+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:15.708+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 16759912838651900043 Name: idx_partitioned Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:52:15.709+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [16759912838651900043] 2023-10-21T04:52:15.709+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [16759912838651900043] 2023-10-21T04:52:15.709+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:15.710+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [16759912838651900043] 2023-10-21T04:52:15.711+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.712+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.712+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.712+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.713+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex] 2023-10-21T04:52:15.713+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.713+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex 2023-10-21T04:52:15.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex] 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.713+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex] 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.713+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.713+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.713+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex] 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex] 2023-10-21T04:52:15.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex] 2023-10-21T04:52:15.713+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:15.713+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex] 2023-10-21T04:52:15.714+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex] 2023-10-21T04:52:15.714+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex 2023-10-21T04:52:15.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex stopped 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.714+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 16759912838651900043 Defn: DefnId: 7790129686881525591 Name: idx_partitioned Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`emalid`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.714+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 11.510199ms 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.714+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex] 2023-10-21T04:52:15.714+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex 2023-10-21T04:52:15.714+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex] 2023-10-21T04:52:15.714+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex] 2023-10-21T04:52:15.714+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex 2023-10-21T04:52:15.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex stopped 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.714+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.714+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 16759912838651900043 2023-10-21T04:52:15.715+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.715+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.715+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.740+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.740+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex] 2023-10-21T04:52:15.715+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [16759912838651900043] 2023-10-21T04:52:15.721+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex 2023-10-21T04:52:15.744+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex] 2023-10-21T04:52:15.745+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex] 2023-10-21T04:52:15.745+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex 2023-10-21T04:52:15.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex stopped 2023-10-21T04:52:15.745+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex 2023-10-21T04:52:15.745+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.745+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.745+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.745+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.745+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex] 2023-10-21T04:52:15.745+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex] 2023-10-21T04:52:15.745+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex 2023-10-21T04:52:15.745+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex stopped 2023-10-21T04:52:15.745+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.746+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex 2023-10-21T04:52:15.746+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex] 2023-10-21T04:52:15.746+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex] 2023-10-21T04:52:15.746+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex 2023-10-21T04:52:15.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex stopped 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.746+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex 2023-10-21T04:52:15.746+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex] 2023-10-21T04:52:15.746+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex] 2023-10-21T04:52:15.746+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex 2023-10-21T04:52:15.746+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex stopped 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.746+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.747+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.747+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.747+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex 2023-10-21T04:52:15.747+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex] 2023-10-21T04:52:15.747+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex] 2023-10-21T04:52:15.747+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex 2023-10-21T04:52:15.751+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 86.334761ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:15.756+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:52:15.757+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51546 2023-10-21T04:52:15.757+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 42.738654ms 2023-10-21T04:52:15.757+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 54.580953ms 2023-10-21T04:52:15.761+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.762+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex] 2023-10-21T04:52:15.769+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex closed 2023-10-21T04:52:15.769+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.822+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.823+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.823+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.823+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.747+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex stopped 2023-10-21T04:52:15.844+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.844+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.818+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.843+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex] 2023-10-21T04:52:15.777+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex 2023-10-21T04:52:15.845+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex] 2023-10-21T04:52:15.845+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex] 2023-10-21T04:52:15.845+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex 2023-10-21T04:52:15.844+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex stopped 2023-10-21T04:52:15.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.845+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex] 2023-10-21T04:52:15.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex closed 2023-10-21T04:52:15.847+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex] 2023-10-21T04:52:15.845+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex 2023-10-21T04:52:15.846+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.848+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex closed 2023-10-21T04:52:15.869+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.869+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex] 2023-10-21T04:52:15.869+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex] 2023-10-21T04:52:15.869+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex 2023-10-21T04:52:15.869+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex stopped 2023-10-21T04:52:15.869+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.869+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.869+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.870+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.873+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.878+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.879+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex] 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex closed 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex] 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex closed 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex] 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex closed 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex] 2023-10-21T04:52:15.879+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex closed 2023-10-21T04:52:15.880+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.880+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.880+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.880+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex] 2023-10-21T04:52:15.880+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex 2023-10-21T04:52:15.880+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex] 2023-10-21T04:52:15.880+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex] 2023-10-21T04:52:15.880+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex 2023-10-21T04:52:15.880+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex stopped 2023-10-21T04:52:15.880+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.880+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.880+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex 2023-10-21T04:52:15.882+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex 2023-10-21T04:52:15.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex stopped 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex 2023-10-21T04:52:15.882+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex 2023-10-21T04:52:15.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex stopped 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex 2023-10-21T04:52:15.882+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex] 2023-10-21T04:52:15.882+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex 2023-10-21T04:52:15.882+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex stopped 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.881+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.882+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.883+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex] 2023-10-21T04:52:15.883+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex 2023-10-21T04:52:15.883+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex] 2023-10-21T04:52:15.883+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex] 2023-10-21T04:52:15.883+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex 2023-10-21T04:52:15.883+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex stopped 2023-10-21T04:52:15.883+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.883+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.883+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.883+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.888+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.888+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex] 2023-10-21T04:52:15.888+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex 2023-10-21T04:52:15.889+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex] 2023-10-21T04:52:15.889+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex] 2023-10-21T04:52:15.889+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex 2023-10-21T04:52:15.889+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex stopped 2023-10-21T04:52:15.889+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.889+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.889+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.889+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.925+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex] 2023-10-21T04:52:15.926+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex closed 2023-10-21T04:52:15.926+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.926+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex ... 2023-10-21T04:52:15.927+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex ... 2023-10-21T04:52:15.932+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.936+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex] 2023-10-21T04:52:15.937+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex closed 2023-10-21T04:52:15.937+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.937+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.940+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex ... 2023-10-21T04:52:15.945+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex ... 2023-10-21T04:52:15.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex] 2023-10-21T04:52:15.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex closed 2023-10-21T04:52:15.955+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.955+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:15.955+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:15.955+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:15.955+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex] 2023-10-21T04:52:15.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex ... 2023-10-21T04:52:15.956+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex ... 2023-10-21T04:52:15.960+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.964+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_5.index destroyed 2023-10-21T04:52:15.965+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 5, IndexDefnId 7790129686881525591. 2023-10-21T04:52:15.965+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 5 Destroy Done 2023-10-21T04:52:15.965+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex 2023-10-21T04:52:15.972+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex] 2023-10-21T04:52:15.972+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex] 2023-10-21T04:52:15.972+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex 2023-10-21T04:52:15.972+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex stopped 2023-10-21T04:52:15.972+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:15.972+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:15.972+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:15.972+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:15.968+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:15.972+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:15.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex] 2023-10-21T04:52:15.973+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex closed 2023-10-21T04:52:15.973+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.973+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.987+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:15.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex] 2023-10-21T04:52:15.987+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex closed 2023-10-21T04:52:15.987+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:15.987+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:15.988+05:30 [Info] switched currmeta from 587 -> 591 force true 2023-10-21T04:52:15.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:15.988+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/mainIndex sucessfully destroyed 2023-10-21T04:52:15.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex ... 2023-10-21T04:52:15.989+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 8 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex ... 2023-10-21T04:52:15.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex ... 2023-10-21T04:52:15.991+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex ... 2023-10-21T04:52:16.004+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.010+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex ... 2023-10-21T04:52:16.024+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex ... 2023-10-21T04:52:16.011+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.012+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.028+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.030+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_6.index destroyed 2023-10-21T04:52:16.031+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 6, IndexDefnId 7790129686881525591. 2023-10-21T04:52:16.031+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 6 Destroy Done 2023-10-21T04:52:16.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex] 2023-10-21T04:52:16.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex closed 2023-10-21T04:52:16.031+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.031+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.031+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex] 2023-10-21T04:52:16.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex closed 2023-10-21T04:52:16.032+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.032+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex] 2023-10-21T04:52:16.032+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex closed 2023-10-21T04:52:16.033+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.033+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex ... 2023-10-21T04:52:16.033+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 3 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex ... 2023-10-21T04:52:16.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index/docIndex sucessfully destroyed 2023-10-21T04:52:16.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex] 2023-10-21T04:52:16.039+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex closed 2023-10-21T04:52:16.040+05:30 [Info] bucket_1/idx_secondary/Backstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.040+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 2 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index in /opt/build/ns_server/data/n_3/data/@2i ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_1.index destroyed 2023-10-21T04:52:16.041+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 1, IndexDefnId 7790129686881525591. 2023-10-21T04:52:16.041+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 1 Destroy Done 2023-10-21T04:52:16.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.041+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex ... 2023-10-21T04:52:16.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex ... 2023-10-21T04:52:16.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex ... 2023-10-21T04:52:16.042+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 10 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex ... 2023-10-21T04:52:16.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.046+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_8.index destroyed 2023-10-21T04:52:16.047+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 8, IndexDefnId 7790129686881525591. 2023-10-21T04:52:16.047+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 8 Destroy Done 2023-10-21T04:52:16.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.047+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex ... 2023-10-21T04:52:16.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex ... 2023-10-21T04:52:16.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex ... 2023-10-21T04:52:16.048+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 9 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex ... 2023-10-21T04:52:16.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex ... 2023-10-21T04:52:16.053+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex ... 2023-10-21T04:52:16.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.055+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index/docIndex sucessfully destroyed 2023-10-21T04:52:16.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex ... 2023-10-21T04:52:16.056+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 4 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_7.index destroyed 2023-10-21T04:52:16.059+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 7, IndexDefnId 7790129686881525591. 2023-10-21T04:52:16.059+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 7 Destroy Done 2023-10-21T04:52:16.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.060+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex ... 2023-10-21T04:52:16.062+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 7 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex ... 2023-10-21T04:52:16.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_2.index destroyed 2023-10-21T04:52:16.063+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 2, IndexDefnId 7790129686881525591. 2023-10-21T04:52:16.063+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 2 Destroy Done 2023-10-21T04:52:16.063+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex ... 2023-10-21T04:52:16.064+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 5 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex ... 2023-10-21T04:52:16.066+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.067+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_3.index destroyed 2023-10-21T04:52:16.067+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 3, IndexDefnId 7790129686881525591. 2023-10-21T04:52:16.068+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 3 Destroy Done 2023-10-21T04:52:16.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.068+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex ... 2023-10-21T04:52:16.069+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 6 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex ... 2023-10-21T04:52:16.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.074+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_16759912838651900043_4.index destroyed 2023-10-21T04:52:16.075+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 16759912838651900043, PartitionId 4, IndexDefnId 7790129686881525591. 2023-10-21T04:52:16.075+05:30 [Info] Indexer::cleanupIndexData IndexInst 16759912838651900043 Partition 4 Destroy Done 2023-10-21T04:52:16.194+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:16.194+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:16.201+05:30 [Info] switched currmeta from 591 -> 592 force true 2023-10-21T04:52:16.241+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 4186965058349029465 &{0} 2023-10-21T04:52:16.241+05:30 [Info] Indexer::handleDropIndex - IndexInstId 4186965058349029465 2023-10-21T04:52:16.241+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [4186965058349029465] 2023-10-21T04:52:16.241+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [4186965058349029465] 2023-10-21T04:52:16.241+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:16.241+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [4186965058349029465] 2023-10-21T04:52:16.241+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 8, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.242+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 1, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:52:16.242+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:52:16.242+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:52:16.242+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:52:16.242+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex 2023-10-21T04:52:16.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex stopped 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.242+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 2, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:52:16.242+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:52:16.242+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:52:16.242+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:52:16.242+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex 2023-10-21T04:52:16.242+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex stopped 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.242+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.243+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 3, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:52:16.243+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:52:16.243+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:52:16.243+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:52:16.243+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex 2023-10-21T04:52:16.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex stopped 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.243+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 4, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:52:16.243+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:52:16.243+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:52:16.243+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:52:16.243+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex 2023-10-21T04:52:16.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex stopped 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.243+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 5, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.243+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.243+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:52:16.243+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:52:16.244+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:52:16.244+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:52:16.244+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex 2023-10-21T04:52:16.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex stopped 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.244+05:30 [Info] Indexer::handleDropIndex Cleanup Successful for Index Data InstId: 4186965058349029465 Defn: DefnId: 16823504650100619022 Name: idx_partitioned_defer Using: plasma Bucket: bucket_1 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 1 SecExprs: ([`balance`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: true PartitionScheme: KEY HashScheme: CRC32 PartitionKeys: [(meta().`id`)] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_CREATED RState: RebalActive Stream: NIL_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[1:{1 1 [:9123] []} 2:{2 1 [:9123] []} 3:{3 1 [:9123] []} 4:{4 1 [:9123] []} 5:{5 1 [:9123] []} 6:{6 1 [:9123] []} 7:{7 1 [:9123] []} 8:{8 1 [:9123] []}] 8 KEY 0} 2023-10-21T04:52:16.244+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.088251ms 2023-10-21T04:52:16.244+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 3.132087ms 2023-10-21T04:52:16.244+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 6, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:52:16.244+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:52:16.244+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:52:16.244+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:52:16.244+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex 2023-10-21T04:52:16.244+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex stopped 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.244+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.245+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 4186965058349029465, PartitionId 7, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:52:16.245+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:52:16.245+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:52:16.245+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:52:16.245+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex 2023-10-21T04:52:16.245+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex stopped 2023-10-21T04:52:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.245+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.245+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 4186965058349029465 2023-10-21T04:52:16.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:52:16.246+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:52:16.246+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:52:16.246+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:52:16.246+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex 2023-10-21T04:52:16.246+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex stopped 2023-10-21T04:52:16.246+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.246+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.246+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.246+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.267+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 33.784167ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex] 2023-10-21T04:52:16.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex closed 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:52:16.431+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:52:16.431+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:52:16.431+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:52:16.431+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex 2023-10-21T04:52:16.431+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex stopped 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.431+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.444+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex] 2023-10-21T04:52:16.444+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex closed 2023-10-21T04:52:16.444+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:1 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.445+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 1 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex ... 2023-10-21T04:52:16.445+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex ... 2023-10-21T04:52:16.448+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex ... 2023-10-21T04:52:16.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex ... 2023-10-21T04:52:16.456+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex] 2023-10-21T04:52:16.456+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex closed 2023-10-21T04:52:16.457+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.457+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.457+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.457+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:52:16.457+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex] 2023-10-21T04:52:16.458+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex closed 2023-10-21T04:52:16.458+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.458+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.459+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.459+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.459+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:52:16.460+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex] 2023-10-21T04:52:16.460+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex closed 2023-10-21T04:52:16.460+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.461+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.461+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.461+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.461+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:52:16.500+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.508+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex] 2023-10-21T04:52:16.509+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex closed 2023-10-21T04:52:16.509+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.510+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.516+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.516+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.517+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:52:16.521+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 544693394183450628 &{0} 2023-10-21T04:52:16.521+05:30 [Info] Indexer::handleDropIndex - IndexInstId 544693394183450628 2023-10-21T04:52:16.521+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:52:16.522+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:16.522+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 544693394183450628 Name: #primary Keyspace: bucket_1:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 0 2023-10-21T04:52:16.523+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [544693394183450628] 2023-10-21T04:52:16.524+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [544693394183450628] 2023-10-21T04:52:16.524+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:16.524+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [544693394183450628] 2023-10-21T04:52:16.525+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:16.525+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 544693394183450628 Defn: DefnId: 668551233566132751 Name: #primary Using: plasma Bucket: bucket_1 Scope/Id: _system/8 Collection/Id: _query/9 IsPrimary: true NumReplica: 1 InstVersion: 1 SecExprs: ([]) Desc: [] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 1 ReplicaId: 0 PartitionContainer: &{map[0:{0 1 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:16.525+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 4.357052ms 2023-10-21T04:52:16.522+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.526+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_1.index destroyed 2023-10-21T04:52:16.527+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 1, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.527+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 1 Destroy Done 2023-10-21T04:52:16.527+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:52:16.527+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:52:16.527+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:52:16.528+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex 2023-10-21T04:52:16.528+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex stopped 2023-10-21T04:52:16.528+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.528+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.528+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.529+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.529+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 544693394183450628, PartitionId 0, IndexDefnId 668551233566132751. 2023-10-21T04:52:16.529+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.529+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.529+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.529+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex] 2023-10-21T04:52:16.529+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex 2023-10-21T04:52:16.529+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex] 2023-10-21T04:52:16.529+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex] 2023-10-21T04:52:16.529+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex 2023-10-21T04:52:16.530+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex stopped 2023-10-21T04:52:16.530+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.530+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.530+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.530+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.531+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 544693394183450628 2023-10-21T04:52:16.522+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.531+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex] 2023-10-21T04:52:16.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex closed 2023-10-21T04:52:16.532+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.532+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.532+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.532+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.532+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:52:16.533+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:52:16.533+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:52:16.533+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:52:16.533+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex 2023-10-21T04:52:16.533+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex stopped 2023-10-21T04:52:16.533+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.533+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.534+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.535+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:52:16.535+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:52:16.535+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:52:16.535+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex 2023-10-21T04:52:16.535+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex stopped 2023-10-21T04:52:16.535+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.535+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.535+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.538+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.535+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [544693394183450628] 2023-10-21T04:52:16.535+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:52:16.542+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:52:16.542+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:52:16.542+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex 2023-10-21T04:52:16.542+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex stopped 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.543+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:52:16.543+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:52:16.543+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:52:16.543+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex 2023-10-21T04:52:16.543+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex stopped 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.543+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.543+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:52:16.543+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51546 2023-10-21T04:52:16.544+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 17.767516ms 2023-10-21T04:52:16.544+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 22.472322ms 2023-10-21T04:52:16.560+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex] 2023-10-21T04:52:16.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex closed 2023-10-21T04:52:16.561+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.561+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.564+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.565+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.565+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:52:16.565+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:52:16.568+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:52:16.569+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:52:16.569+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex 2023-10-21T04:52:16.569+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex stopped 2023-10-21T04:52:16.570+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.570+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.570+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.571+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.568+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.572+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex] 2023-10-21T04:52:16.573+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex closed 2023-10-21T04:52:16.574+05:30 [Info] bucket_1/idx_partitioned_defer/Mainstore#4186965058349029465:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.575+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.575+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.575+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.575+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:52:16.575+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:52:16.575+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:52:16.575+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:52:16.575+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex 2023-10-21T04:52:16.575+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex stopped 2023-10-21T04:52:16.575+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.575+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.575+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.575+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.584+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex] 2023-10-21T04:52:16.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex closed 2023-10-21T04:52:16.584+05:30 [Info] bucket_1/idx_secondary/Mainstore#17085981505417849194:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.584+05:30 [Info] Indexer::cleanupIndexData IndexInst 544693394183450628 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex ... 2023-10-21T04:52:16.584+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex ... 2023-10-21T04:52:16.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.594+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_#primary_544693394183450628_0.index destroyed 2023-10-21T04:52:16.600+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 544693394183450628, PartitionId 0, IndexDefnId 668551233566132751. 2023-10-21T04:52:16.601+05:30 [Info] Indexer::cleanupIndexData IndexInst 544693394183450628 Partition 0 Destroy Done 2023-10-21T04:52:16.618+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex] 2023-10-21T04:52:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex closed 2023-10-21T04:52:16.618+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:8 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.618+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 8 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex ... 2023-10-21T04:52:16.618+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 27 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex ... 2023-10-21T04:52:16.628+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 161.656911ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:16.673+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9807413941848858, throttleDelayMs (new, old, change): (3, 0, 3) 2023-10-21T04:52:16.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex ... 2023-10-21T04:52:16.674+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex ... 2023-10-21T04:52:16.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.678+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_8.index destroyed 2023-10-21T04:52:16.678+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 8, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.687+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 8 Destroy Done 2023-10-21T04:52:16.681+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.701+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex] 2023-10-21T04:52:16.702+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex closed 2023-10-21T04:52:16.702+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:4 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.702+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 4 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex ... 2023-10-21T04:52:16.703+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex ... 2023-10-21T04:52:16.706+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex] 2023-10-21T04:52:16.707+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex closed 2023-10-21T04:52:16.707+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:3 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.707+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 3 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.706+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.708+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex ... 2023-10-21T04:52:16.709+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex ... 2023-10-21T04:52:16.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex ... 2023-10-21T04:52:16.710+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 30 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex ... 2023-10-21T04:52:16.711+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.714+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.715+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.743+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.745+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.752+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_4.index destroyed 2023-10-21T04:52:16.753+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 4, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.753+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 4 Destroy Done 2023-10-21T04:52:16.753+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex] 2023-10-21T04:52:16.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex closed 2023-10-21T04:52:16.754+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.755+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 5 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.754+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex ... 2023-10-21T04:52:16.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex ... 2023-10-21T04:52:16.755+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex ... 2023-10-21T04:52:16.756+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex ... 2023-10-21T04:52:16.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.760+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.761+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index/docIndex sucessfully destroyed 2023-10-21T04:52:16.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex] 2023-10-21T04:52:16.763+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex closed 2023-10-21T04:52:16.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex] 2023-10-21T04:52:16.764+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex closed 2023-10-21T04:52:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex] 2023-10-21T04:52:16.765+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex closed 2023-10-21T04:52:16.764+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:7 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.765+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:2 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.766+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 2 Close Done 2023-10-21T04:52:16.766+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex ... 2023-10-21T04:52:16.767+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 31 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex ... 2023-10-21T04:52:16.766+05:30 [Info] bucket_1/idx_partitioned_defer/Backstore#4186965058349029465:6 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.766+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 7 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index in /opt/build/ns_server/data/n_3/data/@2i ... Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index in /opt/build/ns_server/data/n_3/data/@2i ... All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_3.index destroyed 2023-10-21T04:52:16.768+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 6 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.768+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex ... 2023-10-21T04:52:16.771+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 26 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex ... 2023-10-21T04:52:16.770+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 3, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.772+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 3 Destroy Done 2023-10-21T04:52:16.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.773+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_5.index destroyed 2023-10-21T04:52:16.773+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 5, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.776+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 5 Destroy Done 2023-10-21T04:52:16.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.781+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex ... 2023-10-21T04:52:16.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 25 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex ... 2023-10-21T04:52:16.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex ... 2023-10-21T04:52:16.782+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 29 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex ... 2023-10-21T04:52:16.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex ... 2023-10-21T04:52:16.786+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 33 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex ... 2023-10-21T04:52:16.790+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:16.796+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:16.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.792+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index/docIndex sucessfully destroyed 2023-10-21T04:52:16.797+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_7.index destroyed 2023-10-21T04:52:16.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex ... 2023-10-21T04:52:16.798+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 28 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex ... 2023-10-21T04:52:16.800+05:30 [Info] switched currmeta from 592 -> 599 force true 2023-10-21T04:52:16.801+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 7, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.802+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 7 Destroy Done 2023-10-21T04:52:16.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.802+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_2.index destroyed 2023-10-21T04:52:16.803+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex ... 2023-10-21T04:52:16.804+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 32 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex ... 2023-10-21T04:52:16.803+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 2, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.805+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 2 Destroy Done 2023-10-21T04:52:16.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.811+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_partitioned_defer_4186965058349029465_6.index destroyed 2023-10-21T04:52:16.812+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 4186965058349029465, PartitionId 6, IndexDefnId 16823504650100619022. 2023-10-21T04:52:16.812+05:30 [Info] Indexer::cleanupIndexData IndexInst 4186965058349029465 Partition 6 Destroy Done 2023-10-21T04:52:16.835+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:52:16.844+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:16.844+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:16.847+05:30 [Info] switched currmeta from 599 -> 599 force true 2023-10-21T04:52:16.882+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:16.882+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:16.898+05:30 [Info] switched currmeta from 599 -> 599 force true 2023-10-21T04:52:16.939+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 1786384967016908289 &{0} 2023-10-21T04:52:16.939+05:30 [Info] Indexer::handleDropIndex - IndexInstId 1786384967016908289 2023-10-21T04:52:16.939+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:16.940+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:16.940+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 1786384967016908289 Name: idx_secondary Keyspace: bucket_%2:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 0 2023-10-21T04:52:16.940+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [1786384967016908289] 2023-10-21T04:52:16.940+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [1786384967016908289] 2023-10-21T04:52:16.940+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:16.940+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [1786384967016908289] 2023-10-21T04:52:16.940+05:30 [Info] MutationMgr::handleRemoveIndexListFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:16.941+05:30 [Info] Timekeeper::handleRemoveIndexFromStream Message: MsgStreamUpdate Type: REMOVE_INDEX_LIST_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: BuildTS: [] IndexList: [ InstId: 1786384967016908289 Defn: DefnId: 12924535004952781813 Name: idx_secondary Using: plasma Bucket: bucket_%2 Scope/Id: _default/0 Collection/Id: c1/c IsPrimary: false NumReplica: 1 InstVersion: 0 SecExprs: ([`age`]) Desc: [false] IndexMissingLeadingKey: false IsPartnKeyDocId: false PartitionScheme: SINGLE HashScheme: CRC32 PartitionKeys: [] WhereExpr: () RetainDeletedXATTR: false AlternateShardIds: map[] State: INDEX_STATE_DELETED RState: RebalActive Stream: MAINT_STREAM Version: 0 ReplicaId: 0 PartitionContainer: &{map[0:{0 0 [:9123] []}] 1 SINGLE 0}] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:16.941+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.502288ms 2023-10-21T04:52:16.941+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 1786384967016908289, PartitionId 0, IndexDefnId 12924535004952781813. 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex] 2023-10-21T04:52:16.941+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex] 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex] 2023-10-21T04:52:16.941+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex 2023-10-21T04:52:16.941+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex stopped 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.941+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.942+05:30 [Info] KVSender::sendDelInstancesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId Instances [1786384967016908289] 2023-10-21T04:52:16.942+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 1786384967016908289 2023-10-21T04:52:16.946+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.946+05:30 [Info] KVSender::sendDelInstancesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceId 2023-10-21T04:52:16.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex] 2023-10-21T04:52:16.947+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73680 2023-10-21T04:52:16.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex closed 2023-10-21T04:52:16.947+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 5.729228ms 2023-10-21T04:52:16.947+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 7.298531ms 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Mainstore#1786384967016908289:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:16.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex] 2023-10-21T04:52:16.947+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex] 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex] 2023-10-21T04:52:16.947+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex 2023-10-21T04:52:16.947+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex stopped 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:16.947+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:16.950+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:16.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex] 2023-10-21T04:52:16.950+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex closed 2023-10-21T04:52:16.951+05:30 [Info] bucket_%2/idx_secondary/Backstore#1786384967016908289:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:16.951+05:30 [Info] Indexer::cleanupIndexData IndexInst 1786384967016908289 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:16.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex ... 2023-10-21T04:52:16.951+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex ... 2023-10-21T04:52:16.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/mainIndex sucessfully destroyed 2023-10-21T04:52:16.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex ... 2023-10-21T04:52:16.962+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex ... 2023-10-21T04:52:16.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : metadata saved successfully 2023-10-21T04:52:16.967+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard11711860383628641160(11711860383628641160)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_idx_secondary_1786384967016908289_0.index destroyed 2023-10-21T04:52:16.972+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 1786384967016908289, PartitionId 0, IndexDefnId 12924535004952781813. 2023-10-21T04:52:16.972+05:30 [Info] Indexer::cleanupIndexData IndexInst 1786384967016908289 Partition 0 Destroy Done 2023-10-21T04:52:16.995+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 63.69507ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:17.059+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:17.059+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:17.061+05:30 [Info] switched currmeta from 599 -> 602 force true 2023-10-21T04:52:17.071+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:17.071+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:17.079+05:30 [Info] switched currmeta from 602 -> 602 force true 2023-10-21T04:52:17.102+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:17.102+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:17.104+05:30 [Info] switched currmeta from 602 -> 603 force true 2023-10-21T04:52:17.286+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 2657603273182005502 &{0} 2023-10-21T04:52:17.286+05:30 [Info] Indexer::handleDropIndex - IndexInstId 2657603273182005502 2023-10-21T04:52:17.286+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:52:17.286+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:17.286+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 2657603273182005502 Name: idx_secondary Keyspace: bucket_1:_default:c1 State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 0 ReplicaId: 1 2023-10-21T04:52:17.286+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [2657603273182005502] 2023-10-21T04:52:17.286+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [2657603273182005502] 2023-10-21T04:52:17.286+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:17.286+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [2657603273182005502] 2023-10-21T04:52:17.286+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:17.286+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:17.287+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:17.287+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 2657603273182005502, PartitionId 0, IndexDefnId 12549405239467949221. 2023-10-21T04:52:17.287+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:17.287+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:17.287+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:17.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex] 2023-10-21T04:52:17.287+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex 2023-10-21T04:52:17.287+05:30 [Info] MutationStreamReader::handleSupervisorCommands Message: MsgUpdateKeyspaceIdQueue KeyspaceIdQueueMap: map[bucket_%2:{0xc0006943c0 }] 2023-10-21T04:52:17.287+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex] 2023-10-21T04:52:17.287+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex] 2023-10-21T04:52:17.287+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex 2023-10-21T04:52:17.287+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex stopped 2023-10-21T04:52:17.287+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:17.287+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:17.287+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:17.287+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:17.287+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_1 BuildTS: [] IndexList: [] Async: false SessionId: 51546 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:17.287+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_1 2023-10-21T04:52:17.287+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_1 Deleted from Stream MAINT_STREAM 2023-10-21T04:52:17.287+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:52:17.287+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:17.287+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:17.287+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 1.541818ms 2023-10-21T04:52:17.287+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 2657603273182005502 2023-10-21T04:52:17.288+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1] 2023-10-21T04:52:17.290+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:17.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex] 2023-10-21T04:52:17.290+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex closed 2023-10-21T04:52:17.291+05:30 [Info] bucket_1/idx_secondary/Mainstore#2657603273182005502:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:17.291+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:17.291+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:17.299+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:17.301+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex] 2023-10-21T04:52:17.301+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex 2023-10-21T04:52:17.302+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex] 2023-10-21T04:52:17.292+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 10 27353924173936 22 2023-10-21T04:52:17.302+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.302+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 0 67576817345502 18 2023-10-21T04:52:17.302+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.303+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 1 44231811555836 33 2023-10-21T04:52:17.303+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.303+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 2 197901620499212 23 2023-10-21T04:52:17.303+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.303+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 3 39196762516158 19 2023-10-21T04:52:17.303+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.303+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 4 95570275214302 23 2023-10-21T04:52:17.304+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.304+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 5 95215784105904 25 2023-10-21T04:52:17.304+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.304+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 6 143637217187246 23 2023-10-21T04:52:17.304+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.305+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 7 4065155603859 28 2023-10-21T04:52:17.305+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.305+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 8 91960699361891 21 2023-10-21T04:52:17.305+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.305+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 9 147449900650979 28 2023-10-21T04:52:17.305+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.305+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 11 77783296532664 25 2023-10-21T04:52:17.305+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 12 233597919143467 23 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 25 232497361012987 18 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 22 215732063158788 18 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 16 33320884322178 37 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 18 132000914750455 21 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 21 7646518130509 24 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 19 79253753007122 21 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 20 101186302195296 32 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 23 230675090583633 23 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 13 198938078557080 24 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 14 122353375037127 21 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 15 129413110409107 25 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 17 25078414014136 18 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 24 113904165092305 21 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 43 270106388647522 25 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 26 201780742117921 20 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 37 63194637348236 27 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.306+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 40 186283152043223 18 2023-10-21T04:52:17.306+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 41 277109307527403 24 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 27 131315346703752 21 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 38 13155294694475 25 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 39 119373194724283 26 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 42 209574230498887 23 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 28 61904892722472 19 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 32 19399479457331 23 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 33 258788849496338 22 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 36 280972182784873 24 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 44 255609313786736 23 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 45 49352389572227 29 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 29 241043521322196 24 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 30 248212138718363 18 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 31 201745596055144 26 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 34 190878005091193 21 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.307+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 35 173020619068939 21 2023-10-21T04:52:17.307+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.308+05:30 [Info] KVSender::sendDelKeyspacesRequest Success Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_1] 2023-10-21T04:52:17.308+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_1 SessionId 51546 2023-10-21T04:52:17.308+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 20.652097ms 2023-10-21T04:52:17.308+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 22.267483ms 2023-10-21T04:52:17.308+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 47 62156602796507 28 2023-10-21T04:52:17.308+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.308+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 55 65071891036109 27 2023-10-21T04:52:17.308+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.308+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 53 212804514607186 30 2023-10-21T04:52:17.308+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.308+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 54 102294135990395 22 2023-10-21T04:52:17.308+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.308+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 46 83754344146634 23 2023-10-21T04:52:17.308+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 48 80989250905985 20 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 49 279208201382660 23 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 50 193080054939721 22 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 51 204579891639872 27 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 52 150369066093368 18 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 61 257224089448052 24 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 62 184273998690001 17 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 63 202830985870043 24 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 56 152904045160406 26 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 57 218103594015078 27 2023-10-21T04:52:17.309+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.309+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 58 35015089382353 25 2023-10-21T04:52:17.310+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.310+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 59 90104317736129 21 2023-10-21T04:52:17.310+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.310+05:30 [Info] TK StreamEnd MAINT_STREAM bucket_1 60 223555064174984 24 2023-10-21T04:52:17.310+05:30 [Warn] Timekeeper::handleStreamEnd Received StreamEnd for Inactive KeyspaceId bucket_1 Stream MAINT_STREAM. Ignored. 2023-10-21T04:52:17.316+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 38.976599ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:17.329+05:30 [Info] bucket_1_c25fee48adbe357b2d711be58a1deb67: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex] 2023-10-21T04:52:17.329+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex 2023-10-21T04:52:17.329+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex stopped 2023-10-21T04:52:17.329+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:17.329+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:17.329+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:17.329+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:17.332+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:17.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex] 2023-10-21T04:52:17.332+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex closed 2023-10-21T04:52:17.333+05:30 [Info] controller 'bucket_1_c25fee48adbe357b2d711be58a1deb67/bucket_1_c25fee48adbe357b2d711be58a1deb67': swap is disabled 2023-10-21T04:52:17.333+05:30 [Info] bucket_1/idx_secondary/Backstore#2657603273182005502:0 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:17.333+05:30 [Info] Indexer::cleanupIndexData IndexInst 2657603273182005502 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:17.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex ... 2023-10-21T04:52:17.333+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : removed plasmaId 35 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex ... 2023-10-21T04:52:17.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : metadata saved successfully 2023-10-21T04:52:17.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard17327901024118572215(17327901024118572215)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/mainIndex sucessfully destroyed 2023-10-21T04:52:17.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex ... 2023-10-21T04:52:17.336+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : removed plasmaId 34 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex ... 2023-10-21T04:52:17.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : metadata saved successfully 2023-10-21T04:52:17.338+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard5629943257627822009(5629943257627822009)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index/docIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_1_idx_secondary_2657603273182005502_0.index destroyed 2023-10-21T04:52:17.338+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 2657603273182005502, PartitionId 0, IndexDefnId 12549405239467949221. 2023-10-21T04:52:17.338+05:30 [Info] Indexer::cleanupIndexData IndexInst 2657603273182005502 Partition 0 Destroy Done 2023-10-21T04:52:17.363+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:17.363+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:17.415+05:30 [Info] switched currmeta from 603 -> 607 force true 2023-10-21T04:52:17.437+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:17.437+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:17.441+05:30 [Info] switched currmeta from 607 -> 607 force true 2023-10-21T04:52:17.557+05:30 [Info] clustMgrAgent::OnIndexDelete Notification Received for Drop IndexId 17465522551378543576 &{0} 2023-10-21T04:52:17.557+05:30 [Info] Indexer::handleDropIndex - IndexInstId 17465522551378543576 2023-10-21T04:52:17.557+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:52:17.557+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:17.557+05:30 [Info] ClustMgr::handleIndexMap, updated instances: InstanceId: 17465522551378543576 Name: #primary Keyspace: bucket_%2:_system:_query State: INDEX_STATE_DELETED Stream: MAINT_STREAM RState: RebalActive Version: 1 ReplicaId: 1 2023-10-21T04:52:17.558+05:30 [Info] MutationMgr::handleUpdateIndexInstMap, deleted instance ids: [17465522551378543576] 2023-10-21T04:52:17.558+05:30 [Info] MutationMgr::handleUpdateIndexPartnMap, deleted instance ids: [17465522551378543576] 2023-10-21T04:52:17.558+05:30 [Info] ScanCoordinator::initialize rollback times on new index inst map: map[bucket_%2:1697843682957894080 bucket_1:1697843900235320103] 2023-10-21T04:52:17.558+05:30 [Info] ClustMgr::handleIndexMap, deleted instance id's: [17465522551378543576] 2023-10-21T04:52:17.558+05:30 [Info] MutationStreamReader::handleUpdateKeyspaceStatsMap 2023-10-21T04:52:17.558+05:30 [Info] MutationMgr::handleUpdateKeyspaceStatsMap Stream MAINT_STREAM Succceed 2023-10-21T04:52:17.558+05:30 [Info] MutationMgr::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:17.558+05:30 [Info] MutationStreamReader:Shutdown StreamReader MAINT_STREAM 2023-10-21T04:52:17.558+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 total received key-versions: map[0:0 1:1 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:1 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:1 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:1 59:1 60:0 61:0 62:0 63:0] 2023-10-21T04:52:17.558+05:30 [Info] DATP[->dataport ":9123"] keyspaceId: bucket_%2 latest sequence numbers: map[0:0 1:33 2:0 3:0 4:0 5:0 6:0 7:0 8:0 9:0 10:0 11:0 12:0 13:0 14:0 15:0 16:26 17:0 18:0 19:0 20:0 21:0 22:0 23:0 24:0 25:0 26:0 27:0 28:0 29:0 30:0 31:0 32:0 33:0 34:0 35:0 36:0 37:0 38:0 39:0 40:0 41:0 42:0 43:0 44:0 45:41 46:0 47:0 48:0 49:0 50:0 51:0 52:0 53:0 54:0 55:0 56:0 57:0 58:27 59:18 60:0 61:0 62:0 63:0] 2023-10-21T04:52:17.558+05:30 [Error] DATP[->dataport ":9123"] Accept() Error: accept tcp4 0.0.0.0:9123: use of closed network connection 2023-10-21T04:52:17.558+05:30 [Error] receiving packet: read tcp4 127.0.0.1:9123->127.0.0.1:59130: use of closed network connection 2023-10-21T04:52:17.559+05:30 [Error] DATP[->dataport ":9123"] worker "127.0.0.1:59130" exit: read tcp4 127.0.0.1:9123->127.0.0.1:59130: use of closed network connection 2023-10-21T04:52:17.559+05:30 [Info] DATP[->dataport ":9123"] connection "127.0.0.1:59130" closed ! 2023-10-21T04:52:17.559+05:30 [Info] DATP[->dataport ":9123"] ... stopped 2023-10-21T04:52:17.559+05:30 [Error] DATP[->dataport ":9123"] fatal remote "127.0.0.1:59130" already gone 2023-10-21T04:52:17.560+05:30 [Info] plasmaSlice::Close Closed Slice Id 0, IndexInstId 17465522551378543576, PartitionId 0, IndexDefnId 5277316367626246544. 2023-10-21T04:52:17.560+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Begin closing plasma instance 2023-10-21T04:52:17.560+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Received stop message 2023-10-21T04:52:17.560+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.monitor: Stopped monitor 2023-10-21T04:52:17.560+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.StopInstance: Stopping instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex] 2023-10-21T04:52:17.560+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.unregisterInstance: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex 2023-10-21T04:52:17.560+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Begin removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex] 2023-10-21T04:52:17.560+05:30 [Info] Timekeeper::handleRemoveKeyspaceFromStream Message: MsgStreamUpdate Type: REMOVE_KEYSPACE_FROM_STREAM Stream: MAINT_STREAM KeyspaceId: bucket_%2 BuildTS: [] IndexList: [] Async: false SessionId: 73680 CollectionId: CollectionAware: false EnableOSO: false RestartTs: 2023-10-21T04:52:17.560+05:30 [Info] Timekeeper::stopTimer MAINT_STREAM bucket_%2 2023-10-21T04:52:17.560+05:30 [Info] StreamState::cleanupKeyspaceIdFromStream KeyspaceId bucket_%2 Deleted from Stream MAINT_STREAM 2023-10-21T04:52:17.560+05:30 [Info] Timekeeper::resetTsQueueStats Stream MAINT_STREAM CurrInit 0 CurrMaint 0 New 0 2023-10-21T04:52:17.560+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from internalAdminRecvCh channel processing took 3.21861ms 2023-10-21T04:52:17.560+05:30 [Info] clustMgrAgent::OnIndexDelete Success for Drop IndexId 17465522551378543576 2023-10-21T04:52:17.561+05:30 [Info] KVSender::sendDelKeyspacesRequest Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2] 2023-10-21T04:52:17.583+05:30 [Error] KVSender::sendDelKeyspacesRequest Unexpected Error During Del Buckets Request Projector 172.31.5.112:10000 Topic MAINT_STREAM_TOPIC_4325b30e7f9620a998388078247335a8 KeyspaceIds [bucket_%2]. Err genServer.closed 2023-10-21T04:52:17.583+05:30 [Error] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Error Received genServer.closed from 172.31.5.112:10000 2023-10-21T04:52:17.583+05:30 [Info] KVSender::deleteKeyspacesFromStream MAINT_STREAM [bucket_%2] Treating genServer.closed As Success 2023-10-21T04:52:17.583+05:30 [Info] Indexer::removeIndexesFromStream Success Stream MAINT_STREAM KeyspaceId bucket_%2 SessionId 73680 2023-10-21T04:52:17.583+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel stream request lock waits took 22.888473ms 2023-10-21T04:52:17.583+05:30 [Info] Indexer::listenAdminMsgs:msg_loop: CLUST_MGR_DROP_INDEX_DDL message from adminRecvCh channel processing took 26.174705ms 2023-10-21T04:52:17.594+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_DROP_INDEX elapsed 46.398974ms len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:17.629+05:30 [Info] bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b: swapperDaemon.removeInstance: Done removing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex] 2023-10-21T04:52:17.629+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': Controller.removeFromSwapper: /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex 2023-10-21T04:52:17.629+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex stopped 2023-10-21T04:52:17.630+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Received stop message 2023-10-21T04:52:17.630+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.mvccPurgerDaemon: Stopped MVCC Purger 2023-10-21T04:52:17.630+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.holeCleaner: Received stop message 2023-10-21T04:52:17.630+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Begin persist all page, concurrency [4] 2023-10-21T04:52:17.636+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close(): Done persist all page, concurrency [4] 2023-10-21T04:52:17.636+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: Closing instance [/opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex] 2023-10-21T04:52:17.637+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : Shard.closeInstance: instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex closed 2023-10-21T04:52:17.637+05:30 [Info] controller 'bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b/bucket_%2_d5fd9a12d20c726b6a3f9002eb18d52b': swap is disabled 2023-10-21T04:52:17.637+05:30 [Info] bucket_%2/idx_partitioned/Mainstore#12606056968936758026:5 Plasma: Plasma.Close: Done closing plasma instance 2023-10-21T04:52:17.637+05:30 [Info] Indexer::cleanupIndexData IndexInst 17465522551378543576 Partition 0 Close Done Destory instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index in /opt/build/ns_server/data/n_3/data/@2i ... 2023-10-21T04:52:17.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : destroying instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex ... 2023-10-21T04:52:17.638+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : removed plasmaId 2 for instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex ... 2023-10-21T04:52:17.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : metadata saved successfully 2023-10-21T04:52:17.642+05:30 [Info] Shard /opt/build/ns_server/data/n_3/data/@2i/shards/shard10838601320854145501(10838601320854145501)(0-0-0) : instance /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index/mainIndex sucessfully destroyed All instances matching prefix /opt/build/ns_server/data/n_3/data/@2i/bucket_%2_#primary_17465522551378543576_0.index destroyed 2023-10-21T04:52:17.642+05:30 [Info] plasmaSlice::Destroy Cleaned Up Slice Id 0, IndexInstId 17465522551378543576, PartitionId 0, IndexDefnId 5277316367626246544. 2023-10-21T04:52:17.644+05:30 [Info] Indexer::cleanupIndexData IndexInst 17465522551378543576 Partition 0 Destroy Done 2023-10-21T04:52:17.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 1, throttleDelayMs (new, old, change): (8, 3, 5) 2023-10-21T04:52:17.828+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:17.828+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:17.841+05:30 [Info] switched currmeta from 607 -> 611 force true 2023-10-21T04:52:17.919+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:17.919+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:17.930+05:30 [Info] switched currmeta from 611 -> 611 force true 2023-10-21T04:52:18.436+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:52:18.583+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:18.583+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:18.600+05:30 [Info] switched currmeta from 611 -> 611 force true 2023-10-21T04:52:18.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9980086567489177, throttleDelayMs (new, old, change): (10, 8, 2) 2023-10-21T04:52:18.680+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:52:18.734+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:18.734+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:18.758+05:30 [Info] switched currmeta from 611 -> 611 force true 2023-10-21T04:52:18.791+05:30 DEBU REGU.utils.(*ClusterClient).processTasksLOCKED.func1() at nsserver.go:1010 [id 215] updated cluster rebalancing status to true 2023-10-21T04:52:18.791+05:30 DEBU REGU.impl.NewStorageLimitingService.func3() at storagelimiting.go:128 [id 215] updating rebalancing status to true 2023-10-21T04:52:19.433+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:52:19.535+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:19.539+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:19.548+05:30 [Info] switched currmeta from 611 -> 611 force true 2023-10-21T04:52:19.666+05:30 [Warn] CpuThrottle::adjustThrottleDelay: throttleDelayMs is at maximum 10. cpuTarget: 0.95, currCpu: 0.9634583809207254 2023-10-21T04:52:20.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.9103007722585024, throttleDelayMs (new, old, change): (6, 10, -4) 2023-10-21T04:52:20.944+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange {e0336673e5811df4487169484bd60532 [] topology-change-rebalance [] [{d7ff2bfd429b21d9c24d822bc13e3acc 7060000 } {9b27839d59b06374125d80a2b12a21b6 7060000 } {4325b30e7f9620a998388078247335a8 7060000 } {ba6304988a6335d65281381d6cfc5e32 7060000 }]} 2023-10-21T04:52:20.950+05:30 [Info] Indexer::run:msg_loop: INDEXER_CHECK_DDL_IN_PROGRESS message from wrkrPrioRecvCh channel processing took 5.279µs 2023-10-21T04:52:20.950+05:30 [Info] RebalanceServiceManager::prepareRebalance: Found DDL Running [] 2023-10-21T04:52:20.950+05:30 [Info] RebalanceServiceManager::prepareRebalance: Init Prepare Phase 2023-10-21T04:52:20.950+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceRunning Value 2023-10-21T04:52:20.952+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:52:20.952+05:30 [Info] ClustMgr:handleRebalanceRunning mType: CLUST_MGR_REBALANCE_RUNNING indexList: [] bucket: scope: collection: streamId: NIL_STREAM syncUpdate: false respCh: 2023-10-21T04:52:20.952+05:30 [Info] Indexer::run:msg_loop: CLUST_MGR_SET_LOCAL message from wrkrPrioRecvCh channel processing took 1.938117ms 2023-10-21T04:52:20.952+05:30 [Info] schedIndexCreator::stopProcessDDL done 2023-10-21T04:52:20.952+05:30 [Info] RebalanceServiceManager::PrepareTopologyChange Success. isBalanced false 2023-10-21T04:52:20.952+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 27] Nodes:[4325b30e7f9620a998388078247335a8] IsBalanced:true Messages:[]} 2023-10-21T04:52:20.953+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_RUNNING elapsed 6.183µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:20.953+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 26] call. taskList: &{Rev:[0 0 0 0 0 0 0 27] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/e0336673e5811df4487169484bd60532 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:e0336673e5811df4487169484bd60532]}]} 2023-10-21T04:52:20.954+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 27] 2023-10-21T04:52:20.954+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 27] 2023-10-21T04:52:21.132+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:52:21.137+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:21.137+05:30 [Info] Refreshed Indexer List: [127.0.0.1:9106 127.0.0.1:9112 127.0.0.1:9118 127.0.0.1:9124] 2023-10-21T04:52:21.138+05:30 [Info] switched currmeta from 611 -> 611 force true 2023-10-21T04:52:21.665+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6923348062788993, throttleDelayMs (new, old, change): (1, 6, -5) 2023-10-21T04:52:22.040+05:30 [Info] RebalanceServiceManager::handleRegisterRebalanceToken: New Rebalance Token {d7ff2bfd429b21d9c24d822bc13e3acc e0336673e5811df4487169484bd60532 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:52:22.045+05:30 [Info] RebalanceServiceManager::observeGlobalRebalanceToken Global And Local Rebalance Token Match {d7ff2bfd429b21d9c24d822bc13e3acc e0336673e5811df4487169484bd60532 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:52:22.045+05:30 [Info] ClustMgr:handleSetLocalValue Key RebalanceToken Value {"MasterId":"d7ff2bfd429b21d9c24d822bc13e3acc","RebalId":"e0336673e5811df4487169484bd60532","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:52:22.050+05:30 [Info] RebalanceServiceManager::registerLocalRebalanceToken: Registered Rebalance Token In Local Meta &{d7ff2bfd429b21d9c24d822bc13e3acc e0336673e5811df4487169484bd60532 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} 2023-10-21T04:52:22.050+05:30 [Info] NewShardRebalancer nodeId 4325b30e7f9620a998388078247335a8 rebalToken &{d7ff2bfd429b21d9c24d822bc13e3acc e0336673e5811df4487169484bd60532 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master false localaddr 127.0.0.1:9120 runPlanner false runParam &{false []} clusterVersion 9 2023-10-21T04:52:22.050+05:30 [Info] ShardRebalancer::observeRebalance &{d7ff2bfd429b21d9c24d822bc13e3acc e0336673e5811df4487169484bd60532 Rebalance 127.0.0.1 0 REBALANCE_NOT_RUNNING SHARD} master:false 2023-10-21T04:52:22.054+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken {"MasterId":"d7ff2bfd429b21d9c24d822bc13e3acc","RebalId":"e0336673e5811df4487169484bd60532","Source":0,"Error":"","MasterIP":"127.0.0.1","Version":0,"RebalPhase":0,"ActiveRebalancer":2} 2023-10-21T04:52:22.054+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token updated with AllowDDLDuringRebalance 2023-10-21T04:52:22.158+05:30 [Info] DDLServiceMgr::handleListCreateTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33066 2023-10-21T04:52:22.168+05:30 [Info] DDLServiceMgr::handleListGenericTokenPaths Processing Request caller: handleListDeleteTokenPaths, req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33066 2023-10-21T04:52:22.222+05:30 [Info] DDLServiceMgr::handleListDropInstanceTokens Processing Request req: Method GET, Host 127.0.0.1:9120, ContentLength 0, UserAgent Go-http-client/1.1, RemoteAddr 127.0.0.1:33066 2023-10-21T04:52:22.319+05:30 [Info] ShardRebalancer::processShardTokens RebalanceToken /indexing/rebalance/RebalanceToken 2023-10-21T04:52:22.319+05:30 [Info] ShardRebalancer::processShardTokens Rebalance Token Deleted. Mark Done. 2023-10-21T04:52:22.319+05:30 [Info] ShardRebalancer::doFinish cleanup started 2023-10-21T04:52:22.319+05:30 [Info] ShardRebalancer::doFinish Cleanup: 2023-10-21T04:52:22.319+05:30 [Info] RebalanceServiceManager::runCleanupPhase path /indexing/rebalance/RebalanceToken isMaster false 2023-10-21T04:52:22.319+05:30 [Info] ShardRebalancer::processDropShards Done Received 2023-10-21T04:52:22.323+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Initiating restore and shard unlock. Skip shards: map[] 2023-10-21T04:52:22.323+05:30 [Info] StorageMgr::ClearRebalanceRunning Clearing rebalance flags for all slices with shardIds: map[] 2023-10-21T04:52:22.323+05:30 [Info] StorageMgr::ClearRebalanceRunning Done with clearing rebalance flags for all slices. elapsed: 1.599µs 2023-10-21T04:52:22.323+05:30 [Info] RebalanceServiceManager::RestoreAndUnlockShards Exiting 2023-10-21T04:52:22.323+05:30 [Info] RebalanceServiceManager::cleanupLocalRToken Cleanup 2023-10-21T04:52:22.324+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceToken 2023-10-21T04:52:22.326+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:52:22.326+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:52:22.326+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:52:22.326+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:52:22.326+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 38.566µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:22.329+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:52:22.329+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:52:22.329+05:30 [Info] RebalanceServiceManager::onRebalanceDoneLOCKED Rebalance Done: isBalanced true, isMaster false, forceUnbalanced false, err: 2023-10-21T04:52:22.329+05:30 [Info] ShardRebalancer::observeRebalance exiting err 2023-10-21T04:52:22.345+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:52:22.345+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 27] Tasks:[{Rev:[0 0 0 0 0 0 0 0] ID:prepare/e0336673e5811df4487169484bd60532 Type:task-prepared Status:task-running IsCancelable:true Progress:0 DetailedProgress:map[] Description: ErrorMessage: Extra:map[rebalanceId:e0336673e5811df4487169484bd60532]}]} 2023-10-21T04:52:22.352+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:52:22.352+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 27] Nodes:[4325b30e7f9620a998388078247335a8] IsBalanced:true Messages:[]} 2023-10-21T04:52:22.354+05:30 [Info] GenericServiceManager::CancelTask: called with id: prepare/e0336673e5811df4487169484bd60532, rev: [] 2023-10-21T04:52:22.354+05:30 [Info] RebalanceServiceManager::RebalCancelTask: called. id: prepare/e0336673e5811df4487169484bd60532, rev: [] 2023-10-21T04:52:22.354+05:30 [Info] RebalanceServiceManager::cleanupRebalanceRunning Cleanup 2023-10-21T04:52:22.354+05:30 [Info] ClustMgr:handleDelLocalValue Key RebalanceRunning 2023-10-21T04:52:22.354+05:30 [Info] IndexManager.RebalanceDone(): making request for rebalance done 2023-10-21T04:52:22.354+05:30 [Info] LifecycleMgr::handleRebalanceDone Clearing rebalance phase 2023-10-21T04:52:22.354+05:30 [Info] lifecycleMgr.dispatchRequest: op OPCODE_REBALANCE_DONE elapsed 55.373µs len(expediates) 0 len(incomings) 0 len(outgoings) 0 len(parallels) 0 error 2023-10-21T04:52:22.360+05:30 [Info] Indexer:clearRebalancePhase Clearing book-keeping on rebalance done 2023-10-21T04:52:22.360+05:30 [Info] schedIndexCreator::startProcessDDL done 2023-10-21T04:52:22.360+05:30 [Info] GenericServiceManager::CancelTask: return from id: prepare/e0336673e5811df4487169484bd60532, rev: [] call. err: 2023-10-21T04:52:22.360+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [0 0 0 0 0 0 0 27] call. taskList: &{Rev:[0 0 0 0 0 0 0 28] Tasks:[]} 2023-10-21T04:52:22.360+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 28] Nodes:[4325b30e7f9620a998388078247335a8] IsBalanced:true Messages:[]} 2023-10-21T04:52:22.374+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [] 2023-10-21T04:52:22.374+05:30 [Info] GenericServiceManager::GetTaskList: return from rev [] call. taskList: &{Rev:[0 0 0 0 0 0 0 28] Tasks:[]} 2023-10-21T04:52:22.376+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [] 2023-10-21T04:52:22.376+05:30 [Info] RebalanceServiceManager::GetCurrentTopology returns &{Rev:[0 0 0 0 0 0 0 28] Nodes:[4325b30e7f9620a998388078247335a8] IsBalanced:true Messages:[]} 2023-10-21T04:52:22.377+05:30 [Info] GenericServiceManager::GetTaskList: called with rev: [0 0 0 0 0 0 0 28] 2023-10-21T04:52:22.383+05:30 [Info] RebalanceServiceManager::GetCurrentTopology [0 0 0 0 0 0 0 28] 2023-10-21T04:52:22.503+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:52:22.551+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:47964. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:22.551+05:30 [Info] messageListener.start(): message channel closed. Remove peer 3a:29:5b:5c:44:b6:e7:7c as follower. 2023-10-21T04:52:22.556+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:59636 closed 2023-10-21T04:52:22.593+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:47970. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:22.594+05:30 [Info] messageListener.start(): message channel closed. Remove peer c6:5a:9d:0:dd:d4:1f:d4 as follower. 2023-10-21T04:52:22.607+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:47968. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:22.608+05:30 [Info] messageListener.start(): message channel closed. Remove peer 28:17:e4:a8:a6:18:3a:74 as follower. 2023-10-21T04:52:22.617+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:59616 closed 2023-10-21T04:52:22.625+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:48022. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:22.625+05:30 [Info] messageListener.start(): message channel closed. Remove peer 7f:9f:9:d3:a5:1a:bf:ab as follower. 2023-10-21T04:52:22.632+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:59624 closed 2023-10-21T04:52:22.645+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:59650 closed 2023-10-21T04:52:22.666+05:30 [Info] CpuThrottle::adjustThrottleDelay: Adjusted throttle. cpuTarget: 0.95, currCpu: 0.6093286021944007, throttleDelayMs (new, old, change): (0, 1, -1) 2023-10-21T04:52:22.668+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:22.668+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): indexer d7ff2bfd429b21d9c24d822bc13e3acc 2023-10-21T04:52:22.668+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9106. 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): finish for indexer d7ff2bfd429b21d9c24d822bc13e3acc 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): indexer 9b27839d59b06374125d80a2b12a21b6 2023-10-21T04:52:22.668+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9112. 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): finish for indexer 9b27839d59b06374125d80a2b12a21b6 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): indexer 4325b30e7f9620a998388078247335a8 2023-10-21T04:52:22.668+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9118. 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): finish for indexer 4325b30e7f9620a998388078247335a8 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): indexer ba6304988a6335d65281381d6cfc5e32 2023-10-21T04:52:22.668+05:30 [Info] Unwatching metadata for indexer at 127.0.0.1:9124. 2023-10-21T04:52:22.668+05:30 [Info] UnwatchMetadata(): finish for indexer ba6304988a6335d65281381d6cfc5e32 2023-10-21T04:52:22.668+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9106. Reason = read tcp 127.0.0.1:39180->127.0.0.1:9106: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:22.669+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9112. Reason = read tcp 127.0.0.1:39700->127.0.0.1:9112: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:22.669+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9118. Reason = read tcp 127.0.0.1:48062->127.0.0.1:9118: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:22.669+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:9124. Reason = read tcp 127.0.0.1:44440->127.0.0.1:9124: use of closed network connection. Cleaning Pipe. 2023-10-21T04:52:22.669+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:48062. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:22.669+05:30 [Info] messageListener.start(): message channel closed. Remove peer 69:a:a2:6b:9b:ee:e3:7e as follower. 2023-10-21T04:52:22.672+05:30 [Info] switched currmeta from 611 -> 615 force true 2023-10-21T04:52:22.672+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:52:22.672+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] ... stopped 2023-10-21T04:52:22.672+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:52:22.673+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] ... stopped 2023-10-21T04:52:22.673+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:59652 closed 2023-10-21T04:52:22.673+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:52:22.674+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] ... stopped 2023-10-21T04:52:22.674+05:30 [Info] activeConnectionsUpdateChannel channel closed 2023-10-21T04:52:22.674+05:30 [Info] [Queryport-connpool:127.0.0.1:9119] Stopping releaseConnsRoutine 2023-10-21T04:52:22.674+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] ... stopped 2023-10-21T04:52:22.692+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:47950. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:22.692+05:30 [Info] messageListener.start(): message channel closed. Remove peer 2b:8c:e7:c7:10:2c:1b:19 as follower. 2023-10-21T04:52:22.696+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification 2023-10-21T04:52:22.713+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:59596 closed 2023-10-21T04:52:22.750+05:30 [Info] [Queryport-connpool:127.0.0.1:9125] Stopping releaseConnsRoutine 2023-10-21T04:52:22.753+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:22.753+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:52:22.764+05:30 [Info] switched currmeta from 615 -> 615 force true 2023-10-21T04:52:22.785+05:30 [Info] PeerPipe.doRecieve() : connection terminated for 127.0.0.1:47986. Reason = EOF. Cleaning Pipe. 2023-10-21T04:52:22.785+05:30 [Info] messageListener.start(): message channel closed. Remove peer 8a:ca:a:5e:d6:c1:a1:1f as follower. 2023-10-21T04:52:22.800+05:30 [Info] [Queryport ":9119"] connection 127.0.0.1:59630 closed 2023-10-21T04:52:23.010+05:30 [Info] serviceChangeNotifier: received ServiceChangeNotification 2023-10-21T04:52:23.118+05:30 [Info] MetadataProvider.SetClusterStatus(): healthy nodes 3 failed node 0 unhealthy node 0 add node 0 2023-10-21T04:52:23.118+05:30 [Info] Refreshing indexer list due to cluster changes or auto-refresh. 2023-10-21T04:52:23.118+05:30 [Info] Refreshed Indexer List: [] 2023-10-21T04:52:23.120+05:30 [Info] switched currmeta from 615 -> 615 force true 2023-10-21T04:52:23.503+05:30 [Info] [Queryport-connpool:127.0.0.1:9107] Stopping releaseConnsRoutine 2023-10-21T04:52:23.503+05:30 [Info] [Queryport-connpool:127.0.0.1:9113] Stopping releaseConnsRoutine 2023-10-21T04:52:23.561+05:30 [Info] serviceChangeNotifier: received PoolChangeNotification